On-Line Learning of Predictive Compositional Hierarchies
          
                                Karl Pfleger
                       Department of Computer Science
                             Stanford University

Language, music, spatial configurations, action sequences, and many other
types of data exhibit hierarchical compositional structure. Compositional (or
part-whole) relationships, like taxonomic (is-a) relationships, also serve as
critical components of representation in AI, and existing work demonstrates
that hand-built compositional hierarchies (CHs) can make useful predictive
inferences. However, unlike taxonomies, for which numerous basic learning
algorithms exist, there has not been analogous foundational work on learning
predictive CHs. My research demonstrates that predictive CHs can also be
learned in an unsupervised, on-line fashion purely from primitive data. The
essence of on-line CH learning is the bottom-up identification of frequently
occurring repeated patterns in data, which enables the future discovery of
even larger patterns. I introduce two new learning systems, one based on
Boltzmann machine and one based on n-grams, both capable of composing larger
and larger patterns (or chunks) as they see more data, with no prespecified
bound but nonetheless using less storage space than that taken by the data
itself. This hierarchical process has the potential to scale automatically
from fine-grained, low-level data to coarser, high-level representations
tuned to the statistical characteristics of the environment, thereby bridging
a gap that has proved to be one of the largest stumbling blocks on the way to
creating significantly more complex and intelligent autonomous agents.