An unsupervised learning framework for discovering the site-specific ontology from multiple Web pages

We develop an unsupervised learning framework for tackling the problem of automatic site-specific ontology discovery from multiple pages of a Web site. To harness the uncertainty involved, our framework is designed based on a generative model which models the generation of text fragments contained in the pages of a Web site. One characteristic of our framework is that we consider clues from multiple pages collected from the Web site. Another characteristic is that we learn the regularities of the layout format to discover the site-specific ontology via stochastic grammatical inference. To accomplish the goal of ontology discovery, the ontology information blocks of a Web page are identified by making use of the site invariant information. We have conducted extensive experiments using real-world Web sites. Comparisons between existing methods and our framework have been carried out to demonstrate the effectiveness of our framework.