Operationalising “Websites”: lexically, semantically or topologically?

Abstract

Methods to investigate the structure of the Web graph in order to better understand its properties are of interest to many researchers. The scale and complexity of the Web-page digraph is typically managed by aggregating together or clustering individual Web-pages in order to form “Websites”. It is the properties of these Websites which then become the focus of research. The most popular Web-page clustering technique is "lexical" and uses the url syntax in order to assign Web-pages to “Websites”. Semantic clustering, that is clustering Web-pages according to the similarity of their content has also been proposed. In this paper we consider a third approach to Web-page clustering which is based on the topological properties of the Web-page within the Web-page digraph. We present the technique and report the results of an experiment to compare the use of url-lexically and topologically determined Websites in two sub-domains, one within the Spanish country level domain and the other within the UK country level domain of the Web