Please use this identifier to cite or link to this item: http://hdl.handle.net/1822/54562

TitleWeb crawler profiling and containment through navigation pattern mining
Author(s)Lourenço, Anália Maria Garcia
Belo, Orlando
KeywordsClickstream Processing
Crawling Profiling
Data Webhousing
Navigation Patterns
Web Usage Mining
Issue date1-Nov-2009
PublisherInternational Association for Development of the Information Society (IADIS)
Abstract(s)Web profiles may support the analysis of Web site popularity as well as the detection of unwanted and illegitimate activities such as fraud. Yet, profiling techniques often fail to account for different usage, processing regular sessions, crawler sessions and proxy sessions in a similar way. This paper proposes an integrated approach to Web crawler profiling and containment. A data Webhousing embracing standard crawler detection techniques supplies the profiles to be further analysed through navigation pattern mining. The ability to adapt crawler identification to particular Web scenarios, the incremental analysis of navigation patterns, and the capacity of monitoring server performance and preventing crawler-related hazards are considered main strengths of this approach. Experiments over six-month Web server logs of a non-commercial Web site evidence the benefits of focused Web profiling and, in particular, of this approach.
TypeConference paper
URIhttp://hdl.handle.net/1822/54562
ISBN978-972-8924-93-5
Publisher versionhttp://www.iadisportal.org/digital-library/mdownload/web-crawler-profiling-and-containment-through-navigation-pattern-mining
Peer-Reviewedyes
AccessRestricted access (Author)
Appears in Collections:CAlg - Artigos em livros de atas/Papers in proceedings

Files in This Item:
File Description SizeFormat 
2009-CI-WWWIADIS-Lourenco&Belo-CRP.pdf
  Restricted access
282,24 kBAdobe PDFView/Open

Partilhe no FacebookPartilhe no TwitterPartilhe no DeliciousPartilhe no LinkedInPartilhe no DiggAdicionar ao Google BookmarksPartilhe no MySpacePartilhe no Orkut
Exporte no formato BibTex mendeley Exporte no formato Endnote Adicione ao seu ORCID