IRLbot User Agent Strings


IRL-crawler is a Texas A&M University research project sponsored in part by the National Science Foundation that investigates algorithms for mapping the topology of the Internet and discovering the various parts of the web. The crawler downloads random web pages (text only) and follows certain links to find other websites.

Click on any string to get more details

IRLbot 3.0

IRLbot 2.0