2002-11-20

Robots, spiders and other user agents
The goal of this site, is to let you create a mod_rewrite, .htaccess or analog.cfg file with your selection of user agents!...you can come in and check that spider/bot that has crawled your site and see what is it all about and decide whether you should keep it out or let it in."