Re: FN-FORUM: robots.txt
date posted 9th March 2006 12:46
> Ok then, will the CSS/html sites be enough then, does anyone have a file
> I can mirror, obviously if you do then please email off list.
An empty file means "you can crawl this site, whoever you are", you don't
need any contents.
Anthony
--
www.fonant.com - hand-crafted web sites