RE: FN-FORUM: robots.txt

date posted 9th March 2006 13:05

For image folders, I usually have an index.php file that just contains a
header to the main site.

Does that work for robots? I suspect it does.

-----Original Message-----
From: [EMAIL REMOVED] [EMAIL REMOVED] On Behalf Of Lorna
McArdle - Spinette Designs
Sent: 09 March 2006 13:43
To: FN-FORUM / [EMAIL REMOVED]
Subject: Re: FN-FORUM: robots.txt


Thanks guys having read a couple of the tutials I can see they aren't infact

what I thought. I did think they could help in SEO's but I see it's more
about allowing and disallowing access to sites. IE the images folder which
is a good thing to try and disallow.

Regards
Lorna
----- Original Message -----
From: "Anthony Cartmell" [EMAIL REMOVED]
To: [EMAIL REMOVED]
Sent: Thursday, March 09, 2006 1:28 PM
Subject: Re: FN-FORUM: robots.txt


>
>> Ok then, will the CSS/html sites be enough then, does anyone have a file
>> I can mirror, obviously if you do then please email off list.
>
> An empty file means "you can crawl this site, whoever you are", you don't
> need any contents.
>
> Anthony
> --
> www.fonant.com - hand-crafted web sites
>
> --
> Freelancers, contractors earn more with Prosperity4
> Call 0870 870 4414 or visit www.prosperity4.com
> and benefit from Inland Revenue approved expenses today.
>
> To advertise here: http://www.freelancers.net/advertising.html
>
>
--
Freelancers, contractors earn more with Prosperity4
Call 0870 870 4414 or visit www.prosperity4.com
and benefit from Inland Revenue approved expenses today.

To advertise here: http://www.freelancers.net/advertising.html