Thread: How do I use a Robots file or do you have an example of one?
View Single Post
  #4 (permalink)  
Old 08-28-2009, 04:14 AM
Donk's Avatar
Donk Donk is offline
Supreme Babbler
  
Join Date: Jan 2009
Location: Kent UK
Posts: 590
Donk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation pointsDonk is an elite member of WB with 1500+ reputation points
Default
No problems.

One thing that was not included on that page was that you can use robots.txt to point to you (Google) xml site-map.

Code:
Sitemap: http://www.yoursite.com/sitemap.xml
Bob
__________________
PHP Snippets PHP Captcha Code Resizable Html
"A website with a thousand resources starts with a single page".
"Don't fight the gator - clear the swamp"
Reply With Quote