Thread
:
Interesting robot.txt's
View Single Post
09-07-2010, 12:39 AM
#
1
Dan
Status:
Request a custom title
Join date:
Feb 2005
Location:
Expertise:
Software:
Posts:
3,164
iTrader:
15
/ 86%
Interesting robot.txt's
Thought this was cute;
http://www.dailymail.co.uk/robots.txt
- Both intersting and creative for a job listing.
Also while we're at it take a peep at
http://www.last.fm/robots.txt
.
Know of any other websites with interesting robot.txt files? (:
Dan
View Public Profile
Find More Posts by Dan