MyTechSupport.ca :: Your Computer Technical Resource Headquarters! MyTechSupport.ca :: Your Computer Technical Resource Headquarters!
Home arrow Resources & Tools arrow Webmaster Tools arrow Generation Tools arrow Robots.txt Generator
Saturday, 23 March 2019
 
 
Main Menu
Home
Forums
Online Store
Web Directory
Affiliates
Sitemap
Featured Sites


Help Support Us!
Support MyTechSupport.ca in making a small donation:

Currency:



Robots.txt Generator

  Adv. Graphic Navbar Generator   CSS Generator
  CSS Navbar Generator   DHTML Menu Generator
  DHTML Tooltip Generator   Flash Drop-Down Menu Generator
  Flash Navbar Generator   Flash Status Bar
  Flash Tooltip Generator   Flash Topbar Generator
  Graphic Navbar Generator   IE 5.5 Scrollbar Generator
  Meta Tag Generator   Pop-Up Generator
  Robots.txt Generator   Sitemap Menu Generator
Robots.txt files are created by webmasters to mark (disallow) files and directories of a web site that search engine spiders (and other types of robots) should not access.

Default - All Robots are:
Crawl-Delay:
Specific Search Robots: Google googlebot
MSN Search msnbot
Yahoo yahoo-slurp
Ask/Teoma teoma
GigaBlast gigabot
Scrub The Web scrubby
DMOZ Checker robozilla
Nutch nutch
Alexa/Wayback ia_archiver
Baidu baiduspider
Specific Special Bots: Google Image googlebot-image
Yahoo MM yahoo-mmcrawler
MSN PicSearch psbot
SingingFish asterias
Yahoo Blogs yahoo-blogs/v3.9
Restricted Directories: The path is relative to root and must contain a trailing "/"

Now, copy and paste this text into a blank text file called "robots.txt" and put it in your root directory. Like all other files on your server, make sure it's permissions are set so that visitors (such as search engines) can read it.



Comments
Add New Search
+/- 
Write comment
Name:
Email:
 
Title:
UBBCode:
[b] [i] [u] [url] [quote] [code] [img] 
 
 
:angry::0:confused::cheer:B):evil::silly::dry::lol::kiss::D:pinch:
:(:shock::X:side::):P:unsure::woohoo::huh::whistle:;):s
:!::?::idea::arrow:
 
Please input the anti-spam code that you can read in the image.

3.22 Copyright (C) 2007 Alain Georgette / Copyright (C) 2006 Frantisek Hliva. All rights reserved."

 
 
 
 
Disclaimer
This site is NOT responsible for any damage that the information on this site may cause to your system. Everything you try, whether inspired by the response given from this site or not, is entirely at your own risk. All product names and company names used herein are for identification purpose only and may be trademarks or registered trademarks of their respective owners. We are in no way affiliated or representing any of the companies on this site unless specified.
 
Back to Top Back to Top
Generated in 0.23920 Seconds