Publishers of technology books, eBooks, and videos for creative people

Home > Articles > Web Design & Development > Usability

  • Print
  • + Share This
Like this article? We recommend

Like this article? We recommend

Robots Site Settings

Some sites use the robots.txt file or the ROBOTS meta tag to tell spiders and others which parts of the Web site they may access. Be careful not to misuse this feature because it is quite easy to accidentally remove your site completely from search engine listings.

To allow spiders and robots to access all content on your site, include the following meta tag:

<meta name="ROBOTS" content="allow;all"> 

To prevent them from accessing any content on your site, include the following meta tag:

<meta name="ROBOTS" content="disallow;all">
  • + Share This
  • 🔖 Save To Your Account