Free tools

Robots.txt Examples

Robots.txt file content for worldatlas.com.

Robot.txt file for: worldatlas.com

      User-agent: *
Disallow: /search?q=*
Disallow: /webimage/countrys/namerica/usstates/printpage/
Disallow: /webimage/countrys/samerica/printpage/
Disallow: /webimage/countrys/namerica/camerica/printpage/
Disallow: /aatlas/infopage/printpage/
Disallow: /webimage/countrys/printpage/
Disallow: /webimage/countrys/namerica/province/printpage/
Disallow: /webimage/countrys/oceania/printpage/
Disallow: /webimage/countrys/polar/printpage/
Disallow: /webimage/countrys/namerica/caribb/printpage/
Disallow: /aatlas/printpage/
Disallow: /webimage/countrys/africa/printpage/
Disallow: /webimage/countrys/asia/printpage/
Disallow: /webimage/countrys/europe/printpage/
Disallow: /webimage/countrys/namerica/printpage/

User-agent: Googlebot
Disallow: /tfBuster.html

Sitemap: https://www.worldatlas.com/webmap/sitemap-index.xml