Free tools
Robots.txt Examples
Robots.txt file content for bartleby.com.
Robot.txt file for: bartleby.com
User-agent: * Disallow: /_next/ Disallow: /auth/ Disallow: /backstoschoolsweepstakes Disallow: /cgi-bin/ Disallow: /config.js Disallow: /content/access Disallow: /contestrules Disallow: /dashboard Disallow: /document/ Disallow: /doubleclick/adx_iframe.html Disallow: /exponential/buster.html Disallow: /homework-help/ Disallow: /lti/ Disallow: /retain Disallow: /payment Disallow: /scripts/ Disallow: /search Disallow: /sign-out Disallow: /unlock-request Disallow: /uploads Disallow: /welcomeback Disallow: /s/ Disallow: /file/ Disallow: /virtual/ Disallow: /essay-snapshot/ Sitemap: https://www.bartleby.com/sitemap_index.xml Sitemap: https://www.bartleby.com/static/sitemap.xml