Free tools
Robots.txt Examples
Robots.txt file content for chegg.com.
Robot.txt file for: chegg.com
User-agent: ia_archiver Disallow: / User-agent: * Disallow: /Search Disallow: /_ajax Disallow: /study/_ajax Disallow: /auth Disallow: /login Disallow: /login/ Disallow: /account Disallow: /checkout Disallow: /homework-help/landing/7-day Disallow: /homework-help/landing/author Disallow: /homework-help/landing/general Disallow: /homework-help/landing/hwh Disallow: /homework-help/landing/offer Disallow: /homework-help/search? Disallow: /my/ Disallow: /myaccount Disallow: /psearch Disallow: /render Disallow: /returnlabel Disallow: /search Disallow: /selltextbooks/search Disallow: /sell-textbooks/search Disallow: /s/ Disallow: /shoppingcart Disallow: /signup Disallow: /extendables Disallow: /tracking Disallow: /tutors/signin/ Disallow: /forgotpassword Disallow: /test-prep/kaplan/redirect Disallow: /flashcards/search/ Disallow: /salliemae Disallow: /salliemae/termsandconditions Disallow: /tutors/s/ Disallow: /tutors/ide/ Disallow: /campaign Disallow: /textbooks/_ajax/log Disallow: /tutors/api/v1/event/ Disallow: /schools/_ajax/log Disallow: /tutors/publish/static/ssl-storage.html Disallow: /flashcards/r/ Disallow: /*/page_view_event/ Disallow: /*/session_trace/ Disallow: /*/page_view_timing/ Disallow: /*/metrics/ Disallow: /*/ajax/ Disallow: /*/jserrors/ Disallow: /*/aggregate/ Disallow: /tools/writing-center Disallow: /course-materials/ Disallow: /chat Disallow: /learn Disallow: /homework-help/definitions Disallow: /flashcards/ Disallow: /*/293KxffvUkJMLzB21vpMzE.json Disallow: /pla/* Disallow: /prepsheet-feedback/* User-agent: Mediapartners-Google Allow: /