Free tools

Robots.txt Examples

Robots.txt file content for oup.com.

Robot.txt file for: oup.com

      User-agent: *
Disallow: /web_lifecycle/*
SITEMAP: https://global.oup.com/academic/sitemap.xml
SITEMAP: https://elt.oup.com/sitemap.xml
SITEMAP: https://global.oup.com/education/sitemap.xml
SITEMAP: https://global.oup.com/ukhe/sitemap.xml
SITEMAP: https://global.oup.com/ushe/sitemap.xml
Disallow: /academic/store/
Disallow: /academic/ecommerce/
Disallow: /academic/promo/
Disallow: /academic/promojump/
Disallow: /academic/-/l/cart*
Disallow: /academic/promotions/
Disallow: /academic/search
Disallow: /academic/advanced_search
Disallow: /emetest/
Disallow: /academic/xyz/
Disallow: /education/store/
Disallow: /education/searchresults
Disallow: /education/advancedsearch
Disallow: /education/basket/
Disallow: /education/account/
Disallow: /ukhe/xyz/
Disallow: /ushe/xyz/