Robots.txt for abebooks.com
User-agent: * Disallow: /servlet/ Disallow: /abe/ Disallow: /abep/ Disallow: /cgi/ Disallow: /search/ Disallow: /docs/Newsletters/BD Disallow: /es/docs/BooksellerCentral/informativos Disallow: /collections/ User-agent: Googlebot Allow: /search/ Allow: /servlet/SearchResults Allow: /servlet/BookDetailsPL Allow: /servlet/BookstoreSearch Allow: /servlet/ListingDetails Allow: /servlet/StoreFrontDisplay Allow: /abe/ListingDetails Allow: /servlet/DWRestService/ Allow: /servlet/RecommendationsApi Allow: /servlet/HighlightInventory Allow: /collections/ Disallow: /servlet/ Disallow: /abe/ Disallow: /abep/ Disallow: /cgi/ Disallow: /docs/Newsletters/BD Disallow: /es/docs/BooksellerCentral/informativos Disallow: /*IMAGE_URL$ User-Agent: AdsBot Disallow: /servlet/HighlightInventory Disallow: /servlet/RecommendationsApi User-Agent: msnbot Allow: /search/ Allow: /servlet/SearchResults Allow: /servlet/BookDetailsPL Allow: /servlet/BookstoreSearch Allow: /servlet/ListingDetails Allow: /servlet/StoreFrontDisplay Allow: /abe/ListingDetails Allow: /servlet/DWRestService/ Allow: /collections/ Disallow: /servlet/ Disallow: /abe/ Disallow: /abep/ Disallow: /cgi/ Disallow: /docs/Newsletters/BD Disallow: /es/docs/BooksellerCentral/informativos User-Agent: bingbot Allow: /search/ Allow: /servlet/SearchResults Allow: /servlet/BookDetailsPL Allow: /servlet/BookstoreSearch Allow: /servlet/ListingDetails Allow: /servlet/StoreFrontDisplay Allow: /abe/ListingDetails Allow: /servlet/DWRestService/ Allow: /collections/ Disallow: /servlet/ Disallow: /abe/ Disallow: /abep/ Disallow: /cgi/ Disallow: /docs/Newsletters/BD Disallow: /es/docs/BooksellerCentral/informativos User-agent: Slurp Allow: /search/ Allow: /servlet/SearchResults Allow: /servlet/BookDetailsPL Allow: /servlet/BookstoreSearch Allow: /servlet/ListingDetails Allow: /servlet/StoreFrontDisplay Allow: /abe/ListingDetails Allow: /servlet/DWRestService/ Allow: /collections/ Disallow: /servlet/ Disallow: /abe/ Disallow: /abep/ Disallow: /cgi/ Disallow: /docs/Newsletters/BD Disallow: /es/docs/BooksellerCentral/informativos User-agent: Mediapartners-Google Disallow:/ User-agent: rogerbot Disallow: /author/ Disallow: /blog/ Disallow: /search/ Disallow: /book-search/ Disallow: /abe/ListingDetails Disallow: /servlet/ Disallow: /abe/ Disallow: /abep/ Disallow: /cgi/ Disallow: /docs/Newsletters/BD Disallow: /site-map/ Disallow: /textbook/isbn Disallow: /textbook/top-selling-textbooks/ Disallow: /products/isbn/ Disallow: /top-authors/ Disallow: /titles/ Disallow: /docs/Newsletters/ Disallow: /*plp$ Disallow: /*sf$ Disallow: /*bd$ User-agent: ClarityBot Allow: /search/ Allow: /servlet/SearchResults Allow: /servlet/BookDetailsPL Allow: /servlet/BookstoreSearch Allow: /servlet/ListingDetails Allow: /servlet/StoreFrontDisplay Allow: /abe/ListingDetails Allow: /servlet/DWRestService/ Allow: /servlet/RecommendationsApi Allow: /servlet/HighlightInventory Allow: /collections/ Disallow: /servlet/ # Sitemap files Sitemap: https://www.abebooks.com/sitemap.adhoc.bdp47.xml.gz Sitemap: https://www.abebooks.com/sitemap.bdp_index.xml Sitemap: https://www.abebooks.com/sitemap.plp_index.xml Sitemap: https://www.abebooks.com/brightspot-sitemap.xml