Free tools
Robots.txt Examples
Robots.txt file content for techradar.com.
Robot.txt file for: techradar.com
################################################## # Vanilla-wide rules # ################################################## User-agent: * # Common path patterns (* prefix to handle localisation) Disallow: */deals/compare Disallow: */html/ Disallow: */p/*/embed/captioned # Common query string patterns Disallow: *searchTerm=* Disallow: *sortBy=* Disallow: *productBrand=* Disallow: *%7B*%7D* # Infinite scroll paths Disallow: /infinite-scroll-article/* Disallow: /infinite-scroll-review/* Disallow: /infinite-scroll-recipe/* ################################################## # Site-specific rules # ################################################## User-Agent: * Disallow: */price-comparison Disallow: */specification ################################################## # Sitemaps # ################################################## User-agent: * Sitemap: https://www.techradar.com/sitemap.xml Sitemap: https://www.techradar.com/sg/sitemap.xml Sitemap: https://www.techradar.com/uk/sitemap.xml Sitemap: https://www.techradar.com/au/sitemap.xml Sitemap: https://www.techradar.com/nz/sitemap.xml Sitemap: https://global.techradar.com/es-mx/sitemap.xml Sitemap: https://global.techradar.com/da-dk/sitemap.xml Sitemap: https://global.techradar.com/fi-fi/sitemap.xml Sitemap: https://global.techradar.com/no-no/sitemap.xml Sitemap: https://global.techradar.com/sv-se/sitemap.xml Sitemap: https://global.techradar.com/it-it/sitemap.xml Sitemap: https://global.techradar.com/nl-nl/sitemap.xml Sitemap: https://global.techradar.com/nl-be/sitemap.xml Sitemap: https://global.techradar.com/fr-fr/sitemap.xml Sitemap: https://global.techradar.com/de-de/sitemap.xml Sitemap: https://global.techradar.com/es-es/sitemap.xml