Free tools
Robots.txt Examples
Robots.txt file content for dw.com.
Robot.txt file for: dw.com
User-agent: * Disallow: /search/ Disallow: /overlay/ Disallow: /popups/mediaplayer/ Disallow: /popups/popup_gallery/ Disallow: /*/layoutvorlagen/ Disallow: /*/user/account$ Disallow: /*/user/activity$ Disallow: /*/user/profile$ Disallow: /*/user/password/change$ Disallow: /*/user/password/set$ Disallow: /*/user/feedback/status?type=* Disallow: /*/user/register/confirm$ Disallow: /*/user/email/change$ User-agent: Googlebot User-agent: Bingbot Disallow: /*?* Disallow: /p/* Sitemap: https://www.dw.com/sitemap.xml