Free tools
Robots.txt Examples
Robots.txt file content for wired.com.
Robot.txt file for: wired.com
User-agent: * Disallow: /*?redirectURL= Disallow: /*/amp Disallow: /cgi-bin/ Disallow: /user/ Disallow: /templatedata/ Disallow: /partnerfeeds/ Disallow: /magazine/polls/ Disallow: /services/emailFriend Disallow: /politics/law/news/2001/12/48946 Disallow: /politics/law/news/2001/12/49013 Disallow: /politics/law/news/2001/12/49083 Disallow: /politics/law/news/2001/12/49141 Disallow: /politics/law/news/2002/03/51042 Disallow: /politics/law/news/2001/12/48938 Disallow: /politics/law/news/2001/12/48940 Disallow: /politics/law/news/2001/12/48943 Disallow: /politics/law/news/2001/12/49013 Disallow: /politics/law/news/2001/12/48929 Disallow: /politics/law/news/2001/12/48938 Disallow: /politics/law/news/2001/12/48940 Disallow: /politics/law/news/2001/12/48941 Disallow: /politics/law/news/2001/12/48942 Disallow: /politics/law/news/2001/12/48943 Disallow: /news/politics/0,1283,48946,00.html Disallow: /news/politics/0,1283,49132,00.html Disallow: /news/politics/0,48940-0.html Disallow: /news/politics/1,49013-0.html Disallow: /culture/lifestyle/news/2001/07/45247 Disallow: /culture/lifestyle/news/2001/02/41846 Disallow: /2001/07/dying-for-attention/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/cache/ Disallow: /wp-content/themes/ Disallow: /search Disallow: /culture/lifestyle/news/2001/08/46315 Disallow: /2001/08/gruesome-movie-sparks-outrage/ Disallow: /techbiz/media/news/2001/09/46556 Disallow: /2001/09/controversial-shock-site-for-sale/ Disallow: /2007/07/fbi-spyware-how/ Disallow: /partners/ Disallow: /sponsored/ Disallow: /video/right_rail* Disallow: /auth/* Disallow: /account/* Disallow: /preview/ Allow: /wp-content/ Allow: /wp-content/themes/Phoenix/assets/css/style.css Allow: /wp-content/themes/Phoenix/assets/css/hf.css Allow: /wp-content/themes/Phoenix/assets/images/apple-touch-icon.png Allow: /wp-content/themes/Phoenix/assets/images/ Allow: /wp-content/themes/wired/assets/images/gallery-cam@2x.png Allow: /wp-content/themes/wired/assets/images/subscribe_290x110.gif Allow: /wp-content/themes/wired/assets/images/wired_insider_1118_1000x190.jpg Allow: /wp-content/themes/wired/assets/images/subscribe_1000x190.gif Allow: /wp-content/themes/wired/assets/images/post_wired_logo_150x60.gif Allow: /wp-content/themes/wired/assets/images/wired_video_1000x190.jpg Allow: /wp-content/themes/wired/assets/images/subscribe_310x80.gif Allow: /wp-content/themes/wired/assets/images/disruptive_by_design_2012.gif Allow: /wp-content/themes/wired/assets/images/gallery-illo@2x.png Allow: /wp-content/themes/Phoenix/assets/js/vendor/picturefill.min.js Allow: /wp-content/themes/Phoenix/assets/images/apple-touch-icon.png Allow: /wp-content/themes/Phoenix/assets/images/nav-icons-dist/sprite-2-25.png Allow: /wp-content/themes/Phoenix/assets/images/rad-fallback/fallback-short.jpg Allow: /wp-content/themes/Phoenix/assets/images/rad-fallback/fallback-tall-b.jpg Allow: /wp-includes/js/wp-emoji-release.min.js Allow: /wp-includes/js/wp-emoji-release.min.js?ver=4.5.2 Allow: /wp-content/themes/Phoenix/assets/css/cyphon.css Allow: /wp-content/plugins/feature-story-builder/builder-frontend/assets/css/fsb.css Allow: /wp-includes/js/mediaelement/mediaelementplayer.min.css Allow: /wp-includes/js/mediaelement/wp-mediaelement.min.css Allow: /wp-content/themes/Phoenix/assets/css/exchange.css Allow: /wp-content/themes/Phoenix/assets/css/wired-blank-canvas.css Allow: /wp-content/plugins/google-to-table/css/google-to-table.css Allow: /wp-content/plugins/feature-story-builder/builder-frontend/assets/images/arrow.png Allow: /wp-content/plugins/feature-story-builder/builder-frontend/assets/images/footer-wired.png Allow: /wp-content/plugins/feature-story-builder/builder-frontend/assets/images/logo-w.png Allow: /wp-content/plugins/feature-story-builder/builder-frontend/assets/images/w.png Allow: /wp-includes/js/mediaelement/mediaelementplayer.min.css Allow: /wp-includes/js/mediaelement/wp-mediaelement.min.css Allow: /partners/bnymellon/futureofmoney/images Disallow: /*{{url}} Disallow: /*%7B%7Burl%7D%7D Disallow: /*{{section}} Disallow: /*%7B%7Bsection%7D%7D Disallow: /product/ Disallow: /user-context?referrer Disallow: /services.min.js?_= Disallow: /com.condenast/yv8* User-agent: Googlebot Allow: /*/amp Disallow: /*{{url}} Disallow: /*%7B%7Burl%7D%7D Disallow: /search Disallow: /*?redirectURL= Disallow: /auth/* Disallow: /account/* Disallow: /*{{section}} Disallow: /*%7B%7Bsection%7D%7D Disallow: /product/ Disallow: /user-context?referrer Disallow: /services.min.js?_= Disallow: /preview/ User-agent: Bingbot Allow: /*/amp Disallow: /*{{url}} Disallow: /*%7B%7Burl%7D%7D Disallow: /search Disallow: /*?redirectURL= Disallow: /auth/* Disallow: /account/* Disallow: /*{{section}} Disallow: /*%7B%7Bsection%7D%7D Disallow: /product/ Disallow: /user-context?referrer Disallow: /services.min.js?_= User-agent: Google-InspectionTool Allow: / User-agent: CCBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Amazonbot Disallow: / User-agent: FacebookBot Disallow: / User-agent: meta-externalagent Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: cohere-ai Disallow: / User-agent: Omgilibot Disallow: / User-agent: omgili Disallow: / User-agent: PerplexityBot Disallow: / User-agent: YouBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Diffbot Disallow: / User-agent: Applebot-Extended Disallow: / Sitemap: https://www.wired.com/sitemap.xml Sitemap: https://www.wired.com/branded-sitemap.xml #Sitemap: https://www.wired.com/coupons/sitemap.xml Sitemap: https://www.wired.com/video/sitemap.xml Sitemap: https://www.wired.com/feed/google-latest-news/sitemap-google-news Sitemap: https://www.wired.com/feed/rss Sitemap: https://www.wired.com/contributors-sitemap.xml Sitemap: https://www.wired.com/categories-sitemap.xml?hierarchy=tags #Sitemap: https://www.wired.com/categories-sitemap.xml Sitemap: https://www.wired.com/sitemap-archive-1.xml Sitemap: https://www.wired.com/bundles-sitemap.xml