Free tools
Robots.txt Examples
Robots.txt file content for udn.com.
Robot.txt file for: udn.com
# KB13270 robots.txt User-agent: * Disallow: /srank/* #Disallow: /.well-known/amphtml/apikey.pub Disallow: /BT/* Disallow: /fcm/* Disallow: /NEWS/* Disallow: /NEWS/BREAKINGNEWS/* Disallow: /Project/* Disallow: /UDN/FOUNDER/* Disallow: /UDN/UDNENGLISH/* Disallow: /morakot/* Allow: /page/topic/184 Allow: /page/topic/495 Allow: /page/topic/496 Disallow: /page/topic/* Sitemap: https://udn.com/sitemapxml/news/mapindex.xml Sitemap: https://udn.com/sitemap/gnews/2 Sitemap: https://udn.com/sitemap/gnews/1013 Sitemap: https://udn.com/sitemap/gnews/1015 User-agent: GPTBot Disallow: / User-agent: Claude Disallow: / User-agent: ClaudeBot Disallow: /