Free tools
Robots.txt Examples
Robots.txt file content for jalan.net.
Robot.txt file for: jalan.net
User-agent: * Sitemap: https://www.jalan.net/sitemapxml/kaigai/abroad_sitemap.xml Sitemap: https://www.jalan.net/sitemapxml/dp/pc_dp.xml Sitemap: https://www.jalan.net/sitemapxml/sitemap/gourmet/pc_gourmetguide.xml Sitemap: https://www.jalan.net/sitemapxml/sitemap/travel/pc_travelguide.xml Sitemap: https://www.jalan.net/sitemapxml/kankou/pc_kankouguide.xml Sitemap: https://www.jalan.net/sitemapxml/sitemap/event/pc_eventguide.xml Sitemap: https://www.jalan.net/sitemapxml/old_toksyu.xml Sitemap: https://www.jalan.net/sitemapxml/dp/pc_dp.xml Sitemap: https://www.jalan.net/sitemapxml/pc_static.xml Sitemap: https://www.jalan.net/rentacar/sitemap.xml Sitemap: https://www.jalan.net/news/author-sitemap.xml Sitemap: https://www.jalan.net/news/sitemap_index.xml Sitemap: https://www.jalan.net/sitemapxml/kankou/pc_kankou_spt_photo_index.xml Sitemap: https://www.jalan.net/sitemapxml/sitemap/gourmet/pc_gourmet_grm_photo_index.xml Sitemap: https://www.jalan.net/sitemapxml/pc_asteria.xml Sitemap: https://www.jalan.net/sitemapxml/attentionword/pc_attention_word.xml Disallow: /ji/service/ Disallow: /cw/ Disallow: /ct/ctp1000/ Disallow: /yw/ Disallow: /sw/ Disallow: /mt/ Disallow: /sm/ Disallow: /cl/ Disallow: /uw/uwp8100/uww8101.do Disallow: /smart/snr/ Disallow: /us/snr/ Disallow: /dp/air/ Disallow: /pj/ Disallow: /jj/ Disallow: /campaign/secret2016/ Disallow: /campaign/secret2017/ Disallow: /campaign/secret2017_onsen/ Disallow: /campaign/secret2017_jal/ Disallow: /campaign/secret2017_service/ Disallow: /gw/ Disallow: /majibu/ Allow: /majibu/$ Allow: /majibu/concept/$