Free tools
Robots.txt Examples
Robots.txt file content for ameblo.jp.
Robot.txt file for: ameblo.jp
User-agent: Googlebot Disallow: /*/page-*.html Allow: /*/entrylist.html Allow: /*/entrylist-*.html Allow: /*/entry-*.html Disallow: /*/entry*-* Disallow: /*/day*.html Disallow: /*/reader.html Disallow: /*/reader-*.html Disallow: /*/favorite.html Disallow: /*/favorite-*.html Disallow: /*/bookmark.html Disallow: /*/bookmark-*.html Disallow: /*/amemberentrylist.html Disallow: /*/amemberentrylist-*.html Disallow: /*/amemberentry-*.html Disallow: /*/imagelist.html Disallow: /*/imagelist-*.html Disallow: /*/image-*-*.html Allow: /nonstyleishida/videolist.html Allow: /nonstyleishida/videolist-*.html Allow: /inouenonstyle/videolist.html Allow: /inouenonstyle/videolist-*.html Allow: /chiaki1007akie/videolist.html Allow: /chiaki1007akie/videolist-*.html Allow: /takaoka-saki-blog/videolist.html Allow: /takaoka-saki-blog/videolist-*.html Allow: /sakuamema/videolist.html Allow: /sakuamema/videolist-*.html Allow: /norokayoblog/videolist.html Allow: /norokayoblog/videolist-*.html Allow: /o-rukyojin1116/videolist.html Allow: /o-rukyojin1116/videolist-*.html Allow: /kandaseiko1030/videolist.html Allow: /kandaseiko1030/videolist-*.html Disallow: /*/videolist.html Disallow: /*/videolist-*.html Allow: /*/video-*.html$ Disallow: /*/video-*.html?*vnm=* Disallow: /*/message-board.html Disallow: /*/comment-* Disallow: /*/themetop.html Disallow: /*/iine_entry-*.html Disallow: /summary/api/ Disallow: /accesslog/BlogAccessLog Disallow: /p/ Disallow: /_api$ Disallow: /_api/ Disallow: /_hook/*.html Disallow: /blog_portal/category/*/news Disallow: /_static/ Disallow: /s/embed/reblog-card/ User-agent: bingbot Crawl-delay: 1