Free tools

Robots.txt Examples

Robots.txt file content for rakuten.co.jp.

Robot.txt file for: rakuten.co.jp

      User-Agent: *
Disallow: /images/
Disallow: /backup/
Disallow: /cgi-bin/
Disallow: /shops/
Disallow: /shop/ascosing/
Disallow: /shop/egs-g/
Disallow: /shop/tripleo-shop/
Disallow: /shop/fooody/
Disallow: /shop/kira-con02/
Disallow: /*/lightbox_*.html
Disallow: /aboutus/am/maint/