Free tools
Robots.txt Examples
Robots.txt file content for 123rf.com.
Robot.txt file for: 123rf.com
User-agent: * Disallow: /downloadcomp.php Disallow: /profileinterview.php Disallow: /pdf_lightbox.php Disallow: /downloadpubliclightbox.php Disallow: /mylightbox.php Disallow: /similar.php Disallow: /similarmodel_ajax.php Disallow: /bova/ Disallow: /images/verify_image.png Disallow: /images/welcome_image.png Disallow: /images/ga_img_handler.php Disallow: /bigdata/logging/ Disallow: /visual/ User-agent: GPTBot Disallow: / User-agent: Amazonbot Disallow: / User-agent: CCBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: Google-Extended Disallow: /