Free tools
Robots.txt Examples
Robots.txt file content for behance.net.
Robot.txt file for: behance.net
Sitemap: https://www.behance.net/sitemap # Disallow AI bots User-agent: anthropic-ai User-agent: Applebot-Extended User-agent: Bytespider User-agent: CCbot User-agent: Claude-Web User-agent: ClaudeBot User-agent: Diffbot User-agent: FacebookBot User-agent: Google-Extended User-agent: GPTBot User-agent: ImagesiftBot User-agent: Meta-ExternalAgent User-agent: Omgili User-agent: Omgilibot User-agent: Timpibot User-agent: Webzio-Extended Disallow: / User-agent: * Disallow: /v2/ Disallow: /v3/ Disallow: /log$ Disallow: /search/*?*assetsCategory=* Disallow: /search/*?*assetsPriceTier=* Disallow: /search/*?*assetsExtensionTypes=* Disallow: /search/*?*color_hex=* Disallow: /search/*?*country=* Disallow: /search/*?*exif_aperture=* Disallow: /search/*?*exif_focal_length=* Disallow: /search/*?*exif_lens_brand=* Disallow: /search/*?*exif_make=* Disallow: /search/*?*exif_shutter_speed=* Disallow: /search/*?*exif_zoom=* Disallow: /search/*?*field=* Disallow: /search/*?*gif=* Disallow: /search/*?*premium=* Disallow: /search/*?*sort=* Disallow: /search/*?*schools=* Disallow: /search/*?*tools=* Disallow: /search/*?*time=* Disallow: /search/*?*userAvailability=* Disallow: /hire/browse