Free tools

Robots.txt Examples

Robots.txt file content for prom.ua.

Robot.txt file for: prom.ua

      User-Agent: YadirectBot
Disallow: /

User-Agent: YandexDirect
Disallow: /

User-agent: *
Disallow: /account
Disallow: /admin
Disallow: /cabinet
Disallow: /company-contacts
Disallow: /context/
Disallow: /error/
Disallow: /feedback/
Disallow: /map/frame_map
Disallow: /opensearch.xml
Disallow: /opinions/create
Disallow: /order_mobile
Disallow: /order_mobile_confirm
Disallow: /order_v2
Disallow: /preview
Disallow: /product_opinion/create
Disallow: /product_view/ajax_
Disallow: /product_view/get_products_for_overlay
Disallow: /company/mark_invalid_phone
Disallow: /redirect
Disallow: /remote
Disallow: /search
Disallow: /shopping_cart
Disallow: /shop_settings/
Disallow: /social_auth/
Disallow: /tracker/
Disallow: /*/shopping_cart
Disallow: /*/partner_links
Disallow: /m*/offers*.html
Disallow: /for-you
Allow: /*?_escaped_fragment_=
Disallow: /*?

Allow: *?srsltid=*
Allow: *?utm_source=*
Allow: *?gclid=*

Allow: *?product_id=*

Allow: *?manufacturer_id=*
Allow: *?region_domain=*
Allow: *?a18=*

Allow: /.well-known/assetlinks.json