Free tools

Robots.txt Examples

Robots.txt file content for hilton.com.

Robot.txt file for: hilton.com

      User-agent: AdIdxBot
Allow: */search/

User-agent: Mediapartners-Google
Allow: */search/
Allow: */book/

User-agent: AdsBot-Google-Mobile
Allow: */search/
Allow: */book/

User-agent: AdsBot-Google
Allow: */search/
Allow: */book/

User-agent: *
Disallow: */search/
Disallow: */checkin/
Disallow: */partner/
Disallow: /*.jhtml*
Disallow: */book/
Disallow: */guest/
Disallow: */auth/
Disallow: */group-search/
Disallow: */find-hotels/
Disallow: */event-locations/
Disallow: */group-locations/
Disallow: */confirmation/
Disallow: */personalized/
Disallow: */attend-my-event/
Disallow: */business/
Disallow: */modify/

User-agent: GoogleBot
Disallow: */search/
Disallow: */checkin/
Disallow: */partner/
Disallow: /*.jhtml*
Disallow: */book/
Disallow: */guest/
Disallow: */auth/
Disallow: */group-search/
Disallow: */find-hotels/
Disallow: */event-locations/
Disallow: */group-locations/
Disallow: */confirmation/
Disallow: */personalized/
Disallow: */attend-my-event/
Disallow: */business/
Disallow: */modify/

User-agent: BingBot
Disallow: */search/
Disallow: */checkin/
Disallow: */partner/
Disallow: /*.jhtml*
Disallow: */book/
Disallow: */guest/
Disallow: */auth/
Disallow: */group-search/
Disallow: */find-hotels/
Disallow: */event-locations/
Disallow: */group-locations/
Disallow: */confirmation/
Disallow: */personalized/
Disallow: */attend-my-event/
Disallow: */business/
Disallow: */modify/

User-agent: null
Sitemap: https://www.hilton.com/sitemap.xml