Free tools

Robots.txt Examples

Robots.txt file content for vrbo.com.

Robot.txt file for: vrbo.com

      User-agent: *
Allow: /search/*unitId=*
Allow: /search/*noDates=*
Allow: /search/*petIncluded=*
Allow: /search/*adultsCount=*
Allow: /search/*childrenCount=*
Allow: /search/*arrival=*
Allow: /search/*departure=*
Allow: /search/*uni_id=*
Allow: /*/list

Disallow: */units
Disallow: */pm
Disallow: /listingpage.mvc/
Disallow: /my/
Disallow: /preview/
Disallow: /calendar
Disallow: */write
Disallow: /*?q
Disallow: /*?Q
Disallow: /*td/profiles/
Disallow: */listings/
Disallow: /reviews/write/
Disallow: /linktrip
Disallow: */td/
Disallow: /gd/
Disallow: /vpv/
Disallow: /forward
Disallow: /auth/
Disallow: /auth/ui/*
Disallow: /*/auth/ui/* 
Disallow: */traveler/
Disallow: /checkoutconfirmation
Disallow: *?*adultscount*
Disallow: *?emailaddress*
Disallow: /paymentconfirmation
Disallow: */home.sendtoafriendform
Disallow: */myshortlist
Disallow: */sendToAFriend
Disallow: */favorites
Disallow: */newsletter
Disallow: */inquiry
Disallow: */author/
Disallow: */searchTerms/
Disallow: */ratePlans/
Disallow: */thirdparty/
Disallow: */feriehus/reviews/write
Disallow: */property/compare
Disallow: /en-nz/search
Disallow: /en-au/search
Disallow: /en-nz/accommodation
Disallow: /en-au/accommodation
#hotel-from-bex-robots.txt
Disallow: /Hotel-Search
Disallow: /search?
Disallow: /ugc/urs/api/hotelreviews/hotel/HOTELID/
Disallow: /LastBooking/ElapsedTime
Disallow: /*Hotel-Information?
Disallow: /*Informacion-Hotel?
Disallow: /HotelCheckout
Disallow: /Checkout
Disallow: /checkout
Disallow: /HotelBookingConfirmation
Disallow: /*classic=*
Disallow: /*dateless=*
Disallow: /*chkin=*
Disallow: /*chkout=*
Disallow: /*startDate=*
Disallow: /*endDate=*
Disallow: /*unitId=*
Disallow: /*noDates=*
Disallow: /*petIncluded=*
Disallow: /*adultsCount=*
Disallow: /*childrenCount=*
Disallow: /*arrival=*
Disallow: /*departure=*
Disallow: /*uni_id=*
#bex-style-PDP
Disallow: /*.Hotel-Information
#contact-disallow
Disallow: */l/contact-us/
Disallow: */pdp/lo/

Allow: */api/vacation-rentals*
User-agent: Mediapartners-Google
Disallow:
User-agent: google-hoteladsverifier
Disallow: