Free tools
Robots.txt Examples
Robots.txt file content for vrbo.com.
Robot.txt file for: vrbo.com
User-agent: * Allow: /search/*unitId=* Allow: /search/*noDates=* Allow: /search/*petIncluded=* Allow: /search/*adultsCount=* Allow: /search/*childrenCount=* Allow: /search/*arrival=* Allow: /search/*departure=* Allow: /search/*uni_id=* Allow: /*/list Disallow: */units Disallow: */pm Disallow: /listingpage.mvc/ Disallow: /my/ Disallow: /preview/ Disallow: /calendar Disallow: */write Disallow: /*?q Disallow: /*?Q Disallow: /*td/profiles/ Disallow: */listings/ Disallow: /reviews/write/ Disallow: /linktrip Disallow: */td/ Disallow: /gd/ Disallow: /vpv/ Disallow: /forward Disallow: /auth/ Disallow: /auth/ui/* Disallow: /*/auth/ui/* Disallow: */traveler/ Disallow: /checkoutconfirmation Disallow: *?*adultscount* Disallow: *?emailaddress* Disallow: /paymentconfirmation Disallow: */home.sendtoafriendform Disallow: */myshortlist Disallow: */sendToAFriend Disallow: */favorites Disallow: */newsletter Disallow: */inquiry Disallow: */author/ Disallow: */searchTerms/ Disallow: */ratePlans/ Disallow: */thirdparty/ Disallow: */feriehus/reviews/write Disallow: */property/compare Disallow: /en-nz/search Disallow: /en-au/search Disallow: /en-nz/accommodation Disallow: /en-au/accommodation #hotel-from-bex-robots.txt Disallow: /Hotel-Search Disallow: /search? Disallow: /ugc/urs/api/hotelreviews/hotel/HOTELID/ Disallow: /LastBooking/ElapsedTime Disallow: /*Hotel-Information? Disallow: /*Informacion-Hotel? Disallow: /HotelCheckout Disallow: /Checkout Disallow: /checkout Disallow: /HotelBookingConfirmation Disallow: /*classic=* Disallow: /*dateless=* Disallow: /*chkin=* Disallow: /*chkout=* Disallow: /*startDate=* Disallow: /*endDate=* Disallow: /*unitId=* Disallow: /*noDates=* Disallow: /*petIncluded=* Disallow: /*adultsCount=* Disallow: /*childrenCount=* Disallow: /*arrival=* Disallow: /*departure=* Disallow: /*uni_id=* #bex-style-PDP Disallow: /*.Hotel-Information #contact-disallow Disallow: */l/contact-us/ Disallow: */pdp/lo/ Allow: */api/vacation-rentals* User-agent: Mediapartners-Google Disallow: User-agent: google-hoteladsverifier Disallow: