Free tools

Robots.txt Examples

Robots.txt file content for grubhub.com.

Robot.txt file for: grubhub.com

      User-agent: *
Disallow: /*gulp.action*
Disallow: /search
Disallow: /checkout/
Disallow: /account/
Disallow: /oauth2/
Disallow: /rds/
Disallow: /restaurant/*/menu/item/*/choice/*
Disallow: /restaurant/*/menu-item/*
Disallow: /restaurant/*?proof
Disallow: /program/
Disallow: /my-bag
Disallow: /restaurant/*?category
Disallow: /restaurant/*&category
Disallow: /restaurant/*?querySearch
Disallow: /restaurant/*&querySearch
Disallow: /lets-eat/*