Free tools

Robots.txt Examples

Robots.txt file content for birdeye.com.

Robot.txt file for: birdeye.com

      User-agent: *
Disallow: /redirect.php*
Disallow: /papi/*
Disallow: /bfapi/*
Disallow: */review/*
Disallow: /gapi/*
Disallow: /_next/*
Disallow: /_next-public/*
Disallow: */profiles-api/*
Disallow: /*?*srid=*
Disallow: /*?*rsource=*
Disallow: /*?*filter=*
Disallow: */claim/*
Disallow: */edit/*
Disallow: */sso/*
Disallow: /widget/*

# Allow specific paths for Twitter and Facebook bots
User-agent: Twitterbot
Allow: */review/*
Allow: /*?*srid=*

User-agent: facebookexternalhit
Allow: */review/*
Allow: /*?*srid=*

# Allow specific URLs for all bots
Allow: /blog/category/reviews/*
Allow: /blog/*review*


Sitemap: https://birdeye.com/sitemapindex.xml
Sitemap: https://birdeye.com/blog/sitemap_index.xml