Free tools

Robots.txt Examples

Robots.txt file content for jooble.org.

Robot.txt file for: jooble.org

      User-agent: *
User-agent: GoogleBot
User-agent: GoogleBot-Mobile
User-agent: Mediapartners-Google
User-agent: Yandex
Disallow: /*.ashx
Disallow: /*.aspx
Disallow: /*?closedJob=True
Disallow: /Account/
Disallow: /account/
Disallow: /alert-vacancy
Disallow: /away/
Disallow: /click/AFSClick
Disallow: /desc/
Disallow: /employer/api
Disallow: /employer/company
Disallow: /employer/employerpage
Disallow: /employer/payment
Disallow: /Export/
Disallow: /Header/AccountMenu
Disallow: /Handlers
Disallow: /m/Account
Disallow: /m/account
Disallow: /m/alert-vacancy
Disallow: /m/away/
Disallow: /m/desc/
Disallow: /m/Handlers
Disallow: /m/JobDesc
Disallow: /m/MobileCompanyInfo/Redirect
Disallow: /m/redir
Disallow: /m/Subscribe
Disallow: /m/sub-vacancy
Disallow: /MobileCompanyInfo/Redirect
Disallow: /msg-vacancy/*
Disallow: /redir
Disallow: /saved
Disallow: /search1-*
Disallow: /SearchResult.aspx
Disallow: /Subscribe
Disallow: /sub-vacancy
Disallow: /visited
Disallow: /profile
Disallow: /*?sgnl=1
Disallow: /*?sgn=1 
Disallow: /auction/
Disallow: /*?ckey=
Disallow: /*&elckey=
Disallow: /*&sid=
Disallow: /*&scid=
Disallow: /*&scr=
Disallow: /SearchResult
Disallow: /m/SearchResult
Disallow: /*?searchSource=0
Disallow: /external/
Disallow: */spa-metrics
Disallow: */Statistics/*
Disallow: */BannerStat/*
Disallow: /BannusHandler

User-agent: eSobiSubscriber
User-agent: Exabot
User-agent: MJ12bot
User-agent: SemrushBot
User-agent: SemrushBot-SA
Disallow: /
Disallow: */spa-metrics
Disallow: */Statistics/*
Disallow: */BannerStat/*

Host: https://jooble.org