# ServiceIQ — https://fieldserviceiq.ai
#
# GENERATED by scripts/generate-robots.mjs at build time. Do not hand-edit a
# copy in public/ — it would be a second source of truth, which is how the
# previous version came to cover 9 of 75 private routes. The Disallow list below
# is the complement of the public route set in src/App.tsx: add a private route
# and it is protected automatically.
#
# THREE rules govern the shape of this file. Break any one and the Disallow
# rules silently stop applying, with no error anywhere.
#
# 1. ONE group, no named User-agent blocks.
#    A crawler obeys the SINGLE most specific User-agent group that matches it
#    and ignores every other group, including "*". It does not merge them. A
#    block like
#
#        User-agent: Googlebot
#        Allow: /
#
#    does not "also allow Googlebot" — it REPLACES Googlebot's rules, and every
#    Disallow elsewhere stops applying to it. This file used to carry four such
#    blocks, so the crawlers that matter most were free to index /dashboard
#    and /auth. Only name an agent when it needs DIFFERENT rules.
#
# 2. NO BLANK LINES between "User-agent:" and the last Disallow.
#    A blank line terminates the record. Google tolerates it; the original
#    specification does not, and every rule after the first blank line then
#    belongs to no group at all. Comment-only lines are fine.
#
# 3. NO blanket "Allow: /".
#    Anything not disallowed is already allowed, so it buys nothing — and on
#    crawlers that resolve conflicts by FIRST match rather than longest match it
#    matches everything first and voids every Disallow below it.
#
# AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended,
# Applebot-Extended, meta-externalagent, Bytespider, Amazonbot, cohere-ai) are
# deliberately permitted, through the wildcard group rather than by name.
# "ServiceIQ" is a contested name, and which ServiceIQ an assistant describes is
# decided by what it was allowed to read; blocking them would cede the answer to
# whichever ServiceIQ stayed crawlable.
#
# If an AI crawler stops fetching, this file is probably not the cause.
# Cloudflare's "Managed robots.txt" / AI Crawl Control feature injects AI-bot
# blocks at the edge and OVERRIDES what is served here. Check that toggle first.
User-agent: *
Disallow: /admin
Disallow: /app
Disallow: /auth
Disallow: /cert/
Disallow: /checklists
Disallow: /contractor
Disallow: /contracts
Disallow: /customer-updates
Disallow: /customers
Disallow: /dashboard
Disallow: /deals
Disallow: /equipment
Disallow: /help
Disallow: /hubspot$
Disallow: /hubspot-callback
Disallow: /hubspot/install
Disallow: /integrations
Disallow: /invite/accept
Disallow: /invoices
Disallow: /jobs
Disallow: /media-upload
Disallow: /p/
Disallow: /partner
Disallow: /portal/
Disallow: /products
Disallow: /projects
Disallow: /proposals/new
Disallow: /reports
Disallow: /reset-password
Disallow: /resources/ai-in-construction-infographics/thank-you
Disallow: /resources/ai-questions-for-construction-owners/thank-you
Disallow: /resources/contractors-comprehensive-guide/thank-you
Disallow: /resources/contractors-guide-to-getting-paid-faster/thank-you
Disallow: /resources/data-centers-gold-mine-for-gcs/thank-you
Disallow: /resources/hubspot-fsm-schema/thank-you
Disallow: /resources/learners-guide-to-job-costing/thank-you
Disallow: /schedule
Disallow: /settings
Disallow: /setup
Disallow: /territories
Disallow: /trial/continue
Disallow: /unsubscribe
Disallow: /utilities
Disallow: /*/thank-you

Sitemap: https://fieldserviceiq.ai/sitemap.xml