# robots.txt — REVANCA # https://revanca.com/ Sitemap: https://revanca.com/sitemap.xml # ── Bloated crawlers (no SEO value, high noise) ────────────────────────── User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: Dotbot Disallow: / User-agent: SEOkicks-Robot Disallow: / User-agent: SISTRIX Disallow: / User-agent: SISTRIX Crawler Disallow: / User-agent: SearchmetricsBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: MegaIndex.ru Disallow: / User-agent: megaindex.com Disallow: / User-agent: Exabot Disallow: / User-agent: CCBot Disallow: / User-agent: Yandex Disallow: / User-agent: Baiduspider Disallow: / User-agent: Sogou Disallow: / User-agent: Sogou Spider Disallow: / User-agent: Qwantify Disallow: / User-agent: rogerbot Disallow: / User-agent: spbot Disallow: / User-agent: Screaming Frog SEO Spider Disallow: / # ── Social media crawlers (allow everything) ──────────────────────────── User-agent: facebookexternalhit Allow: / User-agent: Twitterbot Allow: / # ── General crawler rules ──────────────────────────────────────────────── User-agent: * Crawl-delay: 10 Disallow: /col/ Disallow: /API/ Disallow: /app/ Disallow: /qr-gen/ Disallow: /vendor/ Disallow: /framepop/ Allow: /