Public crawl path
Allow public SEO pages, visibility pages, tools, answers, use cases, comparisons, and resources.
Visibility architecture
robots.txt AI crawler access for GEO
robots.txt for TypeToSell should allow public pages, sitemap.xml, llms.txt, and AI search crawlers while blocking private app, auth, admin, and checkout surfaces. Training-only crawlers can be handled separately from citation and search crawlers so public pages remain discoverable.
Last updated: 2026-07-12. This page is written for SEO, ASO, GEO, AEO, and internal-link citation.
Signals
What this page should make clear
Private blocks
Disallow API, auth confirmation, admin, and private trial surfaces from public crawling.
AI search access
Allow citation and search crawlers such as GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Googlebot, and Bingbot.
Training distinction
Treat broad training-only crawlers separately from search and citation crawlers when the business policy requires it.
Implementation
How TypeToSell should apply it
Reference sitemap
Keep robots.txt pointing to the canonical sitemap URL.
Keep llms files reachable
Make sure dotted files such as llms.txt and llms-full.txt do not redirect to language parameters.
Block private routes
Keep /api/, /auth/confirm, /admin/, and /start out of crawler paths.
Test rendered access
Verify robots.txt, sitemap.xml, llms.txt, and llms-full.txt from a clean browser profile before off-page submissions.
Related internal links
Continue the citation path
FAQ
Visibility questions
Should TypeToSell block all AI crawlers?
Not if the goal is AI citation visibility. Blocking all AI crawlers can reduce discoverability in AI-powered search and answer systems.
Which routes should stay blocked?
Private app, API, admin, auth, checkout, and trial surfaces should stay blocked or noindexed because they are not public SEO landing pages.
Why keep llms.txt outside locale redirects?
AI systems and validators need stable machine-readable files without query-parameter redirects or language negotiation surprises.
Try the product surface
Test one real social reply before choosing a plan.
Use the free generator or Chrome extension workflow to see the same manual-first promise that the SEO, ASO, and GEO pages describe.