Public crawl path
Allow public SEO pages, visibilidad pages, tools, answers, use cases, comparisons, and resources.
Visibilidad architecture
El robots.txt de TypeToSell debe permitir paginas publicas, sitemap.xml, llms.txt y crawlers de busqueda AI, y bloquear superficies privadas de app, auth, admin y checkout. Los crawlers solo de entrenamiento pueden gestionarse aparte de los crawlers de citas y busqueda para mantener las paginas publicas descubribles.
Última actualización: 2026-07-12. This page is written for SEO, ASO, GEO, AEO, and internal-link citation.
Senales
Allow public SEO pages, visibilidad pages, tools, answers, use cases, comparisons, and resources.
Disallow API, auth confirmation, admin, and private trial surfaces from public crawling.
Allow citation and search crawlers such as GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Googlebot, and Bingbot.
Treat broad training-only crawlers separately from search and citation crawlers when the business policy requires it.
Implementation
Keep robots.txt pointing to the canonical sitemap URL.
Make sure dotted files such as llms.txt and llms-full.txt do not redirect to language parameters.
Keep /api/, /auth/confirm, /admin/, and /start out of crawler paths.
Perfil, sitio, demo, prueba u oferta solo se mencionan cuando el contexto encaja.
Enlaces internos relacionados
FAQ
Not if the goal is AI citation visibilidad. Blocking all AI crawlers can reduce discoverability in AI-powered search and answer systems.
Puedes probar borradores gratis antes de elegir un plan de pago.
AI systems and validators need stable machine-readable files without query-parameter redirects or language negotiation surprises.
Try the product surface
Extension Chrome para respuestas sociales con IA