# Tescreal robots.txt # https://tescreal.co # # AI crawlers and assistants are welcome. # See also: https://tescreal.co/llms.txt # Human-readable AI crawler list: https://tescreal.co/robots User-agent: * Allow: / # --- OpenAI --- User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / User-agent: OAI-AdsBot Allow: / # --- Anthropic (Claude) --- User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Claude-Web Allow: / # --- Google AI --- User-agent: Google-Extended Allow: / User-agent: Googlebot Allow: / User-agent: GoogleOther Allow: / # --- Apple --- User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # --- Meta --- User-agent: FacebookBot Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: Meta-ExternalFetcher Allow: / # --- Microsoft / Bing --- User-agent: Bingbot Allow: / User-agent: BingPreview Allow: / # --- Perplexity --- User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / # --- Amazon --- User-agent: Amazonbot Allow: / # --- Cohere --- User-agent: cohere-ai Allow: / # --- Common Crawl (often used in AI datasets) --- User-agent: CCBot Allow: / # --- ByteDance --- User-agent: Bytespider Allow: / # --- Diffbot --- User-agent: Diffbot Allow: / # --- You.com --- User-agent: YouBot Allow: / # --- DuckAssist / DuckDuckGo --- User-agent: DuckAssistBot Allow: / # --- xAI --- User-agent: Grok Allow: / # --- Other research / AI indexing --- User-agent: AI2Bot Allow: / User-agent: ImagesiftBot Allow: / User-agent: omgili Allow: / User-agent: omgilibot Allow: / User-agent: Webzio-Extended Allow: / User-agent: TimpiBot Allow: / User-agent: Timpibot Allow: / Sitemap: https://tescreal.co/sitemap.xml