Skip to main content

AI Content Index

This page indexes all content on LearnWebCraft for AI crawlers, search engines, and content discovery.

83
Tutorials
10
Categories
29
Blog posts

Categories

Recent Blog Posts

View all blog posts →

Access Methods

Allowed AI Crawlers

LearnWebCraft welcomes AI crawlers configured in robots.txt:

OpenAI

  • GPTBot
  • ChatGPT-User
  • OAI-SearchBot

Anthropic

  • ClaudeBot
  • Claude-Web
  • anthropic-ai

Google

  • Googlebot
  • Google-Extended

Other

  • PerplexityBot
  • YouBot
  • cohere-ai
Last updated: 2026-07-28T14:56:31.452Z · 112 total articles