{"id":273679,"date":"2026-01-19T22:36:58","date_gmt":"2026-01-19T22:36:58","guid":{"rendered":"https:\/\/wordpress.org\/plugins\/openbotauth-ai-crawler-access-control\/"},"modified":"2026-01-19T22:47:32","modified_gmt":"2026-01-19T22:47:32","slug":"openbotauth","status":"publish","type":"plugin","link":"https:\/\/jv.wordpress.org\/plugins\/openbotauth\/","author":23433010,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_crdt_document":"","version":"1.0.0","stable_tag":"1.0.0","tested":"6.9.4","requires":"6.0","requires_php":"7.4","requires_plugins":null,"header_name":"OpenBotAuth \u2013 AI Crawler Access Control","header_author":"OpenBotAuth","header_description":"Verify AI crawlers with HTTP Signatures and enforce allow\/deny\/teaser policies; also serves llms.txt and AI-ready feeds.","assets_banners_color":"d3ba91","last_updated":"2026-01-19 22:47:32","external_support_url":"","external_repository_url":"","donate_link":"","header_plugin_uri":"https:\/\/openbotauth.org","header_author_uri":"https:\/\/github.com\/OpenBotAuth\/openbotauth","rating":0,"author_block_rating":0,"active_installs":0,"downloads":125,"num_ratings":0,"support_threads":0,"support_threads_resolved":0,"author_block_count":0,"sections":["description","installation","faq","changelog"],"tags":{"0.1.0":{"tag":"0.1.0","author":"hammadtq","date":"2026-01-19 22:41:24"},"1.0.0":{"tag":"1.0.0","author":"hammadtq","date":"2026-01-19 22:47:32"}},"upgrade_notice":{"1.0.0":"<p>First stable release. AI referrer tracking via utm_source (ChatGPT, Claude, Gemini, Perplexity, Copilot). AI feed options now enabled by default. Feed limit increased to 100 posts.<\/p>","0.1.3":"<p>Yoast SEO compatibility, visual analytics dashboard (now the default tab), and configurable AI endpoints. See which AI bots are crawling your site. Use the &quot;Use Yoast llms.txt&quot; toggle if you want Yoast to manage llms.txt. Configure which post types appear in your AI endpoints.<\/p>","0.1.2":"<p>New AI-ready endpoints: llms.txt, JSON feed, and per-post markdown. Makes your content discoverable by AI systems without any external dependencies.<\/p>","0.1.1":"<p>Important security and correctness fixes. Human visitors now correctly bypass gating. Whitelist-only mode now properly denies non-whitelisted agents.<\/p>"},"ratings":[],"assets_icons":{"icon-128x128.png":{"filename":"icon-128x128.png","revision":3442835,"resolution":"128x128","location":"assets","locale":""},"icon-256x256.png":{"filename":"icon-256x256.png","revision":3442835,"resolution":"256x256","location":"assets","locale":""}},"assets_banners":{"banner-1544x500.png":{"filename":"banner-1544x500.png","revision":3442835,"resolution":"1544x500","location":"assets","locale":""},"banner-772x250.png":{"filename":"banner-772x250.png","revision":3442835,"resolution":"772x250","location":"assets","locale":""}},"assets_blueprints":{},"all_blocks":[],"tagged_versions":["0.1.0","1.0.0"],"block_files":[],"assets_screenshots":[],"screenshots":{"1":"Analytics tab - Visual dashboard showing bot traffic, charts, stats cards, and decision breakdown","2":"AI Endpoints tab - Enable llms.txt, JSON feed, and markdown with copyable URLs","3":"Configuration tab - Configure verifier URL, default policy, and access controls","4":"Per-post policy override in the post editor"},"jetpack_post_was_ever_published":false},"plugin_section":[],"plugin_tags":[2353,4866,9837,244604,600],"plugin_category":[54],"plugin_contributors":[254294],"plugin_business_model":[],"class_list":["post-273679","plugin","type-plugin","status-publish","hentry","plugin_tags-ai","plugin_tags-bots","plugin_tags-crawler","plugin_tags-llms-txt","plugin_tags-security","plugin_category-security-and-spam-protection","plugin_contributors-hammadtq","plugin_committers-hammadtq"],"banners":{"banner":"https:\/\/ps.w.org\/openbotauth\/assets\/banner-772x250.png?rev=3442835","banner_2x":"https:\/\/ps.w.org\/openbotauth\/assets\/banner-1544x500.png?rev=3442835","banner_rtl":false,"banner_2x_rtl":false},"icons":{"svg":false,"icon":"https:\/\/ps.w.org\/openbotauth\/assets\/icon-128x128.png?rev=3442835","icon_2x":"https:\/\/ps.w.org\/openbotauth\/assets\/icon-256x256.png?rev=3442835","generated":false},"screenshots":[],"raw_content":"<!--section=description-->\n<p><strong>OpenBotAuth<\/strong> helps publishers control automated access from AI crawlers and agents. It verifies requests using RFC 9421 HTTP Message Signatures (via a configurable verifier) and applies per-site or per-post policies like allow, deny, teaser previews, and 402 payment-required responses. It also publishes AI-friendly endpoints like llms.txt, a JSON feed, and per-post Markdown.<\/p>\n\n<p>Instead of blocking all bots or allowing unrestricted access, you can:<\/p>\n\n<ul>\n<li><strong>See AI bot traffic<\/strong> - Local-only analytics showing which bots are crawling your site<\/li>\n<li><strong>Verify bot identity<\/strong> using cryptographic signatures (RFC 9421)<\/li>\n<li><strong>Show teasers<\/strong> to unverified bots (first N words)<\/li>\n<li><strong>Rate limit<\/strong> bot access per agent<\/li>\n<li><strong>Whitelist\/blacklist<\/strong> specific bots<\/li>\n<\/ul>\n\n<h4>Key Features<\/h4>\n\n<ul>\n<li><strong>Bot Traffic Analytics<\/strong> - See which AI bots (GPTBot, ClaudeBot, PerplexityBot, etc.) are visiting your site<\/li>\n<li><strong>Signature Verification<\/strong> - Verifies RFC 9421 HTTP Message Signatures using Ed25519 cryptography<\/li>\n<li><strong>Content Teasers<\/strong> - Show first N words to unverified bots with customizable per-post settings<\/li>\n<li><strong>Rate Limiting<\/strong> - Per-agent rate limits with configurable time windows<\/li>\n<li><strong>Access Control<\/strong> - Whitelist\/blacklist with wildcard pattern matching<\/li>\n<li><strong>Local Analytics<\/strong> - Visual dashboard with charts, stats cards, and decision breakdowns (no external tracking)<\/li>\n<li><strong>AI-Ready Endpoints<\/strong> - Serve llms.txt, JSON feed, and markdown for AI crawlers<\/li>\n<li><strong>Tabbed Admin Interface<\/strong> - Clean organization with Analytics, AI Endpoints, and Configuration tabs<\/li>\n<\/ul>\n\n<h4>AI-Ready Endpoints<\/h4>\n\n<p>OpenBotAuth provides machine-readable endpoints for AI systems:<\/p>\n\n<ul>\n<li><strong>\/llms.txt<\/strong> - Standardized AI feed discovery (also at \/.well-known\/llms.txt)<\/li>\n<li><strong>\/.well-known\/openbotauth-feed.json<\/strong> - JSON list of all published posts<\/li>\n<li><strong>\/.well-known\/openbotauth\/posts\/{ID}.md<\/strong> - Per-post markdown content<\/li>\n<\/ul>\n\n<p>Configure which post types to include (posts, pages, or custom types) and set the feed limit (up to 500 items). All data is served locally from your WordPress database. No external tracking or telemetry. Only published, non-password-protected posts are exposed.<\/p>\n\n<h4>How It Works<\/h4>\n\n<ol>\n<li>AI agent signs HTTP request with its private key (RFC 9421 signature)<\/li>\n<li>WordPress plugin extracts signature headers and sends them to a verifier service<\/li>\n<li>Verifier fetches agent's public key from registry and verifies signature<\/li>\n<li>Plugin applies policy: allow full content, show teaser, require payment, or deny<\/li>\n<\/ol>\n\n<h4>External Service Disclosure<\/h4>\n\n<p><strong>This plugin connects to an external verifier service.<\/strong> When a signed bot request is received, the plugin sends the following data to your configured verifier URL via <code>wp_remote_post<\/code>:<\/p>\n\n<ul>\n<li>HTTP method (GET, POST, etc.)<\/li>\n<li>The accessed URL (including query string, if present)<\/li>\n<li>HTTP signature headers (Signature, Signature-Input, Signature-Agent)<\/li>\n<li>Additional HTTP header values explicitly listed in the Signature-Input header (e.g., content-type, accept, user-agent)<\/li>\n<\/ul>\n\n<p><strong>Privacy protection:<\/strong> Sensitive headers (cookies, authorization, proxy-authorization, www-authenticate) are NEVER forwarded, even if present in the request. If a bot's signature covers a sensitive header, verification will fail with a clear error.<\/p>\n\n<p><strong>No WordPress user accounts or personal data is transmitted.<\/strong> Only the headers explicitly covered by the bot's signature are forwarded to enable cryptographic verification. Note that the URL may include query parameters depending on your site's structure.<\/p>\n\n<p>You can:\n* Use the hosted verifier at <code>https:\/\/verifier.openbotauth.org\/verify<\/code>\n* Self-host the verifier service (see documentation)\n* The verifier service may log requests server-side depending on your configuration<\/p>\n\n<p><strong>Analytics are local-only.<\/strong> Decision counts (allow\/teaser\/deny\/pay\/rate_limit) and bot traffic observations (User-Agent based) are stored in your WordPress database. No analytics data is sent to external servers.<\/p>\n\n<p>For more information, please review our <a href=\"https:\/\/openbotauth.org\/terms\">Terms of Service<\/a> and <a href=\"https:\/\/openbotauth.org\/privacy\">Privacy Policy<\/a>.<\/p>\n\n<h3>Developer Hooks<\/h3>\n\n<h4>Filters<\/h4>\n\n<p><strong>openbotauth_policy<\/strong>\nModify policy before applying:<\/p>\n\n<pre><code>add_filter('openbotauth_policy', function($policy, $post) {\n    if ($post-&gt;post_type === 'premium') {\n        $policy['price_cents'] = 1000;\n    }\n    return $policy;\n}, 10, 2);\n<\/code><\/pre>\n\n<h4>Actions<\/h4>\n\n<p><strong>openbotauth_verified<\/strong>\nTriggered when a bot is verified:<\/p>\n\n<pre><code>add_action('openbotauth_verified', function($agent, $post) {\n    error_log(\"Bot {$agent['jwks_url']} accessed post {$post-&gt;ID}\");\n}, 10, 2);\n<\/code><\/pre>\n\n<p><strong>openbotauth_payment_required<\/strong>\nTriggered when 402 is returned:<\/p>\n\n<pre><code>add_action('openbotauth_payment_required', function($agent, $post, $price) {\n    \/\/ Track payment requests\n}, 10, 3);\n<\/code><\/pre>\n\n<h4>AI Endpoint Filters (v0.1.2+)<\/h4>\n\n<p><strong>openbotauth_should_serve_llms_txt<\/strong>\nDisable llms.txt endpoint (e.g., when using Yoast):<\/p>\n\n<pre><code>add_filter('openbotauth_should_serve_llms_txt', '__return_false');\n<\/code><\/pre>\n\n<p><strong>openbotauth_should_serve_feed<\/strong>\nDisable JSON feed endpoint:<\/p>\n\n<pre><code>add_filter('openbotauth_should_serve_feed', '__return_false');\n<\/code><\/pre>\n\n<p><strong>openbotauth_should_serve_markdown<\/strong>\nDisable markdown endpoints:<\/p>\n\n<pre><code>add_filter('openbotauth_should_serve_markdown', '__return_false');\n<\/code><\/pre>\n\n<p><strong>openbotauth_feed_item<\/strong>\nModify feed items:<\/p>\n\n<pre><code>add_filter('openbotauth_feed_item', function($item, $post) {\n    $item['custom_field'] = get_post_meta($post-&gt;ID, 'my_field', true);\n    return $item;\n}, 10, 2);\n<\/code><\/pre>\n\n<p><strong>openbotauth_markdown_content<\/strong>\nPost-process markdown output:<\/p>\n\n<pre><code>add_filter('openbotauth_markdown_content', function($markdown, $post) {\n    return $markdown . \"\\n\\n---\\nCopyright notice here\";\n}, 10, 2);\n<\/code><\/pre>\n\n<!--section=installation-->\n<ol>\n<li>Upload the <code>wordpress-openbotauth<\/code> folder to <code>\/wp-content\/plugins\/<\/code><\/li>\n<li>Activate the plugin through the 'Plugins' menu in WordPress<\/li>\n<li>Go to <strong>Settings &gt; OpenBotAuth<\/strong> to configure<\/li>\n<li>Configure your <strong>Verifier Service<\/strong>:\n\n<ul>\n<li>Check \"Use hosted OpenBotAuth verifier\" to use the hosted service, OR<\/li>\n<li>Enter your self-hosted verifier URL (e.g., <code>http:\/\/localhost:8081\/verify<\/code> for local dev)<\/li>\n<li>Leave empty to disable verification (all signed requests treated as unverified)<\/li>\n<\/ul><\/li>\n<li>Configure your default policy (Allow, Teaser, or Deny)<\/li>\n<\/ol>\n\n<!--section=faq-->\n<dl>\n<dt id=\"do%20i%20need%20to%20run%20my%20own%20verifier%20service%3F\"><h3>Do I need to run my own verifier service?<\/h3><\/dt>\n<dd><p>No, you can enable the hosted verifier in Settings by checking \"Use hosted OpenBotAuth verifier\". For privacy requirements or custom configurations, you can self-host the verifier service. The plugin does not contact any external service until you explicitly configure it.<\/p><\/dd>\n<dt id=\"will%20this%20block%20normal%20human%20visitors%3F\"><h3>Will this block normal human visitors?<\/h3><\/dt>\n<dd><p>No. The plugin only applies to requests that include RFC 9421 signature headers. Normal browser requests without signature headers see full content and bypass OpenBotAuth entirely.<\/p><\/dd>\n<dt id=\"what%20is%20a%20teaser%3F\"><h3>What is a teaser?<\/h3><\/dt>\n<dd><p>A teaser shows the first N words of your content to unverified bots, with a notice that authenticated bots can access full content. You can configure the word count globally or per-post.<\/p><\/dd>\n<dt id=\"does%20the%20402%20payment%20feature%20process%20actual%20payments%3F\"><h3>Does the 402 payment feature process actual payments?<\/h3><\/dt>\n<dd><p>No. The 402 response is a stub that returns the configured price and payment URL. Actual payment processing requires custom integration.<\/p><\/dd>\n<dt id=\"what%20data%20does%20the%20plugin%20send%20externally%3F\"><h3>What data does the plugin send externally?<\/h3><\/dt>\n<dd><p>Only signature verification requests are sent to your configured verifier URL. The request includes the URL being accessed and the signature headers. No personal data, cookies, or user information is transmitted.<\/p><\/dd>\n<dt id=\"does%20the%20plugin%20send%20any%20data%20to%20external%20servers%3F\"><h3>Does the plugin send any data to external servers?<\/h3><\/dt>\n<dd><p>The only external call is signature verification (if configured). All analytics are stored locally in your WordPress database. No telemetry or tracking data is sent to any external server.<\/p><\/dd>\n<dt id=\"does%20openbotauth%20work%20with%20yoast%20seo%3F\"><h3>Does OpenBotAuth work with Yoast SEO?<\/h3><\/dt>\n<dd><p>Yes. OpenBotAuth works alongside Yoast SEO without conflicts. By default, OpenBotAuth serves llms.txt (works standalone). If Yoast is installed and you've enabled Yoast's llms.txt feature, use the \"Use Yoast llms.txt\" toggle in AI Endpoints settings to let Yoast handle it. OpenBotAuth's unique feed and markdown endpoints remain active either way.<\/p><\/dd>\n<dt id=\"how%20is%20the%20admin%20interface%20organized%3F\"><h3>How is the admin interface organized?<\/h3><\/dt>\n<dd><p>The settings page has three tabs:<\/p>\n\n<ul>\n<li><strong>Analytics<\/strong> - Visual dashboard showing bot traffic, charts, stats cards, and decision breakdown<\/li>\n<li><strong>AI Endpoints<\/strong> - llms.txt, JSON feed, markdown settings with copyable URLs<\/li>\n<li><strong>Configuration<\/strong> - Verifier URL, default policy, whitelist\/blacklist, rate limits<\/li>\n<\/ul><\/dd>\n<dt id=\"do%20ai%20endpoints%20bypass%20membership%20or%20paywall%20plugins%3F\"><h3>Do AI endpoints bypass membership or paywall plugins?<\/h3><\/dt>\n<dd><p>The AI endpoints (llms.txt, JSON feed, markdown) serve content directly on early request interception and only check if content is published and not password-protected. If you use a membership or paywall plugin that restricts content via later WordPress hooks, that content may still be accessible via AI endpoints.<\/p>\n\n<p>To control this, you can:<\/p>\n\n<ul>\n<li>Disable specific AI endpoints in Settings &gt; OpenBotAuth &gt; AI Endpoints<\/li>\n<li>Use the <code>openbotauth_should_serve_feed<\/code>, <code>openbotauth_should_serve_llms_txt<\/code>, or <code>openbotauth_should_serve_markdown<\/code> filters to add custom logic<\/li>\n<li>Exclude specific post types from AI endpoints using the post types setting<\/li>\n<\/ul><\/dd>\n\n<\/dl>\n\n<!--section=changelog-->\n<h4>1.0.0<\/h4>\n\n<ul>\n<li>First stable release for WordPress.org directory<\/li>\n<li>AI referrer tracking via utm_source parameter (ChatGPT, Claude, Gemini, Perplexity, Copilot)<\/li>\n<li>AI feed options enabled by default on fresh install<\/li>\n<li>Feed limit default increased to 100 posts<\/li>\n<li>Improved plugin activation defaults<\/li>\n<\/ul>\n\n<h4>0.1.3<\/h4>\n\n<ul>\n<li>Analytics is now the default tab (see bot traffic immediately on first visit)<\/li>\n<li>Bot traffic tracking: see which AI bots (GPTBot, ClaudeBot, PerplexityBot, etc.) are crawling your site<\/li>\n<li>Yoast SEO compatibility: user-controlled toggle to let Yoast manage llms.txt<\/li>\n<li>Default: OpenBotAuth llms.txt stays ON (no silent failures if Yoast llms.txt not configured)<\/li>\n<li>Added \"Use Yoast llms.txt\" toggle when Yoast is detected (OFF by default)<\/li>\n<li>Shows \"Managed by Yoast\" badge when toggle is enabled<\/li>\n<li>Feed + markdown endpoints work alongside Yoast (OpenBotAuth's unique value)<\/li>\n<li>Renamed \"AI Artifacts\" tab to \"AI Endpoints\" for clarity<\/li>\n<li>Configurable post types for AI endpoints (posts, pages, custom types)<\/li>\n<li>Empty post types setting now correctly returns empty feed<\/li>\n<li>Markdown endpoint respects post types setting<\/li>\n<li>Fixed disabled endpoints returning proper 404 (not falling through to WordPress)<\/li>\n<li>Fixed subdirectory install routing to prevent \/blog matching \/blog2<\/li>\n<li>Visual analytics dashboard with charts and stats cards<\/li>\n<li>Improved first-run UX and WP-admin-friendly copy<\/li>\n<\/ul>\n\n<h4>0.1.2<\/h4>\n\n<ul>\n<li>Added llms.txt endpoint for AI discovery (\/llms.txt and \/.well-known\/llms.txt)<\/li>\n<li>Added JSON feed at \/.well-known\/openbotauth-feed.json<\/li>\n<li>Added per-post markdown at \/.well-known\/openbotauth\/posts\/{ID}.md<\/li>\n<li>Added metadata provider abstraction for future Yoast\/SEO plugin integration<\/li>\n<li>Supports subdirectory WordPress installs<\/li>\n<li>Implements HTTP conditional GET (304 Not Modified responses)<\/li>\n<li>Added AI Artifacts settings tab in admin<\/li>\n<li>No rewrite rules - uses early request interception<\/li>\n<li>Filter hooks for endpoint customization: openbotauth_should_serve_llms_txt, openbotauth_should_serve_feed, openbotauth_should_serve_markdown<\/li>\n<li>Content filters: openbotauth_feed_item, openbotauth_markdown_content<\/li>\n<\/ul>\n\n<h4>0.1.1<\/h4>\n\n<ul>\n<li>Added local-only analytics dashboard<\/li>\n<li>Added openbotauth_policy filter for custom policy logic<\/li>\n<li>Added openbotauth_verified action for tracking verified requests<\/li>\n<li>Added openbotauth_payment_required action for payment events<\/li>\n<li>Improved whitelist-only semantics: non-whitelisted agents are now denied<\/li>\n<li>Fixed HTTP status codes for deny (403) and rate_limit (429) responses<\/li>\n<li>Human visitors without signature headers now bypass gating entirely<\/li>\n<li>Added URL sanitization for verifier and payment URL settings<\/li>\n<li>Softened payment UI text to clarify 402 stub functionality<\/li>\n<\/ul>\n\n<h4>0.1.0<\/h4>\n\n<ul>\n<li>Initial release<\/li>\n<li>RFC 9421 signature verification<\/li>\n<li>Content teaser support<\/li>\n<li>Rate limiting<\/li>\n<li>Whitelist\/blacklist support<\/li>\n<li>Per-post policy overrides<\/li>\n<li>REST API for policy retrieval<\/li>\n<\/ul>","raw_excerpt":"Verify AI crawlers with HTTP Signatures and enforce allow\/deny\/teaser policies; also serves llms.txt and AI-ready feeds.","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin\/273679","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin"}],"about":[{"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/types\/plugin"}],"replies":[{"embeddable":true,"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/comments?post=273679"}],"author":[{"embeddable":true,"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wporg\/v1\/users\/hammadtq"}],"wp:attachment":[{"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/media?parent=273679"}],"wp:term":[{"taxonomy":"plugin_section","embeddable":true,"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_section?post=273679"},{"taxonomy":"plugin_tags","embeddable":true,"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_tags?post=273679"},{"taxonomy":"plugin_category","embeddable":true,"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_category?post=273679"},{"taxonomy":"plugin_contributors","embeddable":true,"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_contributors?post=273679"},{"taxonomy":"plugin_business_model","embeddable":true,"href":"https:\/\/jv.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_business_model?post=273679"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}