October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
The Finance Base
The Money Desk · Blog
Re:

Cloudflare’s Pay Per Crawl: Can Your Website Get Paid When AI Crawlers Use Its Content?

Cloudflare’s Pay Per Crawl is a closed-beta system for charging participating AI crawlers—not a universal royalty program for past AI training. Here is how it works and what website owners can realistically expect.
From TheFinanceBase Team7 min to read
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Cloudflare is building a way for eligible website owners to charge participating AI crawlers for content retrieval. That is narrower than getting royalties whenever artificial-intelligence companies train on “your data.” As of August 18, 2026, Cloudflare’s monitoring product, AI Crawl Control, is available on all Cloudflare plans, but its Pay Per Crawl payment feature remains in closed beta. It pays for successful crawler access—not verified model training, citations, or downstream AI revenue.

What Cloudflare is actually offering

Cloudflare’s AI Crawl Control lets a domain operator identify AI services, inspect crawler activity, and choose whether each crawler is allowed, blocked, or charged. Monitoring and control features are described as available on all Cloudflare plans: Cloudflare AI Crawl Control documentation.

Pay Per Crawl is the monetization layer. It is still closed beta, and participation requires applying through Cloudflare or contacting an account executive if you are an Enterprise customer. The feature is intended for owners or operators who control a website’s traffic at Cloudflare’s edge—not for every individual whose words, images, comments, or other material appears online.

Cloudflare also lists an AI Index private-beta program for making participating domains discoverable to AI platforms under publisher-controlled conditions: AI Index early access.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
#1 Best Overall
ZimaBoard 2 1664 x86 Home Server, N150, 16GB LPDDR5,PCIe 3.0×4 Expansion
  • Server-Class Home Server Built for 24/7 Workloads - Designed as a purpose-built home server rather than general-purpose SBCs, Mini PCs, entry NAS systems, or routing-only devices. As a compact, pocket-sized single board server platform, ZimaBoard 2 1664 combines x86 architecture, quad-core performance up to 3.6GHz, 16GB DDR5 memory, and 64GB eMMC storage for reliable always-on home servers, homelabs, and self-hosted workloads.
  • PCIe 3.0 x4 Expansion for Real Server Builds - Built as a server-class platform with native PCIe expansion, ZimaBoard 2 features a full PCIe 3.0 x4 slot for high-speed, low-latency upgrades beyond USB-based limitations. Supports 10GbE NICs, NVMe adapters, GPUs, and AI accelerators to build scalable home servers, homelabs, and advanced self-hosted systems—offering greater expansion flexibility than typical SBCs, Mini PCs, and entry-level NAS devices.
  • Native Dual SATA & Dual 2.5GbE Networking - Built with server-class storage and networking I/O, ZimaBoard 2 integrates dual SATA ports for direct HDD/SSD connectivity and dual 2.5GbE Ethernet for high-throughput, low-latency networking. This architecture enables reliable DIY NAS, fast storage, routing, and multi-service home server deployments—while avoiding USB-based performance constraints common in ARM SBCs, Raspberry Pi–based setups, Mini PCs, and entry-level NAS devices.
  • ZimaOS Preinstalled + Wide OS Compatibility - Comes preinstalled with ZimaOS for a clean, ad-free private cloud experience—centralized file dashboard, automatic backups, P2P downloads, private photo/video sharing, 500+ plug-ins, and secure on-device AI that keeps your data at home. Also supports TrueNAS, Proxmox, Debian, Ubuntu Server, pfSense, OpenWrt, and Linux containers, making it perfect for Plex media servers, Pi-hole, firewalls, backups, Docker labs, home-cloud services, and multi-service deployments.
  • All-in-One NAS, Router, Docker & Homelab Server - Replace multiple devices with one low-power. ZimaBoard 2 can serve as a NAS, router, Docker host, firewall, media server, or homelab node—delivering a flexible, open alternative to ARM SBCs, Mini PCs, and entry-level NAS systems.

Why Cloudflare thinks crawler access should cost money

Traditional search crawlers generally exchanged indexing for potential referral traffic. Generative-AI systems can retrieve, summarize, or ingest pages while sending comparatively little traffic back. Cloudflare reported that, in its June 2025 measurements, OpenAI crawlers made about 1,700 requests per referral and Anthropic crawlers about 73,000 requests per referral. Those are Cloudflare’s measurements, not universal industry ratios: Cloudflare’s crawler analysis.

Cloudflare’s proposed choices are therefore simple: block the crawler, allow it for free, or allow it only when it pays. The aim is to give smaller publishers a machine-enforced option between an informal robots.txt request and a bespoke licensing negotiation.

How Pay Per Crawl works

The transaction happens when an AI crawler requests a protected page.

  1. The crawler requests the page from a domain using Cloudflare.
  2. Cloudflare returns HTTP 402 Payment Required and states the price, for example crawler-price: USD 0.01.
  3. A compatible crawler retries with crawler-exact-price or crawler-max-price.
  4. Those payment headers must be included in the cryptographically signed Web Bot Auth request; an arbitrary payment header is not enough.
  5. If the request succeeds, the page is delivered with HTTP 200 and a crawler-charged header. Failed requests are not billed.

The protocol details are documented in Cloudflare’s crawler instructions. A crawler that does not support the authentication and payment flow may stop, retry unpaid, or be blocked.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Who can receive money?

The likely payee is the verified operator of a participating website or domain. You need control of the site’s responses and sufficient rights to authorize access to the material returned from its pages.

  • Publishers and bloggers: potentially charge for original articles, archives, or frequently updated specialist information.
  • Documentation and education sites: price access to maintained technical or instructional material.
  • Ecommerce and business sites: consider charges for proprietary catalogs, specifications, or market data, subject to supplier agreements.
  • Platforms with user-generated content: need to check whether their contracts permit charging for contributors’ work. Hosting content does not automatically give the platform the right to license it.

An individual creator on a third-party platform generally cannot use Pay Per Crawl to claim a share of every AI use of their posts. Cloudflare’s system does not identify individual authors after the fact or distribute royalties among contributors.

Rank #2
Sale
UGREEN NAS DXP2800 2-Bay for Advanced Home Users, Remote Workers & Creators
  • 【Advanced Home Data & Media Hub】For advanced home users who need phone backup, file storage, and centralized data management. Centralize family photos, 4K videos, movies, computer backups, and personal files in one place while running multiple apps for home entertainment and everyday data management. Suitable for households with growing digital libraries and multiple NAS use cases.
  • 【Built for Creators, Media Servers & Advanced Apps】Powered by the Intel N100 Quad-Core CPU, 8GB DDR5 RAM, 2.5GbE networking, and dual M.2 NVMe slots, DXP2800 handles large files and heavier workloads with ease. Run Docker, virtual machines, and media server applications compatible with Plex—ideal for content creators, tech enthusiasts, and advanced home users managing 4K videos, RAW photos, personal media libraries, and multiple NAS apps.
  • 【Up to 80TB for Growing Digital Libraries】 Supports up to 80TB of storage using two HDD bays and two M.2 NVMe SSD slots for family photos, movies, RAW photos, 4K videos, work files, and device backups. AI photo management supports recognition of people, objects, scenes, and locations, album organization, and duplicate photo detection. HDDs and SSDs are not included.
  • 【AI-powered Home Surveillance】Turn DXP2800 into a centralized home surveillance hub by connecting compatible network cameras and storing recordings locally on your NAS. AI-powered features include Face Recognition, People Detection, and Pet Detection, helping advanced home users review important events more efficiently while managing home surveillance and personal data in one place.
  • 【One data Center Across Your Devices】Keep files from desktops, laptops, phones, tablets, and other devices together instead of scattered across cloud accounts and external drives. Access, back up, organize, and share data across Windows, macOS, Android, iOS, web browsers, and compatible smart TVs—ideal for creators and advanced home users working across multiple devices.

What is paid for—and what is not proved

The current charge is for a successful retrieval. It does not establish:

  • that the crawler used the page to train a model;
  • that a model cited or reproduced the page;
  • how many tokens were extracted;
  • that the material was legally protectable or exclusively yours;
  • that an AI company earned revenue from the page; or
  • that the payment grants a broad copyright or model-training license.

A training crawl, a search crawl, an agent reading live information, and a commercial dataset license are different uses. Pay Per Crawl operates at the access and transaction layer. It does not audit what happens after delivery.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Current pricing, billing, and payouts

Cloudflare’s documentation lists a minimum site-owner price of $0.001 per crawl. That is a floor, not an earnings estimate: price-setting documentation.

Item Current documented position
Price Site owner sets the price; minimum $0.001 per crawl.
Different crawler prices The FAQ describes one price for all crawlers configured for charging.
Free paths /robots.txt, /sitemap.xml, /security.txt, /.well-known/security.txt, and /crawlers.json remain free.
Payout account A dedicated Cloudflare Stripe Connect account is required; an existing Stripe account cannot simply be reused.
Settlement Cloudflare acts as merchant of record and documents aggregated monthly payouts for publishers in good standing; settlement periods and minimum thresholds apply, but the cited page does not state exact thresholds.

The payout setup is documented at Manage payouts, with the dashboard path Manage Account > Settings > Pay Per Crawl > Connect.

A simple illustration, not a forecast

At $0.001 per successful crawl, one million paid retrievals would equal $1,000 before any applicable commercial, settlement, or tax considerations. Actual income depends on crawler volume, the price you set, whether important AI companies support the protocol, and how often pages are recrawled.

Important limits for personal-finance planning

Do not treat a Cloudflare price setting as passive income. AI companies can decline to pay, avoid the page, negotiate elsewhere, or use crawlers that do not support the protocol. There is no published benchmark showing that typical small sites earn meaningful or predictable revenue.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #3
Sale
ZimaBoard 2 Home Server, Intel N150, Build Your First Real Server
  • Server-Class Home Server Built for 24/7 Workloads - Designed as a purpose-built home server rather than general-purpose SBCs, Mini PCs, entry NAS systems, or routing-only devices. As a compact, pocket-sized single board server platform, ZimaBoard 2 832 combines x86 architecture, quad-core performance up to 3.6GHz, 8GB DDR5 memory, and 32GB eMMC storage for reliable always-on home servers, homelabs, and self-hosted workloads.
  • PCIe 3.0 x4 Expansion for Real Server Builds - Built as a server-class platform with native PCIe expansion, ZimaBoard 2 features a full PCIe 3.0 x4 slot for high-speed, low-latency upgrades beyond USB-based limitations. Supports 10GbE NICs, NVMe adapters, GPUs, and AI accelerators to build scalable home servers, homelabs, and advanced self-hosted systems—offering greater expansion flexibility than typical SBCs, Mini PCs, and entry-level NAS devices.
  • Native Dual SATA & Dual 2.5GbE Networking - Built with server-class storage and networking I/O, ZimaBoard 2 integrates dual SATA ports for direct HDD/SSD connectivity and dual 2.5GbE Ethernet for high-throughput, low-latency networking. This architecture enables reliable DIY NAS, fast storage, routing, and multi-service home server deployments—while avoiding USB-based performance constraints common in ARM SBCs, Raspberry Pi–based setups, Mini PCs, and entry-level NAS devices.
  • ZimaOS Preinstalled + Wide OS Compatibility - Comes preinstalled with ZimaOS for a clean, ad-free private cloud experience—centralized file dashboard, automatic backups, P2P downloads, private photo/video sharing, 500+ plug-ins, and secure on-device AI that keeps your data at home. Also supports TrueNAS, Proxmox, Debian, Ubuntu Server, pfSense, OpenWrt, and Linux containers, making it perfect for Plex media servers, Pi-hole, firewalls, backups, Docker labs, home-cloud services, and multi-service deployments.
  • All-in-One NAS, Router, Docker & Homelab Server - Replace multiple devices with one low-power, fanless system. ZimaBoard 2 can serve as a NAS, router, Docker host, firewall, media server, or homelab node—delivering a flexible, open alternative to ARM SBCs, Mini PCs, and entry-level NAS systems.

Repeated access can create repeated charges. A recrawl of an unchanged page and a first retrieval may both be billable unless you use exceptions or dynamic pricing. Cloudflare’s FAQ also says that WAF or Bot Management blocking rules override a charge setting, so a crawler you intended to monetize may simply be denied: Pay Per Crawl FAQ.

Dynamic pricing and technical controls

A June 2026 update added URI-pattern exceptions, dynamic pricing, an origin or Worker-provided crawler-price response header, and a cf-pay-per-crawl request header when dynamic pricing is enabled. Because the product is beta, verify the current implementation before building a business process around it: Cloudflare changelog.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Risks publishers should assess

Search visibility can suffer

A rule aimed at training bots can affect discovery or referrals when companies use overlapping infrastructure for search, training, and agent functions. Review logs and test policies before broad enforcement.

Classification is imperfect

Crawler identity and purpose can change. Misclassification can charge a useful search bot, block a valuable referral, or let an unwanted crawler through.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

You may not own every returned item

Licensed photographs, syndicated articles, guest posts, user comments, database records, and embedded feeds can carry restrictions. Charging for a page does not resolve those underlying rights.

Past copying is outside the system

Pay Per Crawl controls future requests reaching your domain. It cannot determine whether a model was trained on historical copies or force retroactive compensation.

Rank #4
Blackmagic Design Web Presenter HD Bundle with Power Cord and HDMI Cable with Ethernet, 3 Feet
  • SDI Video Inputs: 1
  • SDI Video Outputs: 1 x loop out, 1 x monitor out.
  • SDI Rates: 1.5G, 3G, 6G, 12G
  • HDMI Video Outputs: 1 x monitor out
  • Webcam Output: 1 x Type USB-C

Cloudflare cannot decide legal questions

The system does not determine fair use, copyright infringement, dataset-license validity, or whether a payment creates a sufficient training license. Sample AI-related terms are available from Cloudflare, but they are not a substitute for jurisdiction-specific legal advice: sample terms.

Cloudflare’s announced next steps

Cloudflare’s broader Pay Per Use experiments explore compensation tied to value generated by content—for example, when it appears in an AI search result—rather than charging every fetch. Those experiments are not a generally available payout program: Pay Per Use announcement.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Cloudflare has also announced that, on September 15, 2026, new customers and new sites created by existing customers will have a default in which ad-displaying pages allow search crawlers but block training and agent-use crawlers. As of August 18, 2026, that is a future change, not the current default: Cloudflare announcement.

Alternatives to Pay Per Crawl

Approach What it does Main trade-off
Block AI crawlers Stops selected requests at the edge. Creates no revenue and may reduce discovery.
Robots.txt and site terms Expresses access preferences and policies. Robots.txt is not authentication, payment, or a guaranteed legal remedy.
Paid API Provides authenticated, structured data with quotas. Requires engineering and ongoing maintenance.
Direct licensing Negotiates dataset scope, duration, uses, attribution, and compensation. Can cover historical or defined datasets but takes time and bargaining power.
Licensing marketplace or collective Aggregates rights or negotiates standardized deals. Terms, coverage, and buyer demand vary.
Feeds or syndication Delivers selected content under agreed conditions. Less control than an authenticated API and may involve separate contracts.

How to decide whether to apply

  • Use AI Crawl Control first to measure which crawlers actually reach your site.
  • Inventory who owns the rights to every important page and asset.
  • Decide whether search traffic, AI referrals, training access, or payment is your priority.
  • Model revenue using several crawler-volume assumptions; do not assume the $0.001 floor creates demand.
  • Test policies against search, referrals, site functions, WAF rules, and caching behavior.
  • Apply only if you can complete Cloudflare and Stripe verification and accept closed-beta uncertainty.

Bottom line for website owners

Cloudflare is creating a potentially useful payment and enforcement rail for AI-era web access, but it is not a universal “get paid for your data” fund or a royalty system for past training. The practical opportunity is experimental: observe crawler demand, charge compatible bots for future successful retrievals, and use direct licensing or APIs when your data warrants negotiated terms. The decisive question is whether the AI companies that matter will pay rather than leave.

Frequently Asked Questions

Can I get paid if an AI model already trained on my articles?

No. Pay Per Crawl governs future access requests at participating domains; it does not identify historical training use or create retroactive compensation.

Can a creator charge for posts hosted on another platform?

Usually not through this mechanism unless the creator controls the domain and has the contractual rights to authorize paid access. A platform’s ownership and contributor terms must be checked.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Will every AI crawler be forced to pay?

No. Payment requires crawler support for Cloudflare’s authentication and payment protocol. A company can refuse, stop crawling, or be blocked.

Quick Recap

Bestseller No. 4
Blackmagic Design Web Presenter HD Bundle with Power Cord and HDMI Cable with Ethernet, 3 Feet
Blackmagic Design Web Presenter HD Bundle with Power Cord and HDMI Cable with Ethernet, 3 Feet
SDI Video Inputs: 1; SDI Video Outputs: 1 x loop out, 1 x monitor out.; SDI Rates: 1.5G, 3G, 6G, 12G
$593.00

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More post from the Money Desk

  1. The Money DeskBlogTheFinanceBase07 MAR 2625 minWhat Is a 457 Plan?
  2. The Money DeskBlogTheFinanceBase07 MAR 2621 minTime Value of Money: What It Is and How It Works
  3. The Money DeskBlogTheFinanceBase07 MAR 2627 minAre You Living in One of These Top 10 Most Expensive Cities to Retire?
Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
PC Slower Than It Used to Be?Free scan - under a minute

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.