Cloudflare has introduced a new pay per crawl feature that allows website operators to charge AI developers for accessing their content.
Website operators can set a flat fee for AI bots to access their content, with the ability to customize settings for specific crawlers if needed.
Cloudflare's pay per crawl feature uses public key cryptography to verify AI crawl requests and prevent malicious activities like impersonation.
The company also launched other AI-focused capabilities, including default blocking of AI crawlers and a setting to only block crawlers from accessing pages with ads.