Cloudflare, Inc. (NYSE: NET), the leading connectivity cloud company, has announced new classifications, enhanced analytics, and commercial partnerships aimed at supporting the Cloudflare agentic Internet. The latest updates are designed to help website owners and AI companies improve discoverability, efficiency, and monetization while creating a transparent ecosystem for AI powered web interactions.

The company said the new capabilities establish the infrastructure needed for a sustainable agentic economy. They also provide website owners with greater control over how AI systems access and use their content. According to Cloudflare, the initiatives are intended to balance the interests of content creators, publishers, and AI companies.

Cloudflare noted that Internet traffic has changed significantly, with automated agents and bots now generating more than half of all web requests. As AI increasingly becomes the interface for searching information and conducting online transactions, businesses face new challenges around protecting intellectual property while remaining discoverable.

The company said many website owners want their content to appear in AI systems. However, publishers that depend on advertising or subscriptions also want to prevent AI companies from using their content for training without compensation. To address this, Cloudflare is introducing new default classifications, deeper analytics, faster AI search capabilities, and commercial frameworks that allow creators to earn revenue when their content contributes to AI generated answers.

Matthew Prince, Co-founder and CEO of Cloudflare, said, “Last year we provided site owners with transparency and control over what bots access their content, and we are thrilled with the benefits it has had to the ecosystem. Now that the majority of traffic on the Internet is non-human, we must go further and act faster so that a sustainable ecosystem can emerge. Cloudflare’s new tools and partnerships give website owners increased visibility and commercial opportunities and benefit AI companies that have bots with clear and transparent intent. We hope that our proposed default changes encourage mixed use crawlers to separate out search from agent use and training.”

Meanwhile, Cloudflare announced plans to introduce new default settings for mixed use AI crawlers. Following discussions with publishers and AI companies, the company concluded that most website owners want AI discoverability but also want greater control over AI training access.

Over the next two months, Cloudflare will collect industry feedback before finalizing the changes. Beginning September 15, 2026, new customers and new websites will automatically allow AI search while blocking AI training and agent use on advertising supported pages. Mixed use crawlers that do not separate search, agent use, and training functions will also be blocked on those pages. Customers will retain the ability to modify these settings through their dashboards.

The same default settings will also apply to existing free customers who have not updated their preferences before September 15, 2026. Cloudflare said these measures are intended to create a fairer environment for both AI companies and publishers.

In addition, the company introduced the Attribution Business Insights dashboard. The new analytics platform gives publishers visibility into how AI bots access their content and measures the amount of referral traffic returned by individual AI companies. Cloudflare said the dashboard is designed for business decision makers as well as IT and security teams.

Cloudflare also introduced the concept of Answer Engine Optimization (AEO). The company described AEO as the next evolution of content optimization, helping organizations understand how frequently their content appears in AI generated responses, how prominently it is cited, and how it performs across different AI models.

At the same time, Cloudflare is working to improve AI search efficiency. According to the company, more than half of AI crawler traffic repeatedly fetches unchanged web pages. By identifying when content has actually changed, Cloudflare plans to reduce unnecessary crawling, lower bandwidth consumption for publishers, decrease compute costs for AI companies, and improve the freshness of AI generated responses. The company is currently testing these capabilities with leading AI providers and expects wider availability later this year.

Cloudflare is also expanding its monetization model. A year after launching Pay Per Crawl, the company is evolving the service into Pay Per Use. Under the new approach, publishers will be compensated when their content generates value in AI systems rather than simply when it is crawled.

The company announced partnerships with Ceramic.ai and You.com to support this commercial framework. Ceramic.ai will compensate publishers whenever their content appears in its AI search results while also providing visibility into queries, citations, and rankings. Meanwhile, You.com plans to allow AI agents to pay for specific premium content when needed. Cloudflare said it will provide the common infrastructure supporting both models.

The latest announcements build on Cloudflare’s Content Independence Day initiatives introduced last year. Since then, the company has launched AI Crawl Control, partnered with beehiiv to simplify AI crawler management for creators and publishers, developed the Web Bot Auth framework to verify AI bots, and collaborated with payment providers to enable secure AI agent transactions across millions of merchants.

Several industry partners welcomed the initiative.

Tyler Denk, CEO of beehiiv, said creators need greater control over their content while maximizing visibility across emerging AI search platforms.

Anna Patterson, Founder and CEO of Ceramic.ai, said Cloudflare’s infrastructure enables transparent, scalable AI search while ensuring publishers receive compensation whenever their content appears in search results.

Geoff Campbell, SVP Strategy & Business Development at Condé Nast, said Cloudflare has continued to support premium publishers by advocating for fair compensation and providing technical infrastructure for the evolving AI ecosystem.

Drew Rowny, SVP of Product at Patreon, said creators should have meaningful control over how AI companies use their work. He added that Patreon is expanding its collaboration with Cloudflare to block AI training crawlers while allowing search crawlers that help creators grow their audiences.

With these latest announcements, Cloudflare agentic Internet initiatives continue to expand beyond content protection into discoverability, analytics, AI search optimization, and AI content monetization, positioning the company to support a more transparent and sustainable AI driven web ecosystem.

Related post

View all