AI Plundering Web Content: How Businesses Can Regain Control
Updated: October 16, 2025, 10:37 AM
Published: October 7, 2025, 08:00 AM
A growing number of AI companies are scouring the internet for content to train their models – often without permission. Michael Gustafsson of Cloudflare points out that many website owners lack insight into who is visiting their sites and why.
“That’s why we developed the AI Audit tool, which reveals what’s happening behind the scenes and gives users full control,” he says.
Generative AI has exploded in popularity, but this growth comes at a cost. To build intelligent models, AI companies use vast amounts of data from web pages, frequently without seeking consent. This, explains Michael Gustafsson, Strategic Solutions Engineer at tech company Cloudflare, has created concern among content creators, media houses, and businesses whose digital content risks becoming free fuel for others’ profits.
“We are in a paradigm shift: instead of Googling, users stay in AI services, which reduces direct traffic to sites and therefore control over content becomes crucial – especially for those with ads, paywalls, whitepapers or memberships. In addition to the risks of data leakage and violations of copyright or data protection regulations, AI traffic also means increased server costs, often without business value, as each request must be processed. So it’s not just about technology and transparency, but also about law and economics,” he says.
Who is Visiting Your Website – and When?
Cloudflare’s new AI Audit service gives companies the ability to see in real-time exactly who is trying to retrieve content from their websites. A clear overview allows users to track which AI crawlers are visiting the site, how often they do so, and whether they are following the rules.
“AI Audit creates both transparency and the ability to act. You can see whether the traffic is human or from AI and which pages are scraped the most, and can thus block bots or create your own rules. It should be standard practice for AI actors to identify themselves clearly and state their intentions, but that is far from reality today. Many actors instead retrieve content without asking, like taking goods from a store and then selling them on.”
Paid Service in Beta Testing
In the near future, Cloudflare will also offer the option to charge for access to content – something that is currently being tested in a private beta. Michael Gustafsson notes that this will give companies a way to actually make money from what AI companies otherwise take for free.
“This is a way to regain control, to decide for yourself how your content is used and who gets to make money from it. For companies with exclusive content, this is not something that can be ignored; within one to two years, it will not be a choice but a requirement,” he concludes.
About Cloudflare
Cloudflare is a global tech company that protects and optimizes over 40 million websites with services such as DDoS protection, performance optimization, and API security. Through its global network, they can detect and block malicious traffic in real time.
This article was produced by Brand Studio in collaboration with Cloudflare.
Enjoyed this post by Thibault Helle? Subscribe for more insights and updates straight from the source.
