Online bot traffic will surpass human traffic by 2027, says Cloudflare CEO

Bots are taking over the web, according to Cloudflare CEO Matthew Prince. In an interview at the SXSW conference in Austin this week, he said that at the rate at which artificial intelligence is developing, AI bot traffic will exceed the number of people online by 2027.

Prince explained that the use of web bots has been increasing in line with the growth of artificial intelligence technology because bots are able to visit more sites to find answers to chatbot users’ questions.

“If someone is doing a task — let’s say you’re shopping for a digital camera — and you might go to five websites. Your agent or bot that does that will typically go to 1,000 times the number of places a real person would visit,” Prince said. So it could go to 5,000 sites. And that’s a real density, and that’s a real burden, that everybody has to deal with and consider.”

Before the production era of AI, the Internet was about 20% of bot traffic only, the Google web search engine being the biggest, according to Prince, his infrastructure and security company used by one fifth of all websites. But beyond some reputable crawlers, the only other bots were those used by scammers and bad actors.

“With the rise of productive AI, and its insatiable demand for data, we are seeing an increase where we suspect that, by 2027, the amount of bot traffic online will exceed the amount of people online,” Prince said.

The official also noted that this change to the web will require the development of new technologies, such as sandboxes for AI agents that can be spun up on the fly and then dismantled when their work is done. This may come into play when consumers ask AI agents to perform specific tasks for them, such as planning a vacation.

“What we’re trying to think about is, how can we actually build that infrastructure where you can – as easily as you open a new tab in your browser – you can actually run new code, that can run and run existing agents,” Prince said.

He thinks there will soon be a time when millions of these “sandbox” agents will be created every second.

Of course, online use of bots on this scale would require physical infrastructure in the form of data centers and servers. Prince pointed out that, during Covid, Internet traffic grew so fast, especially among video broadcasters such as YouTube, Disney, and Netflix, that some parts of the Internet were almost under strain.

“This [growth] it’s slow and steady, but unlike COVID, where it peaked in two weeks and then it hit a new high, we’re seeing internet traffic grow and grow and grow, and we don’t see anything that’s going to slow it down or stop it,” Prince added.

All of this concern about overloading is good marketing for Cloudflare, a company whose services focus on helping websites stay more available, load faster, and stay safe from attacks. Among its offerings are a content delivery network, a series of security and DDoS protections, and “Online Always” technology that provides cached versions of websites when the main server fails or is offline. It also provides businesses with tools to block unwanted AI bot traffic.

However, Cloudflare’s scale gives it the opportunity to be able to observe the ongoing evolution of the Internet and the rapidly emerging challenges facing the AI ​​era of manufacturing.

“I think the thing people don’t like about AI is the platform shift,” Prince said, recalling previous platform shifts, such as from desktop to mobile. “AI is another platform shift … the way you’re going to use information is completely different.”