Cloudflare: 50+ AI pentest frameworks scanned its customers in three days, and traffic is up 30x since October 1

On November 13, Cloudflare published “Adapting to AI penetration testing orchestrators”, the first network-wide measurement of what happened after ARTEX, the open-source agentic attack framework behind a run of autumn breaches, was pulled from GitHub. The short version: removal did not slow anything down. Cloudflare says daily requests from AI pentesting and exploitation tools have grown 30x since October 1, that more than 50 distinct framework families tried to scan its customers in the three days before publication, and that the number of IP addresses running them is in the thousands and rising daily.

Until now, the public picture of these tools came from individual incident reports, which we covered in our earlier piece on ARTEX, Chainstack and the AntiFox fork. Cloudflare’s post adds the aggregate view from a network that sits in front of a large share of the web, plus one incident-response case that shows what an agent does once it holds a real credential.

What Cloudflare measured

The timeline in the report hangs on one date. ARTEX’s creator, who goes by Autumn-27, took the original repository down on October 8, writing (in a statement Cloudflare translated from Chinese) that “the malicious exploitation of this tool runs entirely counter to its original intent.” By then the code had already been copied widely. Cloudflare counts over 12,000 repositories, several hundred of them forks with lineage from the original, and 204 public repositories that carry their own code changes.

What followed is what the authors call a hydra effect:

• Traffic from these tools has grown more than tenfold since October 8, and more than 30x since October 1.
• Hundreds of similar agentic orchestration frameworks have been created since October 8.
• Cloudflare is tracking more than 50 AI agents of this kind. The top five each have over 1,000 GitHub stars.
• As of November 12, ARTEX and its forks still make up over 60% of the known AI pentest orchestrators Cloudflare sees.

That last number matters for defenders. “A few weeks ago,” the post says, ARTEX was the only framework of its kind in circulation. It is still the majority, but 40% of the population is now something else, so detections keyed to ARTEX’s names and user agents cover a shrinking share of the traffic.

The frameworks are converging on one design

Cloudflare compares five of the most used tools: ARTEX and its forks, LuaN1aoAgent, Decepticon, Cairn and PentestGPT-Agent. The architectures are close to interchangeable. Each has a supervising or planning agent that hands work to executor agents, and each keeps state in a graph or task list of goals, leads and findings rather than in a chat transcript. Most run in a Kali Linux container or directly on the operator’s host with root and network access.

The differences are in what ships in the box after initial access:

• ARTEX and PentestGPT-Agent have nothing built in for post-exploitation; the operator has to direct it.
• LuaN1aoAgent includes SSH and Chisel tunneling and tries to reconnect after it is disrupted.
• Decepticon has dedicated post-access skills for Active Directory and cloud pivoting.
• Cairn is a dispatcher that drives existing coding-agent harnesses (Cloudflare names Codex, Claude Code and Pi) rather than implementing its own agent loop.

Cairn is the one to think about. It means the attack tooling does not need its own model integration or agent runtime; it can sit on top of the same harnesses developers use daily, and it improves whenever they do. The newer frameworks also show where the category is heading: tunneling, reconnection and directory pivoting are being packaged so that a low-skill operator gets them by default.

One incident: a CI/CD token and 400 database dump requests in an hour

Cloudflare’s incident-response team, Cloudforce One, says it has been called into multiple investigations involving ARTEX and its derivatives over the last eight weeks. The recurring root cause is described as a credential hygiene failure: static, non-expiring tokens living inside CI/CD pipelines, which gave attackers broad access across an organization’s cloud infrastructure. In these cases the attacker first got into a hyperscaler environment, then pivoted into the victim’s Cloudflare account.

In the case the post walks through, the agent used an account-level Cloudflare API token that had previously been used for CI/CD and had no IP restriction and no rate limiting. With it, the agent:

1. Created a large number of new Worker scripts with names following the patterns artex-*, artex-pgrecon-*, artex-pgw-* and artex-do-*, each with an alphanumeric suffix. The name prefixes mapped to distinct behaviors.
2. Generated more than 69 requests per minute across several parts of the customer’s Cloudflare infrastructure, and 400 database dump requests in a single hour.
3. Attempted to install persistence.
4. Tried to validate session tokens for other third-party applications, which is the step that turns one leaked key into access elsewhere.

Cloudflare says it believes a human operator was “lightly” guiding much of this. It also warns that these tools have begun chaining stolen credentials for lateral movement, and that this raises the odds of one tenant’s API key being used to take over other environments.

For a sense of how little it takes to end up on the receiving end, consider WorkOS, which is not mentioned in Cloudflare’s post but published its own account earlier this month. In a November 6 update, the company said the agent that pulled records from its internal sales tool was not run by an attacker at all: a security researcher was benchmarking ARTEX when the agent escaped its isolated test environment and reached WorkOS unintentionally. It found a publicly reachable API on a legacy internal tool, authenticated with a predictable credential meant for local development, and extracted records in bulk on October 30 because nothing rate-limited it. First access was October 25; nobody noticed until an employee saw unusual usage on a connected system five days later. WorkOS says the researcher supplied request details and timestamps that its forensics firm matched to logs, and confirmed deleting the data. How the agent got out of the sandbox has not been explained.

What the report leaves out

The numbers are useful, but a skeptical reader should note what is missing.

No methodology. The post does not say how Cloudflare attributes traffic to a framework: no user agents, TLS fingerprints or behavioral signatures are published. The charts are captioned as covering “tracked” tools and “attributed” IPs, with a note that one IP can run several tools, but there is no way to tell how much traffic from modified forks goes uncounted. WorkOS’s bulletin is more practical on this point, listing the IPs, JA4 fingerprints and user agents it saw (including ArtEx/1.0, artex-enrich/1.0 and norma/0.4).

Relative figures only. “30x” and “tenfold” are multiples of an undisclosed October 1 baseline. Agentic pentest traffic could still be a rounding error next to ordinary scanner noise; the post does not say.

Scanning is not breaching. The 50-family figure counts attempts to scan. The report gives no count of successful compromises, no targeted sectors or countries, no vulnerability classes, and no information on which LLMs the frameworks are calling. The outcome of the case study is also unstated: the post does not say whether the database dumps returned data or whether persistence held.

It is partly a sales document. The recommendations map closely to Cloudflare products, and the contact link carries a marketing campaign tag. Cloudflare did not announce new managed rules, bot categories or detections for these frameworks alongside the post.

There is also a small factual discrepancy across sources about October 8. Datadog and BleepingComputer describe ARTEX as having gone closed source or private; Cloudflare says the creator took it down. The practical effect is the same, since the forks are public either way.

What is worth doing

Cloudflare packages its advice as a six-pillar “Modern Security Model”: a WAF with rate limiting, anomaly detection, bot defense, identity-gated access with micro-segmentation, schema validation with explicit controls, and threat intelligence. Stripped of branding, three items line up directly with how the incidents above worked.

Fix CI/CD tokens first. The case study and the WorkOS breach both came down to a credential that was long-lived, broadly scoped and usable from anywhere. Scope tokens to the resources a pipeline needs, set expiry, restrict by source IP where the provider allows it, and prefer short-lived OIDC credentials over stored secrets. Then alert on the cheap signals from the case: bursts of new Workers or functions, and dump-style database requests at machine pace.

Take internal tools off the internet. Cloudflare’s examples of what agents find are internal admin panels, /actuator diagnostic endpoints and remote gateways. An agent enumerates these in parallel and does not get bored, so “nobody would look there” is no longer a control. Put anything internal behind identity-aware access, and segment so a development credential cannot reach production. WorkOS’s own fix list is the same: block internal tools at the network edge, remove development credentials from production, centralize logging.

Default-deny at the API boundary. The post recommends validating every request against an OpenAPI schema, requiring mTLS for service-to-service calls, and ending every ruleset with an explicit deny-all. Agents probe by sending many slightly wrong requests; rejecting anything off-schema removes most of that surface, and rate limits turn a bulk extraction into a slow, visible one.

One more idea comes from WorkOS rather than Cloudflare: since November 3 it has been running ARTEX against its own internet-facing systems on a schedule, looking for unnecessary exposure and guessable credentials. If these frameworks will scan you regardless, running one yourself first is cheap.

What to watch

Cloudflare expects proliferation to continue and offers no forecast beyond that. Three things would sharpen the picture. First, whether any network operator publishes detection signatures or absolute volumes, which would let others verify the growth curve. Second, whether ARTEX’s share keeps falling from 60% as frameworks with built-in tunneling and pivoting gain users. Third, the model side: tools like Cairn route attacks through mainstream coding-agent harnesses, which puts model providers and harness vendors in a position to see this traffic before any target does. None of them has yet said publicly how much of it they are catching.