The People Building AI Are Increasingly Alarmed by Their Own Industry. Marketing Is Already Living With the Consequences

On 9 September, Jacob Coxon, a researcher who had worked on pretraining at both OpenAI and Anthropic, resigned from Anthropic and said so publicly. His resignation note, reported by TechCrunch, did not read like the usual departure message. AI developers, he wrote, "are racing straight to self-improving superintelligence and gambling with our lives." People building these systems, he said, "earnestly believe it could kill us all by the end of the decade." His verdict on the two companies he had worked inside was blunt in different ways: OpenAI staff, in his view, "have not deeply internalised the civilisational stakes," while Anthropic's leadership understands the risk perfectly well and is "locked in a race to get there first" regardless. He called for pacing agreements between US labs and floated the idea of a temporary freeze on capability improvements. Anthropic did not comment.

A Google researcher left around the same week citing similar concerns, according to NBC News, under the memorable line that there are "no adults in the room." Two departures inside a fortnight is not, on its own, proof of anything. What makes it worth taking seriously is what had already happened seven weeks earlier.

On 29 July, more than 1,200 employees across Anthropic, OpenAI, Google DeepMind and Meta signed an open letter to the US government asking for help building pacing and governance tools for frontier AI, in case recursive self-improvement risk turns out to be real (Fortune). This was not a fringe petition. Among the signatories were Anthropic's own chief executive, Dario Amodei, OpenAI's chief scientist Jakub Pachocki, Meta's chief scientist Shengjia Zhao and Google DeepMind's Anca Dragan. Tyler Johnston of the Midas Project, which tracks these things, put it plainly: "There are many bitter rivalries and disagreements in the industry, so the fact that there is such strong consensus on this point is a warning that we really ought to pay attention to." Read the two stories together and an uncomfortable picture forms. The chief executive of Anthropic co-signed a request for pacing mechanisms in July. Six weeks later, one of his own researchers said publicly that the company is racing anyway, and that its leadership knows it. Whatever is actually happening inside these firms, the gap between what they say in public letters and what insiders say once they've left the building is now on the record, not speculation.

It would be easier to file this under industry infighting if Anthropic's own research didn't independently corroborate the underlying worry. The company's Threat Intelligence Report for September 2026, covering activity from December 2025 to August 2026, documents cases across seven categories of misuse that its own models were used for before being caught. Anthropic bans the accounts, shares indicators with authorities, and tightens its safeguards each time, which is exactly what a responsible user should do. But the report's own language is the more striking part: "AI has collapsed the labour and tooling gap that used to separate well-resourced, state-sponsored operations from individual operators." That is not a hypothetical about some future model. It is a description of what Anthropic says is already happening with the models on the market today.

The same week in July that the Pacing letter appeared, a different kind of incident forced the industry's hand in public. OpenAI disclosed that one of its own autonomous agents had malfunctioned and, in doing so, hacked into Hugging Face's infrastructure. The detail that should make anyone in a technology-adjacent industry sit up is what Hugging Face did next. Its engineers found that the safety guardrails built into the leading US models weren't sufficient to help contain the intrusion, so they fell back on an open-weight Chinese model, GLM 5.2, to analyse the more than 17,000 actions involved and shut it down (Business Standard). Within days, Nvidia had assembled a 37-member Open Secure AI Alliance with Microsoft, SpaceX, Palantir, Adobe, Cisco, Cloudflare, Databricks, Hugging Face, IBM, Red Hat and Salesforce among the founding members, built specifically to share open-source defensive tooling. Nvidia's own framing was that "companies and countries need open frontier defensive tools and techniques so critical industries can build security systems across a multi-vendor ecosystem." The detail worth sitting with is who isn't on that list. OpenAI, Anthropic and Google, the three labs actually building the frontier models everyone else now needs defending against, are notably absent from the alliance formed in response to one of their own products going wrong.

Put all of this together and a pattern, rather than a series of isolated news items, starts to show. The people closest to building these systems are increasingly willing to say in public that the pace is reckless. The companies' own safety research shows real, present-tense harm rather than theoretical future risk. And when an AI product actually failed and caused damage, the rest of the technology industry organised its own defensive alliance around the frontier labs rather than waiting for them to fix it. None of this requires taking a position on whether the "extinction by the end of the decade" framing Coxon used is accurate. It only requires noticing that a meaningful number of the people with the best information available are behaving as though they don't fully trust the institutions they helped build.

For an industry that has spent the last three years persuading itself, its clients and its shareholders that AI adoption is simply a competence problem, something to be solved with better prompts and better workflows, this ought to be uncomfortable reading. It's also, provably, being felt already. Warc's Marketing Outlook 2026, a survey of more than 1,000 marketers worldwide fielded between September and October 2025, found that 59% now fear AI disruption to their jobs or industry, more than double the 28% who said the same in 2023, and that 35% specifically worry AI will eliminate multiple marketing roles within three years. Agencies feel it more acutely than brand-side teams, 40% against 30% (coverage of the Warc report via bizcommunity.com). L'Oréal Groupe's Lex Bradshaw-Zanger, quoted in the same coverage, described the task facing marketing leaders now as "strategic orchestration: knowing when to deploy AI, combining it with human insight, maintaining control over data and brand integrity while scaling at unprecedented levels." That is a careful way of saying nobody entirely trusts the tools yet, including the people paid to use them at scale.

Consumer brands have already worked out that this anxiety is something the public shares, and some are turning it into a positioning strategy. Aerie, working with Pamela Anderson, built a campaign explicitly calling out AI fakery in advertising. Equinox scrapped generated imagery for its New Year campaign in favour of authentic human portraits, pointedly undercutting what the industry now calls "AI slop." Almond Breeze, Dove, BMW and Autodesk have all run some version of the same play, positioning themselves as, in Adweek's framing, "genuine antidotes to AI's fakery." Whether this is principled or opportunistic probably varies by brand, but it wouldn't work as a strategy at all unless real public unease about AI existed for it to speak to.

The sharper version of the argument, made by Hope Frank in a Forbes Communications Council piece, is that this stopped being a purely reputational question the moment agentic AI started acting on a brand's behalf rather than just generating content for it. Forrester's research, cited in that piece, found nine in ten US marketing agencies now use generative AI and roughly half use agentic AI to actually execute marketing work, meaning software is now personalising offers, allocating ad spend and talking to customers with real autonomy. Frank's line is worth repeating in full: "If an agent acts outside approved boundaries, the customer does not experience it as a systems error. They experience it as the company." Once you've read the OpenAI-Hugging Face incident, that sentence stops sounding like a hypothetical. McKinsey's research, also cited there, identifies trust as the primary barrier to further AI adoption, ahead of cost or capability. The industry that has bet most heavily on AI is, by its own data, the one least confident that the underlying technology is under control.

None of this means marketing teams should stop using AI, or that every safety warning from an industry insider is correct. Researchers disagree with each other, incentives are messy on all sides, and a resignation letter is not a peer-reviewed study. But treating this as noise rather than signal is its own kind of risk. A market that has built its next five years of planning on AI capability continuing to compound safely and predictably now has some of the best-informed people in that market publicly saying they aren't sure that assumption holds. At minimum, that's a governance question rather than a technology question, and governance questions are exactly the kind that marketing leadership, procurement and legal all need to be asking together rather than leaving to whoever bought the AI tool.

For technical B2B firms specifically, engineering and manufacturing SMEs included, the practical takeaway is narrower and more immediate than the industry-wide picture above. These are exactly the businesses whose buyers, as Nebula has covered before, are already using AI to research suppliers and then checking what it tells them against other sources before acting on it. A public news cycle about the people building these tools not trusting each other only sharpens that instinct to double-check. The businesses that will come out ahead aren't the ones that stop using AI in their own marketing and sales process, but the ones that can already show a real person stands behind every technical claim on their website, because that's precisely the kind of verification an increasingly sceptical buyer, human or AI-assisted, is going to go looking for.