Individual objections are easy to ignore, and employee leverage shrinks as models improve. That's why the @utaw.tech / @cwunews.bsky.social recognition effort at Google DeepMind matters: independent ethics oversight, whistleblower protection, and a right to refuse projects on moral grounds.

Demis Hassabis then made a different bet: "Take the energy that was going into the trustless negotiation and put it into creating real trust." And: "Safety isn't about governance structures." Trust instead of governance. The Pentagon contract is the litmus test of that bet.

None of this is new. DeepMind's governance record: 2014: independent ethics board — reportedly a condition of the sale 2015: one informal meeting; then effectively abandoned 2018: AI Principles exclude weapons & surveillance 2025: exclusions dropped 2026: Pentagon contract

Google says it is committed to the consensus that AI "should not be used for domestic mass surveillance or autonomous weaponry without appropriate human oversight." "Should not" is not "must not." If these uses were truly prohibited, saying so would have taken fewer words.

I work at Google DeepMind. This won't make me popular. But it's all public reporting: 2014: DeepMind reportedly sold to Google on conditions: no military use, independent oversight 2026: a Pentagon contract for "any lawful government purpose" Not one safeguard survived intact

mdrenderer.github.io/

Vibe-improved a small useful tool to render markdown & html directly from GitHub URLs, so you don't have to setup GitHub Pages etc E.g. `mdrenderer․github․io/?https꞉//github․com/mdrenderer/mdrenderer․github․com/blob/master/readme․md` (All thanks to Claude Code)

I could be wrong. In the essay, I lay out specific conditions that would change my mind, including what I'd look for in regulated industries, artifact lifecycles and quantity

But those changes still go through verification, staged deployment, and real-world feedback. Code remains the source of truth So my takeaway: while code generation is becoming trivially cheap, software engineering is not

My prediction: the future of software is malleable, not ephemeral Code and artifact stacks become much easier to modify. AI lets you describe a change in plain language, have an agent draft the implementation AND update specs, tests, and migrations, then review the whole package

3. Interface stability: users build mental models. Variance across regenerations = user friction & frustration 4. Ambiguity & auditability: "we ran this prompt" is not a sufficient answer for SOC2, HIPAA, or any incident response, really

Four examples with structural barriers for ephemeral software: 1. Edge cases: discovered only through real-world use. Each regeneration resets the clock on them 2. State & integration: silent data corruption from "almost right" regeneration is worse than obviously wrong

Ambiguity equals variance in the behavior of regenerated code, which leads to risk and friction between the code and other systems (other software or end users). This is unavoidable in any app or software that is used more than once, really:

"But isn't this just the next abstraction shift, like assembly → C → Python?" Every prior shift went from one formal language to another. The ephemeral software thesis suggests, however, that inherently ambiguous natural language serve as the persisted specification layer

This creates a structural dilemma: Regeneration either preserves emergent properties, in which case you're investing heavily in continuity (the opposite of ephemerality), or it doesn't, in which case you're accepting production risk with every cycle There is no third option

Ephemeral software is "just rewrite it" on a faster loop. The failure mode is the same: you throw away lessons embedded in code through trial and error

We can also look at the history of software: In 2000, "Things You Should Never Do" told about Netscape's disastrous rewrite: it took 3 years, IE ate their market, and the rewrite shipped full of bugs because the team had to rediscover edge cases already resolved in the original

Code becomes cheaper, but 1) the bottlenecks remain where code meets reality (integration testing and UX) and 2) we still all use PRs, CI, and Git. this ≠ ephemeral Like Amdahl's law, the speed for creating trustworthy software is limited by what can't be sped up by AI alone

But, first, let's avoid a motte-and-bailey fallacy: Vibe coding and evidence that we produce code cheaper and faster has been used to argue that code as a persisted artifact will disappear mostly everywhere ("ephemeral software")

The ephemeral software thesis has serious people and capital behind it. Also from Karpathy: "super custom, super ephemeral one-off apps by default" Or Acharya (a16z): software "doesn't need to be permanent anymore" PS: the evaluation of Lovable, Replit, etc is already at $50B

A while back, Andrej Karpathy said the app store will be replaced by generated, disposable software," and Amjad Masad predicted that the value of all application software will go to zero I think this "ephemeral software hypothesis" is wrong, though, and I want to explain why:

Because the urban warfare during the counterinsurgency, 60% of the buildings were destroyed during fighting in the course of a couple of months. This is in line with the report you cited above. How is this war in Gaza different to that war? en.m.wikipedia.org/wiki/Falluja...

I want to point to one more claim which is already outdated (the relevant paper was only published a few days ago so hardly anyone's fault): The ProRL paper by Nvidia has shown that RL-based models can truly learn new things - if you run RL long enough! arxiv.org/abs/2505.24864

But as Gemini 2.5 Pro explains, River Crossing's optimal solutions are rather short, but they have a high branching factor and number of possible states with dead-ends. That models fail here is a lot more interesting and points towards areas of improvements.

All in all, the Tower of Hanoi results cannot be given any credence because it seems there are many confounders and simpler explanations. However, I don't think the other games hit the same issues. If we look at River Crossing it seems to hit high token counts very quickly:

We use top-p or top-k sampling with a temperature of 0.7 after all. Thus, there is a chance the model gets unlucky and the wrong token is sampled, resulting in failure. (They should really try min-p 😀) @scaling01 has a nice toy model that matches the paper results:

Otoh, Tower of Hanoi is rather straightforward. BUT it requires many steps (2^N - 1, where N is the number of disks). LLMs already know the optimal algorithm to solve it, so the only problem is writing out all those steps! Performing well on ToH is not about reasoning at all 🫠

The paper explores four puzzle environments: Tower of Hanoi, Checkers Jumping, River Crossing, and Blocks World. It finds some "surprising" behavior of LRMs: they can perform 100 correct steps on the Tower of Hanoi, but only 4 steps on River Crossing. x.com/MFarajtabar...

Important Caveat: Real-world scenarios introduce complexities—approximation errors, model dynamics, and training stochasticity can affect submodularity assumptions. Practical methods must balance theoretical ideals with these realities. 8/11

Practical Example: An active learning experiment on MNIST using a LeNet-5 model with Monte Carlo dropout selecting via BALD scores (expected information gain). We can visualize how sample informativeness evolves dynamically during training (w/ EMA for visualization): 6/11