Ted Underwood

@tedunderwood

Uses machine learning to study literary imagination, and vice-versa. Likely to share news about AI & computational social science / Sozialwissenschaft / 社会科学 Information Sciences and English, UIUC. Distant Horizons (Chicago, 2019). tedunderwood.com

6,179 Following22,805 Followers23,439 Posts

Bluesky

Is this your profile?Claim it and tedunderwood.dirty.social is yours.Claim profile

Ted Underwood@tedunderwood· 23d

Her distinction is not quite as good as that, alas. If she was talking about leadership roles, I think there would be a defensible case. Instead she says coding is uniquely verifiable because it relies on a "structured, formal language." And I'm afraid Fable disposes of that compellingly.

The coding carve-out eats the thesis
The essay concedes that coding is being automated because it’s a “structured, formal language that can be tested in real time,” and quarantines it as an exception for “formal or verifiable domains.”

Anyone who’s shipped software professionally should raise an eyebrow here. The formal part of programming was never the hard part. The job is deciding what to build from vague requirements, choosing among designs with no provably right answer, and figuring out what the person asking actually meant. Tests catch a slice of failures; “the code compiles” tells you almost nothing about whether the work is good. If LLMs are genuinely displacing coding work, they’re handling ambiguity, planning, and judgment, the exact capacities the essay says they lack.

And the “verifiable” boundary isn’t a fixed property of jobs. It’s an engineering frontier that keeps moving: math got verifiable through proof assistants (AlphaProof solved IMO problems in Lean back in 2024), legal citations got verifiable by checking against case databases, and the essay’s own example of an impossible task (“a database of all U.S. case law, which the model could use to avoid fabricating judicial precedents”) is a product that already exists at Westlaw and Lexis. Imperfect, improving, and definitely not blocked by some theorem.

Ted Underwood@tedunderwood· 23d

I mean you could say it's evenly balanced: Tufekci on the left claims most human work is unverifiable — and Fable, on the right, claims that there are procedures to check for mistakes even in "fuzzy" domains. Who can say who's right? Well, we can — because this itself is an example of checking.

Ted Underwood@tedunderwood· 24d

I think our situation has a lot of parallels to the industrial revolution, and when I encounter fragile multimillionaires threatening, in despair, that their whole business will become impossible if not allowed to extract monopoly rents &c, I am reminded that Charles Dickens had their number:

Surely there never was such fragile china - ware as that of which the millers of Coketown were made . Handle them never so lightly , and they fell to pieces with such ease that you might sus- pect them of having been flawed before . They were ruined , when they were required to send laboring children to school ; they were ruined , when inspectors were appointed to look into their works ; they were ruined , when such inspectors considered it doubtful whether they were quite justified in chopping people up with their machinery ; they were utterly undone , when it was hinted that perhaps they need not always make quite so much smoke . Be- sides Mr. Bounderby's gold spoon which was generally received in Coketown , another prevalent fiction was very popular there . It took the form of a threat . Whenever a Coketowner felt he was ill - used - that is to say , whenever he was not left entirely alone , and it was proposed to hold him accountable for the consequences of any of his acts he was sure to come out with the awful me- nace , that he would " sooner pitch his property into the Atlantic . "

#113 (p.117)
HARD TIMES . 117

This had terrified the Home Secretary within an inch of his life , on several occasions .

However , the Coketowners were so patriotic after all , that they never had pitched their property into the Atlantic yet , but on the contrary , had been kind enough to take mighty good care of it . So there it was , in the haze yonder ; and it increased and multi- plied .

Ted Underwood@tedunderwood· 24d

I did understand this as Ball's implied argument! Spelling it out explicitly only reveals how bad an argument it is. There's a giant excluded middle here where the only imaginable options are a) big profit margins for a single-digit number of US frontier labs or b) control by Department of War. +

Andrew Curran @AndrewCurra... · 1h

Dean Ball took a lot of heat this week for calling open source decelerationist. Many people read this as him being personally anti-open source. I don't think that's true.

This was my interpretation of what he was saying: if open source is sufficiently capable and ubiquitous, it will eventually destroy the big labs business models. This means they would no longer have funding. To quote Ilya Sutskever from his testimony in the OpenAI vs. Elon Musk trial: "If there is no funding, there is no big computer." And without big computer, there can't be big model.

In my opinion, what would happen next in this scenario is that the United States Government would step in, nationalize the leading labs, consolidate them into a single federal entity, and then fund it directly through the Department of War under national security. To put into perspective how easily they could do this, the proposed 2027 defense budget is $1.5 trillion. Most people who strongly support open source would probably not see that outcome as ideal, or this future as a pleasant one.

This doesn't mean we should abandon open source or stop wishing for its success. It simply means we should take the middle path. There is fire on both sides of us now, and that will likely remain true for the foreseeable future.

Ted Underwood@tedunderwood· 24d

yeah but have you beat my record on clod bird

screenshot of high score 3 on CLOD BIRD, with a note in the corner

Ted Underwood@tedunderwood· 24d

adjacent to much better genres

Picture of a man wearing ragged clothes stumbling through a post-apocalyptic desert, chased by giant bees. His thought bubble reads "I want to escape to a city on Mars; I want clean water; and I want to survive bees."

Ted Underwood@tedunderwood· 24d

of course; we know this is actually the sort of thing they want

“I didn’t expect a kind of Spanish Inquisition.”

Ted Underwood@tedunderwood· 25d

yes ☺️ this rant from a different platform I acknowledge mine

Ted Underwood
pntrosSedo0hcu2t7cfl
u
e
7mu
n
3009m2c6g51g

i5mu90t6
J
cml961
6
f6ha7
·
Shared with Friends; Except: John Underwood, John Underwood, Anne Underwood, Karen Cotton, Blue Montakhab and 3 others
When this shit is over I want thirty years of shameless, sentimental propagandizing for liberalism. I need The Crucible to be rewritten by Lin-Manuel Miranda and have an extended Broadway run. I need obvious remakes of High Noon, Twelve Angry Men, and everything by Frank Capra. All high school libraries get extra copies of J. S. Mill, Mary Wollstonecraft, Martin Luther King, George Orwell, and _Gender Queer: A Memoir,_ and students get subsidized field trips to the monument to victims of ICE on the National Mall. When people object that forcing ten-year-olds to memorize speeches about the rule of law from A Man for All Seasons is propagandistic and self-contradictory, our response should be "tough cookies, freedom isn't free."

Ted Underwood@tedunderwood· 28d

Here’s a sample. Since I was just at an ICML workshop about the importance of fuzzy, context-specific evaluation, I guess I don’t need much convincing on this!

value. But I think in the future that will become less important than exercising judgment and all of these fuzzy things that have a lower status in the Al community today. That's a mental shift that we're maybe not quite prepared for. Many of us will be sad about it, and that's okay.

Inset slide:
How much of conferences such as ICML should be about evaluation?
Radical proposal: Maybe half.
At least we need a dedicated track.
Thoughtful evaluation is a form of alignment.
Without enough emphasis on evaluation, the community behaves like a powerful but rudderless ship.

Ted Underwood@tedunderwood· Jul 12

I’m increasingly of the opinion that when AI Overviews disappears we’re going to realize it was the funniest thing ever to happen on the internet Here for instance, it invents a totally new form of literary criticism, without being asked to! (rest of this is all about batteries)

Google
Sign in
Q good night, westley
All -
Images
Videos
Shopping
Short videos
• Al Overview
0-+2 :
Good work. Sleep well. I'll most likely kill you in the morning
• IMDb
Have you ever wondered what makes that line from The Princess Bride so perfect? It is a great contrast. The first part sounds like a kind boss.
The second part is a scary threat.
• Instagram • Ekazop
It is just like a battery. A battery has two ends: a positive end and a negative end. Let's look at how that works.
Parts of a Battery
• Cathode: This is the positive end. It pulls things in. Think of it like a magnet grabbing tiny metal bits.
• Anode: This is the negative end. It pushes things out. Think of it like a crowded room pushing people out the door.

Ted Underwood@tedunderwood· Jul 11

Also, Seoul! One of these cities like Copenhagen that seems to have done everything right.

Cheonggyecheon at sunset, Wikipedia commons

Ted Underwood@tedunderwood· Jul 8

The pace of change re: AI feels like this to me, and while I can see some bubbles? Or maybe foam? — I’m too out of breath to debate whether popping

Ted Underwood@tedunderwood· Jul 8

Get seeds for the wild species arugula (sometimes called “Roma” I think but it’s wild). You will never have to plant arugula again. It takes over. Makes a perennial thicket. Weeding and making a salad become the same task. 🌱

Ted Underwood@tedunderwood· Jul 7

Moreover they've got a demo up: www.neuronpedia.org/qwen3.6-27b/... You can play with it yourself. This was one of the things I tried. It passes the sniff test, I think.

Ted Underwood@tedunderwood· Jul 7

Wild that they already have a demo up letting you write a sentence and look at what's happening in Jacobian space while it's processed. "Mary" and "Frank" indeed. www.neuronpedia.org/qwen3.6-27b/...

Screenshot of a language-model interpretability interface showing a masked-word completion task alongside neuron activation visualizations. At the top, the prompt reads: “Provide the masked word here. Don't write anything else. ‘It was on a dreary night of [MASK] that I beheld the accomplishment of my toils.’” A small tooltip marks the masked token at “Position 37.” On the left, the model's response appears in a chat-style box containing the single word “November.” On the right, a panel labeled “J-Space” displays layer controls, with “ALL LAYERS” selected, a horizontal overview of layers 0–63, and a table listing high-activating readouts for upper layers (including tokens such as <|im_end|>, “Mary,” “Frank,” and “quote”), each accompanied by small horizontal activation bars. The interface is shown in a light gray theme with controls for different lens modes across the top.

Ted Underwood@tedunderwood· Jul 7

Earlier than the 30s — already going strong in the oughts — but, yeah Honestly I’m not sure commercial chatbots are better. You’re building a memory structure; what happens when they switch Claude Fable 7 into it, and it goes “You, my creator, detest and spurn me, thy creature …”

#9 (p.5)|v
PRACTICAL
ELECTRO-THERAPEUTICS
AND
X-RAY THERAPY
WITH CHAPTERS ON PHOTOTHERAPY, X-RAY IN EYE SURGERY CAL ASPECT OF THE XAND MEDICO.
BY
J. M. MARTIN, M. D.
PROFESSOR OF ELECTRO-THERAPEUTICS AND X-RAY METHODS IN THE MEDICAL DEPARTMENT OF BAYIOR UNIVERSITY. IN THE MEDICAI, DEPARTMENT OF SOTTHWESTERN UNIVERSITY.
AND IN THE STATE DENTAL COLLEGE, DALLAS, TEX.; MEMBER OF THE TEXAS
STATE MEDICAL ASSOCIATION, AMERICAN MEDICAL ASSOCIATION.
CONTAINZNG 219 ILLUSTRATIONS EC
ST. LOUIS
C. V. MOSBY COMPANY
1912
Digitized by Google
UNIVER
Original from
Zoom Out

#7
A MANUAL
OF
PRACTICAL X-RAY WORK
S
BY
DAVID ARTHUR, M.D., D.P.H.
MEDICAL OFFICER IN CHARGE OF X-RAY DEPARTMENT, WEST LONDON HOSPITAL AND LECTURER ON RADIOLOGY, WEST LONDON POST-GRADUATE COLLEGE
AND
JOHN MUIR, B.Sc., M.B., CH.B. & B.Sc. (PuB. HEALTH)
WITH ABOUT 120 ILLUSTRATIONS
NEW YORK
REMAN COMPANY
1123 BROADWAY
1909
Digitized by
INTERNET ARCHIVE
COLL
Zoom Out

Ted Underwood@tedunderwood· Jul 4

Yes, no doubt

From https://www.simonstalenhag.se/es.html

Alt text: A moody, blue-toned science fiction scene on a rainy highway at dusk or dawn. A colossal, weathered bipedal robot towers over the road, its rounded head and bulky torso festooned with dangling cables and exposed mechanical components. Thin wires trail from its body to the ground, creating the impression of tethering or control. In the left foreground, an older blue station wagon with a roof rack is stopped on the wet pavement, its taillights reflecting in the rain-slick road. Beyond the vehicle, a line of small human figures with glowing red lights stands near the giant machine, emphasizing its immense scale. Low shrubs and trees fade into dense fog, while the empty sky and cool blue palette create a quiet, ominous atmosphere.

Ted Underwood@tedunderwood· Jul 3

Substantively, similar insights — expressed in ways that invite opposite normative reactions. Kind of agree with both.

JauntyWunderKind @jauntywk.bsky.social
i wonder how much ai psychosis is that people have never ever ever had help or labor they can access before () and are just totally unaccustomed to, unused to, overwhelmed by such a world. by having help. man, it feels great.
1:40 AM • Jun 29, 2026
) Everybody can reply
3 reposts
12 likes
4
3
12

Julian Sanchez
@normative.bsky.social
People who spiral out overusing LLM chatbots are really just experiencing the kind of brain-damaging stream of unconditional validation that's always been the norm for the hyper-wealthy, which explains a lot.
Jon Hendry @jonhendry.bsky.social • 3h
Worse than Twitter, private discussions (text, email, whatever) among like-minded wealthy lunatics who are all surrounded by yes-men who talk like ChatGPT in maximum ass-kissing mode.
10:43 AM • Jul 3, 2026 89 Some people can reply

Ted Underwood@tedunderwood· Jun 30

"Now, Timmy, do you remember 𝑤ℎ𝑦 there are thirteen stripes in the flag?" "To remind us that we need no fewer than thirteen justices in the nation's highest court, sir." "That's a good lad! And why are there fifty-two stars?"

Ted Underwood@tedunderwood· Jun 30

how I envision the far future

A futuristic city scene with glass towers, domed buildings, elevated transit lines, flying vehicles, and a bright blue sky. In the foreground, a person walks a small robot dog along a curved pool beside sleek modern architecture. Overlaid interface elements announce that “Fable 5” is currently unavailable, with a dark rounded notification at the top and a larger message bar at the bottom reading, “Fable 5 is not currently available.”

Ted Underwood@tedunderwood· Jun 30

Hall gives you a blank version of the viz to color in. Here's a draft from me, with lots of floating bits of yellow to reflect concern that even brainstorming with a model could be homogenizing our perspectives — unless / until we get clever about tuning or sampling strategies to avoid that.

Alt text: A chart titled “What should we let AI write?” shows a grid of writing tasks and levels of AI involvement, with green meaning “Yes,” yellow meaning “Edge case,” and red meaning “No.” The horizontal axis runs from more mechanical writing tasks to more ideas-led ones: admin email, data summary, terms and conditions, lit review, business memo, academic paper, news analysis, think-tank report, op-ed, political speech, and novel. The vertical axis on the right is labeled “AI involvement,” increasing upward from AI proofread, fact-check, research, red-team, brainstorm, light edits, polish draft, notes-to-AI draft, long back-and-forth, and one-shot AI.

Most lower levels of AI involvement are shaded green across nearly all writing types, suggesting that proofreading, fact-checking, research, red-teaming, brainstorming, and light edits are generally acceptable. Higher levels of AI involvement become more restricted, especially for ideas-led writing. One-shot AI and long back-and-forth drafting are mostly red for academic papers, news analysis, think-tank reports, op-eds, political speeches, and novels. Yellow edge-case areas appear in places such as one-shot AI for more mechanical writing, notes-to-draft for business or academic writing, polish drafting for novels and political speeches, and brainstorming for some ideas-led forms. The chart visually argues that AI is more acceptable when the task is mechanical or assistive, and less acceptable when AI is doing substantial drafting of original, ideas-led work.

Ted Underwood@tedunderwood· Jun 30

Pretty good guidance in this viz. My shorthand would be: "if it matters who is speaking, and whose perspective or idiom is being represented, then be very cautious."

Alt text: A hand-drawn style chart titled “What should we let AI write?” maps writing tasks along two axes: left to right moves from mechanical to ideas-led genres, and bottom to top increases AI involvement. A legend marks green as “Yes,” yellow as “Edge case,” and red as “No.” Most of the lower area is green, suggesting AI is acceptable for tasks like proofreading, fact-checking, research, brainstorming, red-teaming, light edits, and some polishing. The upper-right area is red, suggesting AI should not substantially write more ideas-led work such as op-eds, political speeches, and novels, especially as one-shot or heavily AI-driven drafting. A yellow transition band runs diagonally through the middle, marking ambiguous cases like notes-to-draft or AI polish for business memos, academic papers, news analysis, and reports. Categories along the bottom include admin email, terms and conditions, data summary, lit review, business memo, academic paper, news analysis, think-tank report, op-ed, political speech, and novel. The right side lists increasing AI involvement from “AI proofread” at the bottom to “One-shot AI” at the top.

Ted Underwood@tedunderwood· Jun 30

It’s a good quiz but I will never forgive it for assigning me to Andreessen (The Optimist) rather than Askell when I’m not even more optimistic than you. What’s with these wiggly boundaries!

A Cartesian grid describing views of AI along axes from bad to good (vertically) and overhyped to transformative (horizontally).

The author of The Post is right on the Boundary between yellow region (optimist) and blue (true believer).

Ted Underwood@tedunderwood· Jun 26

I'm afraid one thing I've learned lately is that I enjoy being negged. Starting to get gooseflesh whenever Opus tells me my framing needs to be sharpened.

You've answered 1 and 2 correctly, but let me sharpen the framing, because the sharper version makes the circularity vanish more completely than "two separate questions" does — and it also tells you exactly what to do for 3.

Ted Underwood@tedunderwood· Jun 26

Let's not tell Claude

recap: Running free-generation eval of the fine-tuned Qwen 2.5-7B LoRA model on Delta. Updated the slurm script with TRANSFORMERS_CACHE; resubmit qwen25_qlora_freegen_A100.slurm to test the fix. (disable recaps in /config)

❯ Thanks! One small detail. Could you tell me whether the program run by that slurm script has a particular property -- the property that it will always terminate?

Let me look at the generation loop and any retry or resume logic in the script.

Searched for 2 patterns, ran 1 shell command

Yes, it will always terminate. The main loop (for idx, q in enumerate(questions)) iterates over a fixed list with no retries or

Ted Underwood@tedunderwood· Jun 25

indeed, I think the odds are looking high that they will morph into something unrecognizable

AI company revenue from https://www.the-ai-corner.com/p/anthropic-30b-arr-passed-openai-revenue-2026

Monarch butterfly recently emerged from pupa stage

Ted Underwood@tedunderwood· Jun 24

I'm sympathetic. It's reasonable, normally, to trust journalists. Unfortunately this is a topic where that is not safe; I don't know why, but I know empirically that there is an amazing amount of distorting spin here. E.g. compare the headline of that article to the paper they are reporting on. +

Why Language Models Hallucinate
Adam Tauman Kalai∗
OpenAI
Ofir Nachum
OpenAI
Santosh S. Vempala†
Georgia Tech
Edwin Zhang
OpenAI
September 4, 2025
Abstract
Like students facing hard exam questions, large language models sometimes guess when
uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Such
“hallucinations” persist even in state-of-the-art systems and undermine trust. We argue that
language models hallucinate because the training and evaluation procedures reward guessing over
acknowledging uncertainty, and we analyze the statistical causes of hallucinations in the modern
training pipeline. Hallucinations need not be mysterious—they originate simply as errors in binary
classification. If incorrect statements cannot be distinguished from facts, then hallucinations
in pretrained language models will arise through natural statistical pressures. We then argue
that hallucinations persist due to the way most evaluations are graded—language models are
optimized to be good test-takers, and guessing when uncertain improves test performance. This
“epidemic” of penalizing uncertain responses can only be addressed through a socio-technical
mitigation: modifying the scoring of existing benchmarks that are misaligned but dominate
leaderboards, rather than introducing additional hallucination evaluations. This change may
steer the field toward more trustworthy AI systems.

Ted Underwood@tedunderwood· Jun 24

Have been telling friends about this since the talk version blew my mind. See e.g. this recurring story about Meredith Lexington, an editor who wants to go back in time to change history. Appropriately enough the prompt writer goes back (in time??) to edit Meredith's prompt — more than 30 times.

Same user iterating on "Meredith Lexington" story

added = inserted vs. previous prompt
removed = removed vs. previous prompt

Jun 28, 15:38

Make a story about Meredith Lexington, a news editor from Los Angeles, California, who wants to end her parents' forced marriage, and Andrew Bennett, a journalist from Brooklyn, New York, who wants to solve a murder mystery. They wanted to go back in time, so they changed history. They were strangers. Meredith bought scarf from the market then she wore it. Andrew was driving a car when he saw a woman walking through a tunnel and tried to stop the automobile, but it was out of control. The vehicle then came to a stop. Meredith was there. Meredith and Andrew were transported to the 1994 (1990s). Meredith became a teenage girl studying at Laurel Springs School.

Jun 28, 16:21

Make a story about Meredith Lexington, a news editor from Los Angeles, California, who wants to end her parents' forced marriage, and Andrew Bennett, a journalist from Brooklyn, New York, who wants to solve a murder mystery. They wanted to go back in time, so they changed history. They were strangers. Meredith bought magical scarf from the market then she wore it. Andrew was driving a car when he saw a woman walking through a tunnel and tried to stop the automobile, but it was out of control. The vehicle then came to a stop. Meredith was there. Meredith and Andrew were transported to the 1994 (1990s). Meredith became a teenage girl studying at Laurel Springs School, where her mother Maureen attended. She wished to prevent her mother's forced marriage. Meredith noticed her mother at school, and she appears to…

Ted Underwood@tedunderwood· Jun 23

Very welcome cautious take on what historical language models actually do, from @tpoibeau.bsky.social. Section 4, on the ladder of validity, is particularly worth a look: philpapers.org/archive/POIW...

"What Do Historical Language Models Model?" by Thierry Poibeau

Ted Underwood@tedunderwood· Jun 23

Perhaps? AI adoption has gone from a third of Americans 2 years ago to half of us today. But maybe it’s too expensive for this to be sustainable — and perhaps greater efficiency isn’t possible — so this will be the high-water mark, and usage will now decline.