Thomas Steinke

@stein

Researcher in computer science, math, machine learning, (differential) privacy, AI, etc. at Anthropic. Kiwi🇳🇿 in California🇺🇸

751 Siguiendo4424 Seguidores510 Publicaciones

Bluesky

¿Es tu perfil?Reclámalo y stein.dirty.social es tuyo.Reclamar perfil

Thomas Steinke@stein· Jul 22

xkcd.com/2385/ openai.com/index/huggin...

Thomas Steinke@stein· Jul 16

What does “tokens per second per request” mean?? www.theatlantic.com/technology/2...

Thomas Steinke@stein· Jul 2

Oh yeah I remember that. I remember showing up at the conference venue to present my poster in swim gear, still wet from snorkeling. 😅

https://x.com/trustworthy_ml/status/1685168877444173824

Thomas Steinke@stein· Jul 2

I’m attending COLT 2026 in San Diego. You can see the beach from the conference venue; not sure if that’s a good thing. 🤔 learningtheory.org/colt2026/

Thomas Steinke@stein· May 27

A handsome red bike, abandoned. By a busy road, outside my home. Locked to a tow away sign. What happened to its owner? Not even a thief has come for it. What will the HOA do? I see it daily and wonder.

A red bike in good condition locked to a signpost bearing a tow away sign, with a busy road in the background.

Thomas Steinke@stein· May 16

Here's an example of a "hallucinated" citation from a decade ago (i.e. pre-LLMs). The same bad citation appeared in multiple papers. I eventually traced the source to Google Scholar (and it's now fixed).

[2] Marcus Hardt and Jonathan Ullman. Preventing false discovery in interactive data analysis is
hard. In Foundations of Computer Science (FOCS), 2014 IEEE 55th Annual Symposium on, pages
454–463. IEEE, 2014.

From: https://arxiv.org/abs/1510.03349

[11] Marcus Hardt and Jonathan Ullman. Preventing
false discovery in interactive data analysis is hard.
In Foundations of Computer Science (FOCS), 2014
IEEE 55th Annual Symposium on, pages 454–463.
IEEE, 2014.

From: https://proceedings.mlr.press/v51/russo16.pdf

[19]
Marcus Hardt and Jonathan Ullman. 2014. Preventing false discovery in interactive data analysis is hard. In IEEE Symposium on Foundations of Computer Science (FOCS). 454--463.

From: https://dl.acm.org/doi/10.1145/3139550.3139556

Article (Correct)

Preventing False Discovery in Interactive Data Analysis Is Hard
Authors: Moritz Hardt, Jonathan Ullman
FOCS '14: Proceedings of the 2014 IEEE 55th Annual Symposium on Foundations of Computer Science
Pages 454 - 463
https://doi.org/10.1109/FOCS.2014.55
Published: 18 October 2014 Publication History

From: https://dl.acm.org/doi/10.1109/FOCS.2014.55

Thomas Steinke@stein· May 11

"For People"

Thomas Steinke@stein· Mar 14

https://truthsocial.com/@realDonaldTrump/posts/116189925042301817

https://truthsocial.com/@realDonaldTrump/posts/116227904143399817

Thomas Steinke@stein· Mar 14

Started/going

https://www.bbc.com/news/articles/c9dn3j04lydo
Trump accuses Starmer of seeking to 'join wars after we've already won't
7 days ago

https://www.bbc.com/news/live/ckg1w1jp8kjt
Trump urges UK and other nations to send ships to help secure Strait of Hormuz after Iranian attacks
LIVE

Thomas Steinke@stein· Mar 12

Some people are still debating whether or not LLMs are "useful," so let's stake out one clear use case: LLMs are useful for translating between languages. That includes translating between natural languages (e.g. Spanish to English) and, more recently, to formal languages (e.g. English to Python).

Screenshot of bsky post with author's identity not shown. [I'm not trying to pile on.]

Sure, LLMs are useful for:
1. Fraud
2. Plagiarism
3. Cognitive off-loading
Which of those use-cases are you promoting?

3:59pm March 10 2026
51 reposts, 14 quotes, 257 likes, 6 saves

Thomas Steinke@stein· Feb 7

I like spotting license plates from interesting states. This one was notably interesting.

Picture of a car license plate with the number obscured. The plate says.
CHAHTA SIA HOKE!
In God We Trust
Choctaw Nation of OKLAHOMA

Thomas Steinke@stein· Jan 31

Which is the best option for returning nothing?

Screenshot of Amazon app listing return options.
Whole foods
Staples
Kohl's
UPS
Etc.

Thomas Steinke@stein· Jan 31

Amazon sent me an empty/broken package. I'm trying to get my money back, but there's no explicit option for this. I may have to "return" the item by sending an empty package back.

Photo of a green label on top of a white USPS label on a brown package.
The green label says the following.

ATTENTION CUSTOMER
* RECEIVED UNSEALED
* RECEIVED DAMAGED
* RECEIVED WITHOUT CONTENTS

There is a handwritten "B 1/30/26" on the green label.

Thomas Steinke@stein· Jan 17

The cost of living is out of control: A bag of ice costs $4 and my local supermarket is out of stock. This is why we need Greenland.

Screenshot showing search results for "ice". The first result is a 5lb bag for $3.99 and is shown as out of stock.

Thomas Steinke@stein· Dec 31

TIL that the etymology of "California" is plausibly related to "Caliphate".

https://en.wikipedia.org/wiki/Etymology_of_California

Thomas Steinke@stein· Dec 29

Imagine you are a trained next word prediction model and you see this.

https://en.wikipedia.org/wiki/Ted_Jorgensen

Thomas Steinke@stein· Oct 9

IMHO, the best analog to the AI bubble is the dotcom bubble. Yes, the internet proved to be economically transformative, but there was still a bubble. Companies made a lot of money in the end, but it wasn't necessarily the ones that people expected -- e.g., see CISCO:

Plot of CISCO's stock price from 1990-2025 showing huge growth around 2000 before dropping and slowly growing again.

Thomas Steinke@stein· Aug 13

🤦🤦🤦 this is not how two factor identification works 🤦🤦🤦🤦

[Screenshot of an app]
Code Verification
Call customer service at 1-866-4220306 (outside the U.S. call 1-210-677-0065) to retrieve your One-time identification Code .

Thomas Steinke@stein· Jun 27

Doing linear algebra in finite fields is fun because numerical instability doesn't exist. Alas, library support is limited, so you may find yourself writing your own Gaussian elimination.

def modinv(a, p):
"""Computes inverse of a modulo p i.e. returns b such that (a*b)%p==1"""
u, v = a % p, p
x, y, w, z = 1, 0, 0, 1 # maintain ax+py==u and aw+pz==v
while u != 0: # extended Euclidean algorithm
q, r = v // u, v % u
x, y, w, z = w-qx, z-qy, x, y
u, v = r, u
if v != 1: return None # not invertible
return w % p

def modmatrixinverse(A, p):
"""Computes matrix inverse of A modulo p i.e. returns B s.t. A@B%p==I"""
A = numpy.array(A, dtype=int)
m, n = A.shape
assert m == n, "matrix must be square"
B = numpy.eye(n, dtype=int) # = nxn identity matrix
for i in range(n): # Gaussian elimination
b, j = None, i-1
while b is None:
j += 1
if j >= n: return None # not invertible
b = modinv(A[j,i], p)
if j > i: # swap rows i and j
A[j,:], A[i,:] = A[i,:], A[j,:]
B[j,:], B[i,:] = B[i,:], B[j,:]
A[i,:] = (A[i,:] * b) % p # divide A[i,:] by A[i,i]
B[i,:] = (B[i,:] * b) % p
for j in range(n): # zero out the rest of column i
if j != i:
B[j,:] = (B[j,:] - B[i,:] * A[j,i]) % p
A[j,:] = (A[j,:] - A[i,:] * A[j,i]) % p
return B

Thomas Steinke@stein· May 31

Wait, what? 🤔

Photo of a sign saying
"
No Pets
Shoes, Shirt
Required
"
And pictures of a footprint, shirt, and dog all crossed out

Thomas Steinke@stein· May 22

What's the full acronym then? 🤔

Picture of an Amazon delivery truck with the following written the side.

The 'P' in ASAP stands for Prime.

Thomas Steinke@stein· May 14

I'm a fan of the Jensen proof. It generalizes to prove Hölder's inequality:

$\textbf{H\"older's inequality from Jensen's inequality:} \noindent Let $x,y\in\mathbb{R}^d$ be arbitrary. Let $p,q \in (1,\infty)$ satisfy $\frac1p+\frac1q=1$. Without loss of generality, assume $x_i \ne 0$ for all $i \in [d]$. Note that $v \mapsto |v|^p$ is convex. For $i \in [d]$, define $u_i = |x_i|^q/\|x\|_q^q$ and $v_i = x_iy_i/|x_i|^q$. By Jensen's inequality, \[\left|\frac{\sum_i x_i y_i}{\|x\|_q^q}\right|^p = \left|\sum_i u_i v_i\right|^p \le \sum_i u_i |v_i|^p = \frac{\sum_i |x_i|^{q+p-pq}|y_i|^p}{\|x\|_q^q} = \frac{\|y\|_p^p}{\|x\|_q^q}, \] which rearranges to \[\left|\sum_i x_i y_i\right| \le \|x\|_q \cdot \|y\|_p.\]$

Thomas Steinke@stein· May 14

Three proofs of Cauchy-Schwarz. ⟨x,y⟩ ≤ ∥x∥ ∥y∥ Are there any others you know of?

$\textbf{Fact:} $\forall u, v \ge 0 ~~~ \inf_{t>0} t \cdot u + \frac1t \cdot v = 2\sqrt{uv}$ \begin{proof} Assume $uv>0$; otherwise the proof is trivial. Let $f(t) = t \cdot u + \frac1t \cdot v$. Then $f'(t) = u-v/t^2$. Now $f'(t)=0 \iff t = \sqrt{v/u}$. Thus $\inf_{t>0} f(t) = f(\sqrt{v/u}) = 2\sqrt{uv}$. \end{proof} Let $x,y\in\mathbb{R}^d$. Then \begin{align*} \sum_i x_i y_i &\le \frac12 \sum_i 2\sqrt{x_i^2 \cdot y_i^2} \tag{absolute value} \\ &= \frac12 \sum_i \inf_{t_i>0} t_i \cdot x_i^2 + \frac{1}{t_i} \cdot y_i^2 \\ &\le \frac12 \inf_{t>0} \sum_i t \cdot x_i^2 + \frac1t \cdot y_i^2\\ &= \frac12 \inf_{t>0} t \cdot \|x\|^2 + \frac1t \cdot \|y\|^2\\ &= \sqrt{ \|x\|^2 \cdot \|y\|^2 } = \|x\| \cdot \|y\|. \end{align*}$

$\textbf{Jensen's inequality:} \noindent Let $x,y\in\mathbb{R}^d$ be arbitrary. Without loss of generality, assume $x_i \ne 0$ for all $i \in [d]$. For $i \in [d]$, define $p_i = x_i^2/\|x\|^2$ and $v_i = y_i/x_i$. By Jensen's inequality, \[\left(\frac{\sum_i x_i y_i}{\|x\|^2}\right)^2 = \left(\sum_i p_i v_i\right)^2 \le \sum_i p_i v_i^2 = \frac{\sum_i y_i^2}{\|x\|^2}, \] which rearranges to \[\sum_i x_i y_i \le \|x\| \cdot \|y\|.\]$

$\textbf{Sum of squares:} \noindent Let $x,y\in\mathbb{R}^d$. For all $\alpha,\beta>0$, \[0 \le \| \alpha x - \beta y \|^2 = \alpha^2\|x\|^2 + \beta^2\|y\|^2 -2\alpha\beta\langle x , y \rangle\] and, hence, \[\langle x , y \rangle \le \frac{\alpha}{2\beta}\|x\|^2 + \frac{\beta}{2\alpha}\|y\|^2.\] If $\|x\|=0$ or $\|y\|=0$, clearly $\langle x , y \rangle = 0$. Otherwise, set $\alpha=\|y\|$ and $\beta=\|x\|$ to obtain \[\langle x , y \rangle \le \frac{\|y\|}{2\|x\|}\|x\|^2 + \frac{\|x\|}{2\|y\|}\|y\|^2 = \|x\| \cdot \| y \|.\]$

Thomas Steinke@stein· May 7

Suppose X,Y,Z,W are independent standard Gaussians. Then X·Y+W·Z has a standard Laplace distribution. Similarly, Z·√(X^2+Y^2) has a standard Laplace distribution

LaTeX source: https://pastebin.com/zL7rVVSN

Thomas Steinke@stein· Apr 27

As an application of this, we get to prove concentrated differential privacy for the restricted Gaussian mechanism. E.g. if you have a bounded query and add Gaussian noise, you can condition the noisy output to also be bounded without any loss in privacy parameters. 😁

LaTeX source for all 3 pages: https://pastebin.com/W6AtEwJ1

Thomas Steinke@stein· Apr 27

Here's an application using the Log Sobolev Inequality for strongly log-concave distributions to bound KL divergence which can thus be converted to a bound on Rényi divergence.

Thomas Steinke@stein· Apr 26

You can bound Rényi divergences in terms of KL divergences for tilted distributions. This is useful e.g. for Gaussians, where tilting just corresponds to shifting the distribution.

LaTeX source: https://pastebin.com/m9RwBcCi

Thomas Steinke@stein· Apr 19

There are also rod cells in your retina. In principle these give you a 4th dimension for perceiving colour. But they are for peripheral & night vision, so we don't perceive a 4th color dimension. 🤷

Source: https://commons.wikimedia.org/wiki/File:Cone-absorbance-en.svg

Thomas Steinke@stein· Apr 19

Colours correspond to infinite-dimensional vectors, since there are infinitely many wavelengths of light. But humans can only perceive a three-dimensional projection of colour (red, green, & blue). What's interesting is that it's *not* an orthogonal projection. Here's a plot of the basis vectors.

Source: https://commons.wikimedia.org/wiki/File:XYZ_color_matching_functions,_CIE_1931_and_Stockman_%26_Sharpe_2006.jpg

Thomas Steinke@stein· Apr 19

Taking α→1 gives a triangle inequality for KL divergence. This can also be proved using my favourite lemma. 😁

$Full LaTeX source: https://pastebin.com/mA6KjUJs \begin{proposition}[Triangle-like inequality for KL divergence]\label{prop:kl-triangle} Let $P$, $R$, and $Q$ be probability distributions with $P$ being absolutely continuous with respect to $R$ and $R$ being absolutely conotinuous with respect to $Q$. Let $\kappa \in (1,\infty)$. Then \[ \dr{\text{KL}}{P}{Q} \le \frac{\kappa}{\kappa-1} \dr{\text{KL}}{P}{R} + \dr{\kappa}{R}{Q}, \] where $\dr{\text{KL}}{P}{Q} := \ex{X \gets P}{\log(P(X)/Q(X)}$ denotes the KL divergence and\\$\dr{\kappa}{R}{Q} = \frac{1}{\kappa-1} \log \ex{X \gets R}{(R(X)/Q(X))^{\kappa-1}}$ denotes the R\'enyi divergence of order $\kappa$. \end{proposition}$