Joe Bak-Coleman
@jbakcoleman
Research Scientist at the University of Washington based in Brooklyn. Also: SFI External Applied Fellow, Harvard BKC affiliate. Collective Behavior, Statistics, etc..
Posting my W that I'm officially cured of dysphagia, having gone from a functoinal oral intake scale of 1 (nothing by mouth) to eating this damn bagel inside of ~2 months.
The existing statements include the bank of Canada and UK government, which would seem to imply no one on that giant authorship otherwise has any ties to anything that would stand to benefit from, say, LLMs finding a market in science.
Stoked to read this new @statmodeling.bsky.social, @rmcelreath.bsky.social and every other Bayesian book. I’ve gotta say, a chapter 4 skim has got me me hooked.
Problem 4: Running statistical models on LLM output without a ton of care. Issue here is the same as using stats on simulations, you can always (cheapily, arbitrarily) crank N to shrink CI's and drop p-values.
To answer the question, they simply counted papers in high profile journals and evaluated whether you could gather data of the same sort today. The answer is pretty grim; a lot of data came from proprietary arrangements or APIs that no longer exist as they did.
Look, I've lost some of my snark but hope for a full recovery. I'll just end this thread with the conclusion. This is where we're at after 12 years of replicating thousands of social science studies.
I really need to reread these papers with fresh eyes. It’s just WILD
Didn’t think being interviewed by @brandyzadrozny.bsky.social would get me into the Epstein files but here we are.
I have plenty of reservations about the reliability of funnel-plots and z-curves and such, as well as their interpretation..... But holy shit look at that.
Elsewhere they discuss a meeting in 2018 hosting 30 academics and converting ten from skeptics to allied.
A side note in metas court room drama is this little memo from 2018 discussing the goal of converting ten “academic allies” given that researchers were associating harms to their products.
Just to put a pin in it, that hyped news coverage drove moltbook to a $100 million dollar market cap, the vast majority of which has been erased in just 8 days.
We did find that lower and middle hdi countries tended to be more optimistic.
I'm just reading through the Bunking Browser and having trouble seeing what we learn relative to outcomes like this for participants (here a 66 year old woman). I can't find the post-debrief score and while the averages are encouraging, it's the failed-debrief tails that matter here.
Everyone in a while you come across a band that is great, with a fantastic name and just need to buy the shirt.
New paper from myself and many fantastic collaborators on the value of friction online social networks. A fun blend of principles from complexity science and social media design. www.nature.com/articles/s44...
Some reflections by @broniatowski.bsky.social on our preprint and some of the discussion around what constitutes a conflict of interest in @kakape.bsky.social's write-up. A few thoughts and reflections below 🧪 broniatowski.substack.com/p/when-is-a-...
It's in this 2024 paper published in Science. www.science.org/doi/10.1126/...
Here's the table from their paper. Of the 67 papers significant after FDR correction: -36 Failed to replicate -31 succeeded. Given this, if everyone applied FDR, you'd expect the probability of replication in this corpus (among significant findings) to go from ~36% to 46%.
This paragraph is a pretty good exercise in spot-the-bullshit. Can you see it? Reading this paragraph, you'd walk away with the conclusion that mere FDR correction can address replicability issues. The authors certainly try to make this claim. www.nature.com/articles/s41...