Ian Hussey
@ianhussey
Meta-scientist and psychologist. Senior lecturer @unibe.ch. Chief recommender @error.reviews. "Jumped up punk who hasn't earned his stripes." All views a product of my learning history. If behaviorism did not exist, it would be necessary to invent it.
Any suggestions for semi-popular online chi-square calculators? Sometimes less experienced people use them. To validate my {recalc} package, I'm reverse engineering them to make sure the package accommodate their results, as long as they're correct and sane. These are the ones I have already.
Here you can see the near zero correlations between the PTP (and its subscales) with other established measures and their subscales. Disattenuating for reliability doesn't rescue the situation.
New Jingle assessment: we show that the "Perspective Taking Protocol" has very poor reliability and near-zero convergent validity with other nominal measures of emotional and spatial perspective taking, empathy, and theory of mind. w/ @malte.the100.ci
And recalculating the SMD effect size from Ahmadpanah's M/SD/Ns for sleep quality under a variety of assumptions produces SMD estimate between [0.50. 0.58], not the meta's 2.38. Maybe I'm missing sth obvious of course. errors.shinyapps.io/recalc_indep...
Well, out of the gate, I don't think the numbers in Ahmadpanah match the numbers for Ahmadpanah reported in Zhang's meta-analysis.
Negative controls are very useful for spotting errors and data quality issues. If 1/3 of participants disagree that playing poker in person for real money is gambling, broader questions are raised about the other findings. pubpeer.com/publications...
So this response to critique is that… @kauspurg.bsky.social is *too experienced* in writing critiques?? The argument for bias is also silly. People read many articles and write critiques of the ones with particular issues. Very bad response.
The {recalc} R package has been updated to do this, as well as various other forms of recalculation of reported stats - while taking the uncertainty introduced by rounding into account. github.com/ianhussey/re...
Mediation analysis is surprisingly easy to verify using reported betas, p values, and Ns. Eg the indirect a*b path should be, well, a multiplied by b. Sometimes its not, like in this article claiming SSRI efficacy is mediated by expectations. Whoops. pubpeer.com/publications...
Research Integrity investigations can take time, and EoC's are intended to flag editors' concerns to readers. i.e., COPE's EoC guidelines: publicationethics.org/guidance/gui...
@taymalsalti.bsky.social has done trojan work trying to reproduce the results of three meta-analyses, finding the usual host of issues of effect sizes that are apparently incorrect or non-reproducible Our article is out now: doi.org/10.1111/ejn....
On the basis of a consistency check between reported R2 and Std. Beta values that I did for this pubpeer comment, I have updated the {recalc} R package to make this easy to do elsewhere.
Eagle-eyed @grinschglsandra.bsky.social spotted this mess of a plot, with bars that don't match percentages. We, with @malte.the100.ci, quantify a host of other issues we spotted here. pubpeer.com/publications...
Reminder: If researchers find Cohen's d = 6, no they didn't. trustworthy.scientific.claims/posts/if-res...
Fun facts - The original study reported 23 IQ point gains - Jordan Peterson tweeted about it (sort of) - @jamiecummins.bsky.social new preregistered RCT shows null effects
Not to be an uber pedant/nerd, but i'm fascinated by the plausibility of SMD effect sizes and so collect really large ones. The whole government change between year is about Cohen's d = 1.14, whereas we recently estimated the preference for chocolate over poop as Cohen's d = 5.57.
Plotting the effect sizes included in the meta immediately shows something is likely off, as the distribution is highly bimodal and has huge magnitude d values. One key thing that can cause bimodality in meta effect: systematic confusion of SE and SD for many of the effect sizes.
{psychdsish} R package to create standardized structured projects for psychological research projects processed and analyzed in R: github.com/ianhussey/ps...
Early draft of my ebook for the course: ianhussey.quarto.pub/reproducible...
Basic questions like "how many participants are in the dataset?" can produce surprisingly different answers between analysts. Results from students in my class in data wrangling in tidyverse, who are good at wrangling but still have to make semi-subjective choices: