Ruben C. Arslan

@ruben

Bayescurious evidence enthusiast at the100.ci Topics: evolution, ovulation, mutation, intelligence, personality, sexuality, R, open science & source tools.

802 Following4,644 Followers1,768 Posts

Bluesky

Is this your profile?Claim it and ruben.dirty.social is yours.Claim profile

Ruben C. Arslan@ruben· 4d

maddening. researchers shouldn't abide such policies not to disclose massive errors. pubpeer first, then the journal gets a chance to do the right thing.

The original claim that science was becoming less disruptive has appeared in more than 300 news stories worldwide.

The Belgian researchers submitted their critique to Nature in October 2023. It took the journal 32 months to accept it. While the critique was under review, Nature still published an editorial and ran a News Feature drawing on the original conclusion. During those years, the original paper was cited more than 1,000 additional times and used in at least 16 policy documents.

"Whereas the software bug was fixed by the open-source community in months, it took years for the scientific correction to be published," says Ginis. "And all this time, the journal's policies prohibited us from speaking to the press while our critique was under submission."

Other researchers in the field have suggested restricting the analysis to papers with at least 10 references and citations. When they follow this guideline, the Belgian team finds a slight, statistically significant increase in disruption. Still, this is not their conclusion.

Ruben C. Arslan@ruben· 17d

it's good, but it starts with a misleading statistic that its own source doubts (180,000,000 IQ points saved through iodine). Real effect is likely much smaller than 15 IQ points based on RCTs and the original study comes to this estimate very circuitously.

Ruben C. Arslan@ruben· Jul 17

Ruben C. Arslan@ruben· Jul 15

Ruben C. Arslan@ruben· Jul 9

Ruben C. Arslan@ruben· Jun 24

Ruben C. Arslan@ruben· Jun 15

they literally did it.

We also had concerns about the inferential leap from the video interview data to the paper's broader conclusions. Of 50 contacted low-quality respondents, 20 agreed to participate, and all 20 were human, a finding the authors use to support the claim that problematic respondents are generally not bots. However, automated bots cannot respond to a video interview invitation in the first place, so the sample is necessarily restricted to those capable of responding. The absence of bots among willing video interviewees does not tell us much about their prevalence among non-respondents. If 30 of the 50 contacted respondents were bots, such a result would also have been found.

Ruben C. Arslan@ruben· May 22

interesting. with EA3 from 2018 instead of EA4 from 2022. I wonder why.

There are two main findings of our study. First, the reform led to a decrease in the association between genes and socioeconomic outcomes. This is evidenced by the negative interaction between the PGI for education and the reform across all three outcomes—education, income, and wealth—providing support for hypothesis 1b. Second, we find no significant interaction effects between parental SES and the reform, indicating that the relationship between parental SES and socioeconomic outcomes remained stable after the reform. Thus, we do not find evidence supporting hypothesis 2a or 2b. Our result of a decline in the role of the PGI for education after the reform is consistent with some previous studies (Conley and Domingue 2016; Okbay et al. 2016; Barcellos et al. 2018), while it contradicts others (Colodro-Conde et al. 2015; Lahtinen et al. 2023). Notably, our finding closely mirrors that of Barcellos et al. (2018), who found a decreasing association between the PGI for body mass index and health outcomes after the 1972 educational reform in England. This similarity is important because both studies focus on England and use a similar research design—leveraging a school leaving age reform, using PGIs, and implementing an RDD. In contrast, differences in the methodological approach and the context may explain the conflicting results with other studies. For instance, Colodro-Conde et al. (2015) used twin models instead of PGIs, while Lahtinen et al. (2024) examined Finland’s comprehensive school reform, which postponed tracking from age 11 to age 16. We know from previous literature that, in contrast to the reforms expanding free compulsory education, the reforms suppressing early tracking are especially beneficial for high-ability children from lowSES families (van de Werfhorst 2018).

Ruben C. Arslan@ruben· May 8

I've wanted an R fiddle for a while, so I had Claude make one. fiddle.rforms.org - completely client-side thanks to webr, no cookies, no external reqs without permission - link to scripts via url param (? or #) - rmd/qmd support

Ruben C. Arslan@ruben· Apr 21

Can't resist reposting this.

Measuring self-citation using the patented 100% CI folding rule

Ruben C. Arslan@ruben· Apr 14

from the grant funder. who said anything about awards in court cases? why would people look for and report bugs that don't meet the threshold? it is not hourly pay. nothing wrong with that imo.

Ruben C. Arslan@ruben· Apr 14

If after your initial pour, the scale shows 25 grams, you'll add more until it shows 30 grams. But the scale still has error. What has changed is that now the true value is dependent on the measured value (X -> T) and the error is independent of X, not of T.

Ruben C. Arslan@ruben· Apr 14

Say, you want to measure how much müsli I eat for breakfast, I pour it into my bowl and you weigh my bowl before I eat it. The measured weight depends on the true weight (T -> X). The error of the scale means your measured value is slightly off the true value. Classical error: X=T+E

Illustration of classical measurement error

Ruben C. Arslan@ruben· Mar 27

no you were thinking of this one www.sciencedirect.com/science/arti... but it wasn't mayo!!! it was corn starch. gotta get the details right.

Ruben C. Arslan@ruben· Mar 27

journals.sagepub.com/doi/full/10....

Ruben C. Arslan@ruben· Mar 11

All I know is I've not seen people in the field who use RSA go beyond one trait per model, nor use Lasso. I once did LOOIC model comparison with four t2(trait,pref) splines but never published. I think the psych lit usually focuses on interpreting the derived RSA params. osf.io/download/v6m35

Significance Testing after Model Selection
Assume that the first step of investigating a (simple or complex) congruence hypothesis
revealed that the respective hypothesis-conforming model did not fit the data significantly
worse than the full third-order model. Then the coefficients of this model must be tested for
significance in a second step, to determine whether the suggested effects are significant and in
the expected direction. One open methodological issue refers to the question of how the model
selection uncertainty from the first step can be considered when testing the coefficients in the
second step, in order to avoid potential inflation of Type I error rates (Berk et al., 2010; Leeb
& Pötscher, 2006). Different strategies have been suggested to account for such post-selection
inference (e.g., splitting the data into a model selection sample and a coefficient testing
sample, Berk et al., 2010; computing confidence intervals that consider all other models that
might have been chosen by the model selection procedure, besides the one that was in fact
chosen, Berk et al., 2013; or computing a confidence interval whose coverage probability is
secured given the selected model, e.g., Lee et al., 2016; Rügamer & Greven, 2018). However,
there seems to be no consensus yet on which method is the "most valid" one. Thus, we cannot
yet make a well-informed recommendation on how to account for post-selection inference in
cubic RSA. In the illustrative R Code, we decided to reduce the alpha level to .01, but this
CUBIC RSA 52
approach should be seen as preliminary and should be adapted on the basis of future
methodological work on the question of post-selection inference.

Ruben C. Arslan@ruben· Mar 9

This is me living up to my new year's resolution that I'll keep getting mad about low ceilings in measurement and elsewhere.

me being mildly cross at best about a low ceiling

Ruben C. Arslan@ruben· Feb 14

You're in good company.

Ruben C. Arslan@ruben· Feb 6

Cystic fibrosis is highly heritable. Polydactyly which we just discussed is highly heritable. Lactase persistence is highly heritable (and not universal!). Here's a footnote from @kph3k.bsky.social's book that makes the point clearly IMO.

The evolutionary biologist Richard Dawkins made the point that ge ne tic
causes should be defined as difference makers even for relatively simple pheno-
types that are intuitively “ge ne tic,” such as eye color. He wrote, “The ‘effect’ of any
would-be cause can be given meaning only in terms of a comparison, even if only
an implied comparison, with at least one alternative cause. It is strictly incomplete
to speak of blue eyes as ‘the effect’ of a given gene G1. If we say such a thing, we
really imply the potential existence of at least one alternative allele, call it G2, and
at least one alternative phenotype, P2, in this case, say, brown eyes.”
He continues with an example of two genes both related to skin pigmentation:
“To be sure, A, the gene whose protein product is the black pigment, is necessary
in order for an individual to be black. . . . But I shall not call A a gene for blackness
unless some of the variation in the population is due to lack of A. . . . The point
that is relevant here is that both A and B are potentially entitled to be called genes
for blackness, depending on the alternatives that exist in the population (emphasis

added). The fact that the causal chain linking A to the production of the black
pigment molecule is short, while that for B is long and tortuous, is irrelevant.”
Fi nally, Dawkins pointed out that natu ral se lection is concerned with differ-
ences: some versions of genes become more common than others because those
versions cause differences in fitness. Evolution requires a comparison.
Failure to appreciate the fact that ge ne tic causes, like all other causes, are dif-
ference makers that imply a comparison to some alternative is one major flaw in
the reasoning of a still widely cited essay by the phi los o pher Ned Block. He wrote
(emphasis added), “Ge ne tic determination is a matter of what causes a charac-
teristic: number of toes is genet ically determined because our genes cause us to
have five toes. Heritability, by contrast, is a matter of what causes differences in
a characteristic: heritability of number of toes is a matter of the extent to which
ge ne tic differences cause variation in number of toes (that some cats have five
toes, and some have six).” Block’s error should be readily apparent. What causes
a characteristic is, by definition, what causes differences in a characteristic. To say
that a gene G1 causes us to have five toes is to imply the existence of an alternative
allele and an alternative phenotype— having a gene other than G1 would cause you
to have a diff er ent number of toes.
In fact, the fact that genes are difference makers can be empirically illustrated
using Block’s example of having five toes. Two of the genes that determine toe
number are EVC1 and EVC2. Rare mutations in these genes cause polydactyly
(extra fin gers and toes), as well as short ribs, teeth abnormalities, and cardiac
defects, a syndrome known as Ellis– van Creveld syndrome. The EVC genes code
for a protein that is found on the little hairlike projections that surround each
cell; the protein helps cells communicate wi…

Ruben C. Arslan@ruben· Feb 3

So unless I've missed something in his book, they seem to incorrectly cite Rutherford as saying it's not verifiable in principle. He says we should look to genomics, but EP as a field has barely engaged with genomics with few exceptions.

Since its inception, the eld
of evolutionary psychology has been accused of such lack of
rigor by critics who often assert that evolutionary psycho-
logical hypotheses cannot, in principle, be tested empirically
(e.g., Gould & Lewontin, 1979; Rutherford, 2020). In this

Ruben C. Arslan@ruben· Feb 3

So, let's see what a more modern critic has to say. AFAICT Rutherford doesn't mention EP, though he does criticise adaptationism. I don't see him saying it's unfalsifiable, he criticises assuming that any heritable variation is adaptive. That's fair! Evolution is also about mutation and drift.

$Arguments that the presence of brain disorders at high frequency in Ashkenazi Jews might explain the enrichment of genes that boost brains is woolly conjecture, and can be abandoned with current data to hand. “Winnowing through persecution” is also merely idle speculation, and has no place in a decent scientific paper. These are fractionally more sophisticated versions of the evolutionary crime we call adaptationism*—the assumption that natural selection is responsible for specific human behaviors, rather than happenstance or processes that are neither positive nor negative, but have simply drifted into existence. In the genomic age we are capable of actually seeing the parts of the genome where selection has taken place, and there are population-specific mutations that indicate positive selection of particular genes as adaptations to the local environment. Pigmentation, specific diets, resistance to diseases (such as malaria), and other traits are demonstrably local adaptations that are part of humankind’s success in colonizing the world. Adaptationism is an error because in many cases it results in untestable hypotheses, but ones that are appealing because they sound superficially convincing—Blacks are good sprinters because of selection during slavery; Jews are intellectually gifted because their history of persecution enriched$

Ruben C. Arslan@ruben· Feb 3

yeah, in women validity is as bad as estradiol, also unpublished. but pranj mehta has published critiques, so people should know. also widely understood in medicine.

Ruben C. Arslan@ruben· Feb 3

New meta finds no correlation between testosterone and risk preference, buuuuut testosterone was mainly measured using 2D:4D (nonsense) or saliva (still bad) and we know from Frey et al. 2017 that the risk measures don't tap into a coherent preference. www.sciencedirect.com/science/arti...

violin plot of results by testosterone measure

violin plot of result by risk pref measure

Ruben C. Arslan@ruben· Feb 2

I also like this one from Emily Oster. What's the secret ingredient that books borrowed from the library or read to you by a paid caretaker just do not have? Surely, it must be LOVE

“First, parenting matters. Much more consistent than any of the associations in these studies is the association between parenting and child outcomes. Having books in your house and reading them to your kid is going to matter much more than what books they have at day care. This seems to be true even though your child probably spends as many waking hours with their care providers as with you. I don’t think we know precisely why this is the case, although it may be that you as the parent are the most consistent influence your child has.”

Ruben C. Arslan@ruben· Jan 25

To me, the funniest thing about the BART except individual differences in it not being related to important real life risk taking is that it isn't even representative of how real life balloons burst. psycnet.apa.org/record/2021-...

Markus Steiner exploding balloons for science.

Ruben C. Arslan@ruben· Dec 18

Maybe treat it like a small, potentially biased pilot sample — which you get for free and for more item/scale pairs than any pilot sample would have patience for.

Ruben C. Arslan@ruben· Dec 18

There's still some parts we struggle with. In the pre-trained model, the scatter plot looks like a 🥐. We would like our fine-tuned model's scatter plot to look like a 🥖. But currently the prediction interval (golden area) still looks like one of the misshapen loaves I tend to bake.

the golden area is the prediction interval.

Ruben C. Arslan@ruben· Dec 18

Finally, @bjoernhommel.bsky.social's and my paper introducing the SurveyBot3000 is officially out in AMPPS. It's a fine-tuned language model that guesstimates correlations between survey items from text alone. Not perfectly, but useful for search, for example. journals.sagepub.com/doi/10.1177/...

Fig. 3. Scatter plots of the synthetic and empirical estimates, validation study (Stage 2). Showing N = 30,135 item-pair correlations, N = 257 scale reliabilities, and N = 1,568 scale-pair correlations for (top) the pretrained SBERT model and (bottom) the fine-tuned SurveyBot3000 model. SBERT = all-mpnet-base-v2 model.

Fig. 4. Prediction error of the synthetic estimates, validation study (Stage 2). Our prediction model allowed the error term to vary freely according to the predictor, the synthetic estimate. The thin-plate splines show that some synthetic estimates were predictably more accurate.

Fig. 5. Accuracy by domain. Accuracy differed across domains. SurveyBot3000 accuracy (colored) was always higher than SBERT accuracy (gray). Results were largely consistent whether accuracy of items was tested (left, circle) within domains or (right, cross) across domains.

Fig. 1. Multistep training procedure for the SurveyBot3000, which produces synthetic estimates of interitem correlations. (a) Pretraining base model (SBERT). (b) Fine-tuning SurveyBot3000. (c) Validation. SBERT = all-mpnet-base-v2 model.

Ruben C. Arslan@ruben· Dec 6

Lefthandedness also tends to yield low estimates of heritability. Interestingly, both handedness and sexual orientation are traits that most people (including on the political left) consider quite innate.

Plot from a master thesis I supervised, building on Willoughby et al. Lefthandedness, breast cancer, and sexual orientation stand out as traits where lay estimates for being mainly genetic are much larger than twin study estimates.

Ruben C. Arslan@ruben· Dec 6

The landscape is about "tiny initial differences [that] have large and irreversible final consequences". But most of what behaviour geneticists study is normally distributed, less Waddington landscape, more Galton board, where the ball "scampers deviously down", where differences often balance out.