Lisa DeBruine

@debruine

Professor of MetaScience 🏳️‍🌈 she/they #metacheck #rstats #PsyTeachR #OpenResearch #ManyFaces #iyknyd (overwhelmed by social media)

407 Folgt7.107 Follower599 Posts

Bluesky

Ist das dein Profil?Übernimm es — dann gehört dir debruine.dirty.social.Profil übernehmen

Lisa DeBruine@debruine· 16d

Post a perfect album from the 90’s that isn’t Nirvana, Soundgarden, Pearl Jam, or Alice In Chains

Tori Amos - Little Earthquakes album cover

Tori is wearing jeans and crouching inside a wooden box with a tiny toy piano

Lisa DeBruine@debruine· 19d

gartmorehouse.com/course-calen...

A huge Georgian mansion (Gartmore House)

Lisa DeBruine@debruine· 19d

I have spent the past week at a crafting workshop with my girlfriend. Best birthday present I ever gave someone! We’ve learned so much about leatherwork and had lots of walks in the woods (where we ate so many wild 🍓). 10/10 perfect holiday

Assorted picnic things made with leather: a wicker basket with leather straps and handle, a blanket roll, a cutlery roll, a coin purse, a flask cover, a notebook cover, and a small elephant charm.

Lisa DeBruine@debruine· 25d

Happy Glasgow Pride!

A person with blue hair and a rainbow scarf and rainbow face jewels.

Lisa DeBruine@debruine· 26d

Haha, I was just taking pictures of this on my lunch break!

An outdoor concrete staircase photographed from a low angle.

Lisa DeBruine@debruine· 28d

A pair of centaur parents in the distance, looking at three centaur babies. The horse part is galloping, while the human part is a floppy baby.

Lisa DeBruine@debruine· Jun 12

🏳️‍⚧️🏳️‍🌈 such a pretty campus @glasgow.ac.uk

Inclusive pride flag flying over the University of Glasgow Gilbert Scott building.

Lisa DeBruine@debruine· Jun 5

The IDEA framework outlines the steps needed to make trustworthiness assessment. The first three steps: ingest, define, and extract, can be done deterministically, letting the last step, adjudicate, to be LLM-assisted or done entirely by humans.

Ingest research
materials in structured, systematic manner

Define dimensions along which to compare the research objects

Extract text from each object which best relates to a given dimension of comparison

Adjudicate on whether the extracted text matches between research objects

Lisa DeBruine@debruine· Jun 5

Why not just throw these problems at an LLM?

Why not just throw this at an LLM?

* They are very sensitive to different prompt features, input parameters, etc.
(Cummins, 2026; Sclar et al., 2023) - we need standardized inputs
* They will likely be accessed via commercial chat interfaces, which are not appropriate for robust research activities (Cummins et al., 2025)
* Users will use different models and we will have little control over this
* Data privacy will likely be an afterthought
* Asking an LLM to do comparisons all at once (e.g., sample size, hypothesis tests) necessarily introduces dependencies between these, which makes reproducibility and meaningtul evaluation harder to achieve
* All of the above can be resolved by using a bespoke, modular workflow

Lisa DeBruine@debruine· Jun 5

Keynote speaker @jamiecummins.bsky.social will speak about trustworthiness assessment tools and his tool RegCheck for checking the correspondence between manuscripts and registrations.

A Conceptual and Technical Framework for Automating Research Trustworthiness Assessment

Dr Jamie Cummins

Lisa DeBruine@debruine· Jun 4

I miss bagged milk. And No Name

A bunch of yellow products ona yellow background with simple names.

Lisa DeBruine@debruine· May 18

I just looked at my 2002 edition of Information Architecture for the World Wide Web and found this postcard you could mail in to get a physical copy of the O’Reilly catalog!

Postcard with a picture of a chihuahua reading:

Register your O’Reilly Book!

Lisa DeBruine@debruine· May 7

Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials pmc.ncbi.nlm.nih.gov/articles/PMC...

Results We were unable to identify any randomised controlled trials of parachute intervention.

Conclusions As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.

Lisa DeBruine@debruine· May 6

I’m at the @improvingpsych.org online conference at a session about #PsyArXiv moderation, which switched from a post-moderation system (DOI assigned before moderation) to a pre-moderation system in 2025 for reasons that should be apparent on this plot.

Graph of new submissions by month from 2017 to 2026. Submission increased a bit in 2020 and doubled again in 2025.

Lisa DeBruine@debruine· Apr 30

I had to film a short segment for a uni video today in my role as academic lead for good research practice, and managed to sneak in my #Rstats sculpture from @philmcaleer.bsky.social and the @jaspstats.bsky.social bear from EJ, along with the “pile of books” they asked me to decorate my desk with.

Lisa next to a blue R and a small stuffed bear with a JASP t-shirt sitting on top of a pile of books

(Queer Data, The 7 Deadly Sins of Psychology, Invisible Women, How to Design and Report Experiments, Linux in a Nutshell, Data Analysis Using Regression and Multilevel/Hierarchical Models, and Hays Statistics)

Lisa DeBruine@debruine· Apr 14

This doesn’t really make me want to “Sign up for Nature Briefing: AI and Robotics” (I think Nature Scientific Reports might be best classified as an unreliable low-quality journal now.)

Results and discussion

This part show how change in number make line in graph up or down for speed, spin, hot, and mix in layer. Solve in MATLAB BVP4C and get line and table for skin friction, Nusselt, Sherwood. Magnetic M make speed small, spin more near wall, and hot more because electric hot. Micro N change spin and speed near sheet. Light heat R make hot layer thick and more hot in flow. Soret Sr make mix from hot change, Dufour Du make hot from mix change and less near wall. Chemical Kr eat mix near sheet. Ohmic hot from M and σ put more hot inside, viscous warm Ec make more hot from speed change. When hole in medium big (φ), speed go small, hot and mix go more. Permeability K change how easy flow go in hole. All show in figure line go up down, use in filter, oil take, heat machine, and many place in work.

Lisa DeBruine@debruine· Apr 14

A friend sent me this and said the abstract reads like it was written by a dog with those talking buttons. And OMG she was right! 🐶🔘 www.nature.com/articles/s41...

This work look for Soret and Dufour thing on MHD flow for micropolar fluid on sheet with hole stretch, also see hot from electric and warm from move, which old study no much talk. These hot and mix change speed, hot, spin, and mix of fluid in layer. The big math changes to small math by same change and solve in MATLAB BVP4C. Picture line and table show number for skin friction, Nusselt, Sherwood. We see more M make slow speed, light heat make thick hot layer, more micro make more spin near wall. Also, hot from electric and warm from move make more hot, so need in work like earth heat take, oil get, and hot change machine in hole thing.

Lisa DeBruine@debruine· Apr 7

Lisa DeBruine@debruine· Mar 27

Not a tech bro, but my opinions on overthrowing the scholarly publication industry do predispose me to be more positive about industry disruption. amiatechbro.com

I scored 11% all in the disruption category.

Lisa DeBruine@debruine· Mar 26

The @ukrepro.bsky.social just published a new primer on computational reproducibility, authored by the CR special interest group, including yours truly! I hope it's useful for introducing people to the concept and convincing resource-holders to fund efforts to improve it. zenodo.org/records/1923...

Lisa DeBruine@debruine· Mar 2

Crossref is great for journal articles, and datacite is great for datasets and software. Bibtex is good for citations, but not as detailed. I really tried to avoid it, but I've been forced into creating a 15th standard to rule them all :( Let me know if there's something I've missed. #schema

XKCD comic:

Title: HOW STANDARDS PROLIFERATE: (SEE: A/C CHARGERS, CHARACTER ENCODINGS, INSTANT MESSAGING, ETC.)

Panel 1: SITUATION: THERE ARE I4 COMPETING STANDARDS.

Panel 2:

character 1: 14?! RIDICULOUS! WE NEED TO DEVELOP ONE UNIVERSAL STANDARD THAT COVERS EVERYONE'S USE CASES.

character 2: YEAH!

Panel 3: SOON: SITUATION: THERE ARE I5 COMPETING STANDARDS.

YEAH!

Lisa DeBruine@debruine· Aug 8

I've started 6 chapters in the past 3 days. It's very draft content, but I'd love feedback on whether this is the sort of thing people might use and what you'd want out of it. What metadata about each version would you want to be able to decide if a version is good for a lesson you're planning?

A plot showing water access by development level from 2000 to 2022. High-income countries have nearly 100% safe access, while access steadily worsens for lower income levels, but steadily improves over time.

Lisa DeBruine@debruine· Jul 19

Happy Glasgow Pride! ❤️🧡💛💚💙💜🩷🩵🤍🤎🖤 (The #NoPrideInGenocide event at Kinning Park is fab)

A face in a black mask with rainbow gem eyeshadow, standing in front of a giant trans flag

Lisa DeBruine@debruine· Jul 2

Ed Gerstner from @springernature.com presents data from their survey on researcher perspectives on evaluation practices stories.springernature.com/state-of-res...

METRICS DOMINATE EXISTING ASSESSMENT PRACTICES, BUT MOST RESEARCHERS DESIRE BALANCE

• The majority of respondents told us they are evaluated either entirely or mostly based on metrics — primarily publication metrics.
• All researchers want assessment to be more representative of their contributions other than research outputs.
• Compared to existing practices, there is also a clear desire to reduce reliance on metrics.

Plot showing that people think current assessment is skewed towards quantitative measures, but want more balance

At present, what is the mix of quantitative and qualitative methods for how your work is evaluated? (n=4,910)

In an ideal research assessment process, what would you suggest should be the weighting? (n=4,869)

Lisa DeBruine@debruine· Jul 2

@alexrushforth.bsky.social speaking about the Dutch national initiative to reform research assessment #MetaScience2025

Formative Appraisal

Where things have gone well

- Major establishment bodies bought in
- Withstood public opposition
- Grown the coalition
- (Positive) unanticipated effects and 'small wins'

Challenges

- Expecting too much, too soon (e.g. over half of sector claimed no meaningful benefits yet. Source: Culture Barometer 2024)

- Lack of awareness (53% of PhD researchers not aware of program)

- Persuading 'rank and file' gatekeepers and wider academic community

- Engaging with research fields and publishing reform movement

Lisa DeBruine@debruine· Jul 2

@lizziegadd.bsky.social from @coarassessment.bsky.social speaks on how you assess the success of a reform movement #MetaScience2025

DRIVERS OF ASSESSMENT REFORM

- Open research
- Research Integrity
- Equity, Diversion & Inclusion
- Research Culture
- Research careers

Lisa DeBruine@debruine· Jul 2

Bregt Saenen presents data from a European Survey of Strategic Approaches to, and Research Assessment of, Open Science #MetaScience2025 www.scienceeurope.org/our-resource...

Which elements of open science are part of the documented strategic approach followed by your organisation?

Open access to research articles
Findable, Accessible, Interoperable, Re-usable (FAIR) | research data (including for example data management
plans)
Open access to research data (including for example
metadata)
Open access to research books
Open research infrastructure
Stakeholder engagement with industry partners, policy makers, and/or the public (including for example science
communication)
Open source research software, code, and tools (including for example software management plans)
Citizen science
Open evaluation (including for example open peer review.
open and transparent indicators/metrics)
Open research methods (such as pre-registration of study designs, open protocols and workflows)
Service and leadership (such as engagement in open science policy development, teaching, and/or
supervision)
Open hardware
Open education

Outcome from a 2024 survey among Science Europe Member Organisations (Europe's foremost public research funding and research performing organisations) on their strategic approaches to, and research assessment of, Open Science. See scieur.org/ra-os-survey for more information.

Lisa DeBruine@debruine· Jul 2

Emmanuel Boakye, founder of the African Reproducibility Network is building bridges between communities in Africa #MetaScience2025

The African open science landscape, a slide with logos of organisations like Africa Open Science Hardware and AfricArXiv

Lisa DeBruine@debruine· Jul 1

Both optimistic and pessimistic conclusions from @tomhardwicke.bsky.social #MetaScience2025 Open practices are increasing, but not very fast. doi.org/10.1177/2515...

Plots showing significant but still quite small increases from 2015 to 2022 in prevalence of preregistration, open materials, data, analysis scripts, open access, conflict of interest statements, and funding statements

Lisa DeBruine@debruine· Jul 1

Modifications Required to Pass Reproducibility Verification