Martin Zettersten

@mzettersten

Asst Prof UCSD Cognitive Science PI LIL Lab UCSD: language development | cognitive development | learning (he/his)

1.696 Folgt1.788 Follower74 Posts

Bluesky

Ist das dein Profil?Übernimm es — dann gehört dir mzettersten.dirty.social.Profil übernehmen

Martin Zettersten@mzettersten· 26d

Rec #4: Match targets and distractors on animacy (+anything that affects saliency). Images of animate things grab infants' attention. This makes accuracy look worse for inanimates when matched w/ an animate. If saliency is mismatched for target vs. distractor, it can completely mask word knowledge

Looking time profiles for different combinations of target and distractor animacy. The first panel shows overall accuracy across ages; the second panel shows split by age category (4 groups: less than 18 months, 18-23 months, 24-35 months, 36 months and up). When animacy is matched between target and distractor, the time course of looking to the target across a trial for both animate and inanimate targets is very similar, increasing sharply after the onset of the word and then gradually declining. However, when animacy is mismatched, the time course for animate targets is inflated (when paired with an inanimate distractor) and the time course for inanimate targets is depressed (when paired with an animate distractor). The patterns are similar across age, until the 36 months and up group, at which point all time course curves converge (i.e., animacy mismatches no longer strongly bias looking).

Martin Zettersten@mzettersten· 26d

Rec #3 (actually #5 in the paper, there's a lot!): Increasing # of trials per participant really matters for reliable and valid measurement. Especially for individual difference studies, it's essential to increase the number of observations per participant as much as possible to have reliable signal

Simulations of measurement properties for different trial counts by child, created by downsampling trials for all children with different trial minimums. Basically, increasing the number of trials leads to continuous increases in test-retest reliability and validity (correlations with CDI measures). The largest gains come early (including 5-10 trials) and then the slope decreases somewhat (though reliability and validity continue to increase monotonically with increasing observations)

Martin Zettersten@mzettersten· 26d

Rec #2: Avoid baseline correction. One common way to deal with saliency differences between images is to correct accuracy by subtracting baseline looking to the two images. We find: this practice substantially decreases reliability: noisy measure - (correlated) noisy measure = extra noisy measure!

Lollipop plot showing comparison between baseline-corrected accuracy in red and accuracy in blue across datasets and measures. Baseline correction tends to have substantially better reliability (and little impact on validity as measured by the correlation with a parent-report measure of infant word knowledge, the CDI)

Martin Zettersten@mzettersten· 26d

Rec #1: Measure accuracy over a long analytic window. Researchers tend to constrain analyses to shorter time windows that end just after when average looking to the visual referent peaks. We find: expanding the time window noticeably improves reliability (the later parts of the window have signal!)

Heat map of different parameter options for measuring accuracy and how they on average affected ICC. Window start time is on the x-axis and window end time is on the y-axis. The "hottest" regions (highest reliability) tend to be associated with longer window end times (3000-4000 ms)

Martin Zettersten@mzettersten· 26d

LWL has been an incredibly useful paradigm for studying language development. BUT there are many, many decisions that go into analyzing complex timecourse data and designing studies. Our idea: let's use Peekbank to take a data-driven look at what choices improve measurement. Our top recs below.

Overview of the looking-while-listening design and common measures. The top row shows a basic trial: a picture of a dog is paired with a picture of a cat, and presented with a label ("Find the dog!"). The bottom row visualizes two common measures: accuracy (the proportion looking to the target across an analysis window ranging from shortly after the target word to some later endpoint) and reaction time (how quickly infants shift from looking at the cat image to looking at the dog image after hearing the word "dog").

Martin Zettersten@mzettersten· Jul 22

also, check out the supplement either at the journal or in the preprint (osf.io/preprints/ps...) for a ton more on looking time data and measurement, including one of my favorite figures, showing how test-retest reliability depends on both the number of trials and the sample

Results of a simulation showing how test-rest correlations increase with the number of trial pairs included in the analysis. The data is facetted by the participant sample, determined by the number of overall trials required for inclusion (2-16 overall trials). The x-axis depicts the number of trial pairs (randomly resampled per participant with replacement) in the analysis, and the y-axis shows the mean Pearson correlation coefficient between test and retest IDS preference (difference score). The plot shows two key things: (1) test-retest reliability goes up as the number of randomly resampled trial pairs increases, across all samples (indicated by an increasing linear fit). More observations leads to higher test-retest correlations. (2) The slope increases as the sample becomes more restrictive (more trials needed for inclusion). This provides evidence that there also seems to be more signal generally in the restricted sample of participants who contribute the most data.

Martin Zettersten@mzettersten· Apr 18

At the same time, the 2 evidence sources still do not fully agree. There are key differences in the effects of moderators. E.g., ManyBabies1 found that IDS preference was larger for older infants & headturn-based designs. We did not find similar effects in the meta-analysis. 7/

Three-panel figure providing an overview of the distribution of effect sizes in the meta-analysis and replications for three key moderators: infant age (A), method (B), and test language (C). In (A), the magnitude of IDS preference increases with age in the replication dataset, but not the meta-analysis (the effect size stays relatively constant across age). In panels B and C, the effect is larger for the headturn preference procedure and for native language speakers in the replication dataset, but if anything the effect appears to go in the opposite direction in the meta-analytic dataset.

Martin Zettersten@mzettersten· Apr 18

In the end, this led us to a pretty satisfying result: the meta-analysis and the multi-site replication almost *perfectly* agree on the average effect size (d~0.35). Check out the squint-inducing plot below with *many* effect sizes. 6/

A forest plot summarizing the effect sizes of infant-directed speech preference for the updated meta-analysis and the multi-site replication (ManyBabies 1). There are roughly 100 effect sizes in each dataset, and the upshot is that the average effect size for each dataset is approximately 0.35.

Martin Zettersten@mzettersten· Sep 25

3 tidbits: (1) The effect is bigger in adults than kids - and rule-based categories are quite hard for kids. (2) Kids' verbal knowledge of specific feature names correlates with learning (suggesting a connection w/ language) (3) Love these goofy aliens in the kid-friendly version of task. (2/2)

A figure from the paper with two panels. The first panel shows the stimuli and category structure in the experiment. The stimuli are pie-like circles with three colored wedges, which are the three distinct features that can predict category membership (for example, one stimulus has a purple, red, and blue wedge). The key manipulation is that the stimuli on the left of the panel are made up of colors that are easy to name, while the stimuli on the right are made up of colors that are hard to name (for example, chartreuse, sienna, mauve). The second panel shows the basic design of the task. There are two aliens on each side of the screen, and participants sort the stimuli to one of the two aliens. The cover task is that participants are supposed to figure out which alien would like to eat the “snack” (the category stimulus). There is also a progress bar at the bottom of the screen that shows the number of “snacks” (trials) remaining.

Martin Zettersten@mzettersten· Sep 5

Was anything stable across test sessions? It turns out the answer is YES. Infants’ average *overall* looking times between sessions (and also how many trials they contributed) was robustly correlated across sessions. It’s just that *preferential* looking was not consistent (5/6)

Figure with three panels. Panel A: Correlation between the number of trials contributed in Session 1 and Session 2. There are similar patterns for each testing method. Panel B: Correlations in average looking time between Session 1 and 2. Panel C: Correlations in average looking time between sessions for each condition (infant-directed speech and adult-directed speech). The correlations in all three panels are relatively strong (r of around 0.4-0.5).

Martin Zettersten@mzettersten· Sep 5

Interestingly, as we increased the number of trials required for inclusion, test-retest correlations increased somewhat (more trials per kid helps!)- but even the largest correlations were quite small (and likely too small to be practical for studying indiv differences) (4/6)

Figure showing the correlation between infant-directed speech preference in the first and second session for increasingly strict inclusion criteria requiring participants to contribute more trials. Correlations are small when participants must contribute at least 2 or 4 trial pairs (r < .10), but somewhat larger when participants have to contribute at least 6 (r=0.22) or 8 (r=0.34) trial pairs. At the same time, the total sample size gets dramatically smaller

Martin Zettersten@mzettersten· Sep 5

So, as part of ManyBabies1, a number of labs brought in babies for a second test session. Despite a large infant sample (N=158), we saw no evidence of test-retest reliability in preregistered analyses. Correlations between looking time in session 1 and 2 was small (r=.09) (3/6)