Auditory-Visual Speech Association (AVISA)

@avsp

The official(ish) account of the Auditory-VIsual Speech Association (AVISA) AV 👄 👓 speech references, but mostly what interests me avisa.loria.fr

980 Siguiendo1216 Seguidores748 Publicaciones

Bluesky

¿Es tu perfil?Reclámalo y avsp.dirty.social es tuyo.Reclamar perfil

Auditory-Visual Speech Association (AVISA)@avsp· Jul 28

Check out the Keynote speaker for AVSP 2026 project.inria.fr/avsp2026/key... This is going to be great!

Professor Michael Richardson
School of Psychological Sciences
Performance and Expertise Research Centre
Frontier AI Research Centre
Lifespan Health and Wellbeing Research Centre
https://orcid.org/0000-0001-9159-2774

Auditory-Visual Speech Association (AVISA)@avsp· Jul 14

Food has arrived at the MARCS HDR Forum 😋

A scene looking down from above of boxes of food

Auditory-Visual Speech Association (AVISA)@avsp· Jun 5

In which Greig coins a new concept "emantic processing" 👏

Lines 7 - 9 of the Abstract
"commentary, I argue that multiple sources of evidence, including failures to replicate key findings, demonstrate that emantic processing of action words and action-sentences is unlikely to rely on"

Auditory-Visual Speech Association (AVISA)@avsp· Jun 5

If you're in Sydney October 2nd - come and be a part of AVSP history! Hope to see you there. project.inria.fr/avsp2026/

A collection of logos from AVSP conferences over the years

Auditory-Visual Speech Association (AVISA)@avsp· Jun 3

It seems people now know about their Interspeech submissions ... if you're headed for Sydney please check out the Auditory-visual Speech Processing AVSP 2026 satellite workshop October 2 project.inria.fr/avsp2026/ The 2nd AVSP was in 1998 (after Interspeech 98) - this year will be the 16th edition!

Auditory-Visual Speech Association (AVISA)@avsp· May 23

******INTERSPEECH 2026 SATELLITE EVENT******** The AVSP 2026 Workshop 16th Conference on Auditory-Visual Speech Processing interspeech2026.org/en-AU/pages/... project.inria.fr/avsp2026/cal... We invite authors to submit either a 1-page abstract or a 4-page paper to this email: avsp2026@inria.fr

Auditory-Visual Speech Association (AVISA)@avsp· Apr 23

3026 gs citations by my count 😮 - Anyhow, good to be reminded of the classics 😀

Conclusion
In summary, I have argued that a bilateral temporo–frontal network subserves auditorysentence comprehension. Although syntactic and semantic information are processed predominately by the left hemisphere, processing of prosodic information occurs predominantly in the right hemisphere. Temporal regions support identification processes, with syntactic processes involving the left anterior STG, semantic processes recruiting the left MTG and prosodic processes involving the right posterior STG. Frontal regions, by contrast, support the formation of relationships, with syntactic relationships involving BA 44 and the frontal opercular cortex, and semantic relationships recruiting BA 45/47. These different areas within the network must be activated and coordinated to achieve auditory sentence comprehension. The timing of the syntactic processes of structure building precedes, and are initially independent of, semantic processes, although both interact during a later processing phase. Prosodic processes influence syntactic processes, however, the exact timing of this is a subject for future research.

Auditory-Visual Speech Association (AVISA)@avsp· Mar 24

dnhkng.github.io/posts/rys-ii/ 'Maunder:a beautiful experiment,fed model 3 different language same-meaning sentences, tested layer's hidden state cosine similarity: if mid layers a thinking space, then same topic different language sentences more similar than different topic same language ones'>MORE

Graph that shows Centered cosine similarity. Mean/STD spread-band plots - Illustrates Three phases:

Encoding (layers 0–5). Wild oscillation. The model is doing heavy lifting to normalise radically different surface forms. The model knows it’s reading English or Chinese, and the representations reflect that.

Reasoning (layers ~10–45). The red line dominates. These are the cross-language same-content pairs. The model has converged on a representation where content identity matters far more than language identity. Same-language different-content pairs (green) are much lower. And cross-language different-content (blue) has negative cosine similarity. This is the format-agnostic reasoning space that Part 1 hypothesised.

Decoding (layers ~45–64). The latenst re-differentiates. The model is preparing to emit tokens, and it needs to commit to a specific language and format. Cross-language similarity drops. The representations become surface-specific again.

This is the three-phase architecture, directly visible in a single forward pass. The encoding region is narrow, just a handful of layers. The reasoning region is quite broad, the bulk of the transformer stack. And the decoding region is where everything unravels back into language-specific token predictions.

Auditory-Visual Speech Association (AVISA)@avsp· Feb 20

Horse mussel (Modiolus modiolus)

Auditory-Visual Speech Association (AVISA)@avsp· Dec 23

Am Psychol. 2025 Dec;80(9):1350-1352. doi: 10.1037/amp0001601IF: 7.2 Q1 .

ABSTRACT

The Awards for Distinguished Scientific Contributions are presented to persons who, in the opinion of the Committee on Scientific Awards, have made distinguished theoretical or empirical contributions to basic research in psychology. The 2025 recipients of the APA Distinguished Scientific Contribution Awards were recognized by the 2024 Board of Scientific Affairs and selected by the 2024 Committee on Scientific Awards. For pioneering research on gesture in language creation, language production, language learning, and cognition, Susan Goldin-Meadow is a 2025 award winner. Susan Goldin-Meadow has studied gesture in deaf children who do not receive structured language input, and she has found that the gesture systems that these children generate-called homesign-are highly structured and language-like. She has studied the gestures people produce along with speech, and she has shown that such gestures both reveal and influence thinking. Her work has shaped scientific understanding of human development, learning, and communication. She is a brilliant scientific communicator, a generous leader in the scientific community, and a dedicated and inspiring mentor.

Auditory-Visual Speech Association (AVISA)@avsp· Dec 17

Video-call glitches trigger uncanniness and harm consequential life outcomes www.nature.com/articles/s41... 5 experiments + 3 supplementary ones using both live & recorded interactions suggest that minor AV glitches during video calls can harm interpersonal judgements in consequential life domains

Auditory-Visual Speech Association (AVISA)@avsp· Nov 14

Front page note on Perceptrons draft "We are anxious to collect: ...3. statements that are clear but false....Diligence will be rewarded, somehow or other" : )

This is a nearly final" draft of our analysis of the geometric ability of linear--separation machines. The book will be published by the MIT Press, fairly soon, and will contain some additional material on learning and on relations between serial and parallel machines. We are anxious to collect: 1. simple errata, typos, etc., 2. sentences or sections that are not very clear, 3. statements that are clear but false, 4. hard-:to-understand notations and illustrations. Please make a list of everything you find along these lines. Mathematicians have drifted into a set of conventions for obscurity in exposition, that make it very hard to understand their basically simple ideas. We are trying to resist this in many ways. Therefore we will be especially grateful for every suggestion that will make the book easier to read. But we need your comments around the middle of April. Diligence will be rewarded, somehow or other. Send comments to M. Minsky Room 817 545 Tech Square.

Auditory-Visual Speech Association (AVISA)@avsp· Sep 22

Graduation vs. reality meme - gradate throws cap in air - 2 weeks later at 3:00 AM trying to remember the code

Auditory-Visual Speech Association (AVISA)@avsp· Mar 11

Next year - this t-shirt : )

T-shirt with Mundus sine caesaribus written on it - what Bluesky CEO Jay Graber wore at SXSW 2025

Auditory-Visual Speech Association (AVISA)@avsp· Mar 3

Resilience & vulnerability of neural speech tracking after hearing restoration www.nature.com/articles/s42... Neural speech tracking in children with/without functional hearing in their 1st year of life after cochlear implants - partial resilience & specific vulnerabilities

Auditory-Visual Speech Association (AVISA)@avsp· Feb 24

Jiwar: A database and calculator for word neighborhood measures in 40 languages link.springer.com/article/10.3... Neighbourhood information for 3 levels (orthographic, phonological, and phonographic) across 40 languages

neighborhood information across three levels (orthographic, phonological, and phonographic) across 40 languages.

Auditory-Visual Speech Association (AVISA)@avsp· Feb 16

ABSTRACT

Siblings of children with developmental language disorder (DLD) often have weaker language skills compared to peers with typical development (TD). However, whether their language-relevant audiovisual skills are also atypical is unknown. Study 1 examined whether siblings use information about a talker's mouth shape during phonemic processing as children with TD do. Study 2 examined siblings' ability to match auditory words with observed word articulations. Only children with TD showed a significant MMN to audiovisual phonemic violations, suggesting that, just like in children with DLD, lip shape does not modulate phonemic processing in siblings. Children with DLD and siblings were also less accurate than children with TD at detecting audiovisual word mismatches. The N400 amplitude in children with TD was significantly larger than in children with DLD and marginally larger than in siblings. Phonemic and lexical representations in siblings lack audiovisual details, which may contribute to poor language development.

Auditory-Visual Speech Association (AVISA)@avsp· Feb 12

The neuroethology of ant navigation www.cell.com/current-biol... A working model of the neural basis of the multimodal navigational strategies of ants - outlines the anatomy & functioning of major central brain areas & neural circuits involved in the coordination of navigational behaviour 🐜📡

The ant central complex (CX) and lateral complex (highlighted in colour) are embedded in the centre of the ant brain (top left) and located in close proximity to the mushroom bodies (MBs). Data from flies and moths, as well as other insects, have crystallised the role of the CX as an integrative centre to compare the head direction and goal direction of the insect and to instruct the motor system to make corrective turns. The head direction representation is based predominantly on visual information from the optic lobes (OL). The lateral accessory lobes (LALs) are known as premotor control centres that receive CX output and contain reciprocally connected neurons that, at least in moths110, drive oscillatory steering behaviour. The mushroom bodies (MB) are the likely location for storing view memories that contribute to defining the momentary goal of the insect. Images from the Insect Brain Database148, data from103. (B) Conceptual model of how view learning and corrective steering could be phase locked to zigzagging behaviour110. Left: hypothetical neural activity of lateral accessory lobe (LAL) neurons during ant zigzagging behaviour. Activity patterns are inspired by data from silkworm (Bombyx mori) neurons that control the tracking of pheromone plumes via zigzagging behaviour114. In addition to neurons regularly alternating between up and down states (indicating right/left), other cells show phasic excitation when the states switch (blue). Such neurons in an ant would be highly suited to drive the phase locking of corrective turns. Right: suggested role of dopaminergic feedback from the LAL to the fan-shaped body (FB) and MB to phase lock the occurrence of saccade-like turns and view learning to the turning points of behavioural oscillations, as signalled by hypothetical phasic LAL neurons. Neuroanatomical images © Insect Brain Database (CC BY 4.0)

Auditory-Visual Speech Association (AVISA)@avsp· Jan 17

Old memes never die : )

Auditory-Visual Speech Association (AVISA)@avsp· Jan 14

Yep, checks out!

Claude gives Tal's idea a favourable rating :)

Auditory-Visual Speech Association (AVISA)@avsp· Jan 12

Cartoon shows a person very concerned with the safety of another - punchline "that's reviewer one"

Auditory-Visual Speech Association (AVISA)@avsp· Jan 1

Music-reading expertise associates with configural face processing but not featural face processing www.sciencedirect.com/science/arti... Behavioral & ERP data suggested that music-reading expertise was associated with configural face but not featural face processing

Auditory-Visual Speech Association (AVISA)@avsp· Dec 21

Let me get this straight, you paid how much for that?

See https://arcprize.org/blog/oai-o3-pub-breakthrough and https://simonwillison.net/2024/Dec/20/openai-o3-breakthrough/

Auditory-Visual Speech Association (AVISA)@avsp· Dec 11

Advice on how to use LLMs

Shows the painting "Ulysses and the Sirens" (1891) by John William Waterhouse (1849-1917) with the sirens labelled as LLMs

Auditory-Visual Speech Association (AVISA)@avsp· Nov 27

Yep - credit assignment is it. Hard enough in organizations ;) Challenging in distributed systems; especially when information is aggregated over time & space. This www.nature.com/articles/s41... argues the brain doesn't use back-prop but solves it using a different principle -what say you?

Illustrations (hand drawn) of something meant to be neurons : )

Auditory-Visual Speech Association (AVISA)@avsp· Nov 22

The [‪Blue] Sky's the limit :) modified @simonwillison.net‬ /Claude script -Jetstream WebSocket proxy interface- added upload txt file of searchables & output the time these were 1st found in the stream. Here a search for 17 city names >ran it 9 times & plotted-some cities mentioned quick others...

Plot of how long (secs) it took to find a city name in the BlueSky stream - The cities: New York Paris London Rome Tokyo Los Angeles Shanghai Moscow Singapore Chicago Athens Berlin Cairo Sydney Bangkok Delhi Seoul

Auditory-Visual Speech Association (AVISA)@avsp· Nov 20

Absolutely, adapted @simonwillison.net script to get search terms from a file and time how long it took before a hit. Currently trying city names - been waiting on a hit for Cairo for a while... [2024-11-20T08:28:53.756Z] Found term "New York" in 70.146 seconds ...Found term "Paris" in 2.848 seconds

A screen grab of a browser window showing a feed from the Blue Sky WebSocket -

Auditory-Visual Speech Association (AVISA)@avsp· Nov 20

Bluesky network analyzer bsky-follow-finder.theo.io (made by @theo.io) Enter your Bluesky handle below to find people followed by lots of the people you follow (but not you).

Auditory-Visual Speech Association (AVISA)@avsp· Nov 19