Loïc A. Royer 💻🔬🧪
@loicaroyer
Director of Imaging AI & Sr. Group Leader @czbiohub SF. Researcher at the meeting point of #AI, #Microscopy, and #Biology using #zebrafish as model — co-creator of @napari_imaging and #Omega. #devbio #zebrahub #deeplearning #lightsheet #Imaging #dataviz
Bonus: edge features are easy to correct. A logistic head on *frozen* HOCT features cuts tracking error 59% with just 400 annotations – vs 6.75% for LoRA fine-tuning a node-Transformer baseline (~2 min/round vs ~40–55 min).
On a dense bacteria-division benchmark, HOCT gets the **lowest AOGM with zero deletion errors** using only 19 hand-crafted geometric features – beating Trackastra variants that rely on big pretrained vision encoders (SAM2.1, DINOv2, microSAM)."
We submitted a *single* model trained jointly on all 16 CTC datasets (2D+3D, brightfield/fluorescence/phase-contrast). It ranks **#1 in overall generalizability** across CLB, LNK, and BIO – reaching the top 3 on 14 of 16 datasets.
HOCT's fix: work in *edge* space. Each candidate link is its own token, and tokens attend to one another biased by inter-link **geometry**, not graph adjacency. A key ingredient: a per-head learnable bias from the 3D line-to-line distance between edge pairs.
Most methods embed each detection (node) and match by similarity. But: (1) at a division, a parent must sit close to *both* daughters, merging lineages that should stay separate; (2) the candidate graph is non-homophilic – 71% of a true link's neighbors carry a conflicting label.
📣 Tracking cells through time – including divisions – is a core problem in live-imaging microscopy. We built HOCT, an edge-centric Transformer that hits state-of-the-art on the Cell Tracking Challenge without a pretrained image encoder. Work led by @jookuma.bsky.social arxiv.org/abs/2607.11754 🧪🔬💻
8/ Different scale entirely: predicting zebrafish embryo cell-type composition across development (zscape atlas). Under strict timepoint holdout, TabPFN is the strongest learned model (Spearman 0.808, R² 0.704), ahead of the domain-specific Prophet.
6/ And it's not a compute trade-off. The tabular models run zero-shot in ~3s/fold. PRESAGE is trained from scratch each fold -- ~80× slower -- for no accuracy gain. TabICL at 25% of the training data often matches specialists trained on 100%.
5/ Pseudobulk perturbation prediction, 5 Perturb-seq datasets. TabICL & TabPFN give the lowest relative MSE on every dataset -- beating PRESAGE, scGPT, scLAMBDA, and CatBoost. scGPT sits near the mean baseline.
1/ 🧬🧪 New preprint! Do you actually need a biology-specific foundation model to predict how cells respond to perturbations? Surprising answer: a general-purpose tabular model - never trained on a single cell - matches or beats the specialists. 🧵 📄 doi.org/10.64898/2026.06.28.735106 @biohub
4/ 🗺️ This is the problem behind Zebrahub (the Biohub's "Google Earth" of zebrafish development) and Ultrack 📄, our method in Nature Methods 2025 that topped the Cell Tracking Challenge on dense 3D embryos. Now it's your turn to push the state of the art!
3/ ⚠️ Why it's hard: high cell density, imaging noise, and irregular shapes break automated trackers, and small per-frame errors compound into broken lineages. 🪢 Today, scientists fix this by hand, with countless hours of manual tracking. That's the bottleneck.
2/ 🧫 The data: 3D+time light-sheet movies of fluorescently labeled zebrafish embryo cells (~88 GB training data, stored as Zarr). Thousands of similar-looking nuclei moving, deforming, and dividing. Biggest Dataset EVER! Your task: detect every cell, link it across time, and catch every division. ⏱️
1/ 🧬🔬 How does one cell become an entire animal? We can now film it in 3D 🐟, but turning those movies into accurate cell tracks is brutally hard. Enter our new #Kaggle challenge: $60K prize pool, to help crack 3D+time cell tracking at scale! 🧵👇 www.kaggle.com/competitions... @biohub.org
"No one is remembered for the hills they climbed, but for the mountain they dared scale."
8/ We are aware of other efforts developed independently, such as the project by Haichao Miao at LLNL: github.com/llnl/bioimag.... Try it too!
7/ Power + safety: napari-mcp can expose code execution + package installation tools, so treat it like a local instrument (not a public toy). Docs: royerlab.github.io/napari-mcp/ Contributions welcome: issues/PRs on GitHub.
6/ Already in napari? Use plugin mode. Plugins → napari-mcp: MCP Server Control → Start Server. Your MCP client now controls the *current* session (no new viewer), so it slots into real workflows. royerlab.github.io/napari-mcp/g...
5/ Quick start: 1) pip install napari-mcp 2) napari-mcp-install <app> (claude-desktop / cursor / cline-vscode / gemini / codex)https://royerlab.github.io/napari-mcp/getting-started/quickstart/
4/ The fun part: your assistant can *see* what is on the napari canvas and use that to reason and decide what to do next! With visual feedback, the agent can observe, reason, and iterate rather than guess. royerlab.github.io/napari-mcp/a...
3/ napari-mcp exposes tools for layer management, viewer control (2D/3D, camera, dims), widgets, screenshots, and more! You get the best of both worlds: first-class agentic interfaces + napari viewer! API: royerlab.github.io/napari-mcp/a...
2/ Context: Omega (napari-chatgpt) showed that LLM agents can do real work inside napari (processing + widgets + self-repair). napari-mcp is the next step: any MCP-enabled app can now drive napari! github.com/royerlab/nap...
1/ 🚀 Excited to release: napari-mcp - agentic control of napari from any MCP-capable AI assistant! Use it for interactive image processing, analysis, and visualisation! Really cool project from @ilan-theodoro.bsky.social from my team! #napari #bioimageanalysis #MCP @biohub
