Ragnar {Groot Koerkamp}

@curiouscoding

Postdoc on high troughput bioinformatics @ KIT Karlsruhe; IMO, ICPC, Xoogler, Rust, road-cycling, hiking, wild camping, photography

135 Folgt1.064 Follower1.658 Posts

Bluesky

Ist das dein Profil?Übernimm es — dann gehört dir curiouscoding.dirty.social.Profil übernehmen

Ragnar {Groot Koerkamp}@curiouscoding· 9d

Travis has been cooking something up since IGGSy ;) A fast algorithm for SMEM finding in haplotype panels that uses space proportional to the number of 1s in the (sparse) matrix, inspired by the jump index [13].

[13] Ragnar Groot Koerkamp. Personal communication, 2026.

Ragnar {Groot Koerkamp}@curiouscoding· 17d

Something different: a little report of kayaking in Latvia curiouscoding.nl/posts/latvia...

4 kajaks lying on a beach next to where the river meets the sea. They're packed with bags.

Ragnar {Groot Koerkamp}@curiouscoding· Jul 9

The main k-PHF-set speedup figure, with our results in blue. On my laptop (top row), the k-PHF allows throughput very close to the maximal memory throughput up to n=2^28, whereas previous methods only reached that up to 2^24. Also, it performs equally well for both positive and negative queries.

Ragnar {Groot Koerkamp}@curiouscoding· Jul 9

So now, a non-minimal 8-PHF suffices, which only needs 0.08 bits/key at load factor 80%, compared to 0.86 bits/key for k=1 at load factor 80% or 1.44 bits/key for k=1 and a minimal PHF with load factor 1. Since it's 10x smaller, it fits in a lower cache and supports faster queries or larger inputs.

Table of space lowerbounds for various bucket sizes k and load factors alpha.

Ragnar {Groot Koerkamp}@curiouscoding· Jul 6

Some heated debate at the #iggsy poster session between Andre - Metagraph - Kahles and Evo - Barbell - Garrison!

Evo, a kid, wields the rolled-up Barbell poster in a fight against Andre with the metagraph poster rolled and folded up.

Ragnar {Groot Koerkamp}@curiouscoding· Jun 24

In other news: Great SEA talk by Nathaniel Brown on Orbit, an efficient implementation of the move structure for run-length encoded permutations. Also, congrats on winning a best paper award with this work!

Nathaniel standing in front of his title slide at the start of the presentation.

Ragnar {Groot Koerkamp}@curiouscoding· Jun 21

On the way to SEA! Might be my longest train trip so far, and will for sure be if we don't make the 30 min transfer in Hamburg.

DB app screenshot: train from Kilchberg (Zurich) to Copenhagen, leaving 4:25 and arriving 20:16.

Ragnar {Groot Koerkamp}@curiouscoding· Jun 11

OUCH This assertion does not do what you think it does...

A screenshot of Rust code:

const BIST: u32 = 31;
assert!(pos < (1 << BITS) as usize);

Ragnar {Groot Koerkamp}@curiouscoding· Jun 8

First thing I see on reddit after reading this post 🤔 Anyway: big-pan does not want you to know this one simple trick!

Reddit add for r/castiron writing "One pan to rule them all"

Ragnar {Groot Koerkamp}@curiouscoding· Jun 3

We get a figure like this, where the sweep line goes from left to right and the coloured regions indicate the best preceding anchor for each position. Most of the work is in maintaining the shapes of these regions, which we do with predecessor structures on the horizontal and diagonal boundaries.

Figure showing a 2D edit-graph with diagonal segments showing anchors, and red diagonal segments showing an optimal chain of anchors.
A coloured background of overlapping triangular-shaped regions with a horizontal top and diagonal bottom, with one colour per anchor, show the best anchor to use in each position.

Ragnar {Groot Koerkamp}@curiouscoding· May 26

So Gonzalo Navarro used 'diagonal transition' in his '01 ASM survey. Some digging by him now found that it (first???) appears in Chang & Lampe '92. In fact, the 'transition' is not between adjacent diagonals (as I thought), but *along* a diagonal where the cost increments. doi.org/10.1007/3-54...

$Navarro'01: Ukkonen (1983). In 1983, Ukkonen [1985a] presented an algorithm able to compute the edit distance between two strings x and y in O(ed(x, y)^2) time, or to check in time O(k^2) whether that distance was ≤k or not. This is the first member of what has been called “diagonal transition algorithms,” since it is based on the fact that the diagonals of the dynamic programming matrix (running from the upper-left to the lower-right cells) are monotonically increasing (more than that, C[i+1, j+1] ∈{C[i, j], C[i, j +1]}). The algorithm is based on computing in constant time the positions where the values along the diagonals are incremented. Only O(k^2) such positions are computed to reach the lower-right decisive cell.$

Chang & Lampe '92:
Diagonal Transition Algorithms:
Diagonal monotonicity implies the locations of the first k + 1 transitions along each diagonal are sufficient to characterize D for the solution to the k differences problem [20]. A key ingredient is the "jump" J(j, i) = length of the longest exact match P[j,...] = T[i,...].

Ragnar {Groot Koerkamp}@curiouscoding· May 26

Turns out I presented Sassy yesterday without realising it has been published! Finally officially a coauthor with @rickbitloo.bsky.social 😆 doi.org/10.1093/bioi...

Ragnar {Groot Koerkamp}@curiouscoding· May 22

My WABI submission: anti-lexicographic SUS-anchors. For sigma=4, this selection scheme (a k=1 sampling scheme) has density within 1% of the lower bound! The idea: find the smallest* substring that does not have a 2nd occurrence, and sample its start pos. curiouscoding.nl/posts/sus-an...

Plots with the density of the sus-anchor for sigma in 2, 4, 32 as a function of w. For sigma=4, they have an overhead over the lowerbound of <1%, while other schemes have >15% overhead.

Ragnar {Groot Koerkamp}@curiouscoding· May 20

Just tried it. So ChatGTP shows some pretty small bubbles. There's additional sources in the hover-popup, but hovering doesn't work on mobile... There's also a 'sources' button at the bottom, but on phone that's _also_ broken. On PC it actually shows also some sources that are never in a bubble :/

Screenshot of ChatGTP answering a question about the 'diagonal transition' algorithm, with a low-contrast bubble saying 'CuriousCoding' and +1.

Ragnar {Groot Koerkamp}@curiouscoding· May 12

Barbell: Fast & Accurate nanopore demultiplexing github.com/rickbeeloo/b...

Example of a barbell command showing the flanks and barcodes being searched.

The top 10 most common patterns are shown. 8000 reads are as intended, but many (>1000) have multiple matches of the flanks and/or barcodes.

Ragnar {Groot Koerkamp}@curiouscoding· May 12

Sassy: Fast approximate string matching. Aka: A fuzzy grep for fasta files. Available as both CLI and library. github.com/RagnarGrootK...

Sassy's grep-like output, with matches in green, and indels coloured red/blue.

Ragnar {Groot Koerkamp}@curiouscoding· Apr 29

With AVX-512, the SimdQuickHeap is consistently 2x faster than the radix heap and other engineered heaps that are I/O-efficient, and up to 10x faster than the binary heap and other tree-based heaps.

Log-log time-per-operation plots for different workloads. The x-ax has increasing input sizes, and the y-ax has the normalized time per push-pop pair of operations divided by lg n.
The binary heap, d-ary heap, and weak heap all slow down as n increases, while the I/O-efficient heaps all get faster. The SimdQuickHeap is 2x faster than the radix heap and up to 10x faster than the binary heap.

Ragnar {Groot Koerkamp}@curiouscoding· Apr 29

The data structure has some similarities to QuickSelect and is pretty simple: repeatedly partition the list of values using random pivots until the smallest is left, and store each part in its own array. New elements are compared to the pivots and pushed to the right layer.

Overview of the SimdQuickHeap data structure: a list (vector) of pivots is stored, and the elements between consecutive pivots are stored in their own bucket (also a vector). The elements in a bucket are in between the two pivots surrounding it.

Ragnar {Groot Koerkamp}@curiouscoding· Apr 19

It's 5:20 am, but look what I can do now 😊 Now fingers crossed that it's possible at all to write lecture notes that also work on slides.

Ragnar {Groot Koerkamp}@curiouscoding· Apr 14

Jeez... 6 in a day... this is gonna be so bad for the ecosystem long term. Even if none of these gains any traction, the name squatting and confusion/bloat that others will have to deal with just to figure out which crates are/are not serious is a pain.

crates.io screenshot showin 6 crates, most of which *-rs, all created 2 days ago

Ragnar {Groot Koerkamp}@curiouscoding· Apr 7

Once should be enough when not in meetings ;) But well; 7 newly uploaded crates in the last 2 days does not inspire confidence. Probably not worth my time to make a PR until it has credible maintenance and users?

crates.io screenshot showing 7 *-rs crates created in the last 2 days, like blast-rs, hmmer-pure-rs, and minimap2-pure-rs.

Ragnar {Groot Koerkamp}@curiouscoding· Apr 4

Have I advertised perfcnt yet? It's awesome! Super easy to add fine-grained perf counters benchmarks, for eg branch misses and last-level cache misses. (These things are otherwise annoying to get from `perf stat`, because that includes benchmark setup and such.) crates.io/crates/perfcnt

Screenshot showing the initialization of a perf counter that counts last-level cache misses. It then starts the counter, executes the function to be benchmarked, stops the counter, and reads its value.

Ragnar {Groot Koerkamp}@curiouscoding· Mar 19

More like: remove line numbers, remove stuff like the un-filled header fields below. And OUP footer. And stuff like [PROCEEDINGS] in the title that I now see presumably because of the Recomb-Seq submissions (as opposed to [OVERLAY]). Basically, preprints deserve to be pretty and properly typeset.

Ragnar {Groot Koerkamp}@curiouscoding· Mar 19

Cute new idea in here: Existing ABB+ (green) minimizers prefer kmers starting with 1 and then as many 0 as possible (in the binary case). "Spacers" (orange) instead prefer kmers starting with 10 that have maximum distance to the next occurrence of 10. (For DNA, A maps to 1 and CTG map to 0.)

Plot of minimizer density for DNA alphabet for w=12 and increasing k.
The new "spacers" perform well for k=5..7 in this case.

Ragnar {Groot Koerkamp}@curiouscoding· Mar 13

Lastly, regions that pass this suffix-filter with cost <=k are verified fully in a 2nd full-width stage that uses SIMD with fewer but larger lanes. The result: throughput that is independent of text length, 2-4x higher than v1 (with AVX2/512) for many patterns and long texts, up to 13x for 150bp.

Ragnar {Groot Koerkamp}@curiouscoding· Mar 13

In v2, we filp the SIMD tiling and bitpacking direction: Each lane represents a separate *pattern*, and the bits are adjacent *rows* of the matrix. Basically the original 'pack multiple patterns in a word' of Hyyrö 2005, but on an AVX-512 SIMD level. So now we can search 16 32bp patterns at once!

Ragnar {Groot Koerkamp}@curiouscoding· Mar 13

Just for reference, the tiling strategy in v1: the text is split into 4 chunks that are processed in parallel with 1 chunk per 64 bit SIMD lane. Each lane represents 64 adjacent *columns* of the DP matrix.

Fig. 1: The tiling strategy used by Sassy. The text is first split into word-size blocks of 64 bases. Then, the list of blocks is split into 4 chunks that are processed in parallel, with one SIMD lane per chunk. The text is implicitly padded as needed. Within a chunk, filling the matrix proceeds block-by-block. For each block, all (up to, see Section 2.3 and Figure 3) m = |P | rows are computed before proceeding to the next block. Each chunk is extended into the succeeding chunk as long as there is a sufficiently good “in progress” alignment (not shown).

Ragnar {Groot Koerkamp}@curiouscoding· Mar 13

That's on top of ~10x speedup over Edlib that we got in v1, (which also has a slightly updated preprint)! Specifically, v1 splits the text into 4 chunks and uses SIMD in the text direction, but this is inefficient when the text is only ~W=256 long, which v2 fixes. www.biorxiv.org/content/10.1...

A plot, with caption:

Fig. 5: Throughput of searching texts of varying length. We search a pattern of length m = 100 against texts with length varying from n = 150 to n = 128 000 bp, with k ∈{3, 20}. All points are computed by averaging over 1000 random texts and then converting to throughput. Note that this does not include searching the reverse-complement strand. While Sassy is consistently faster than Edlib, its relative advantage is smaller for shorter texts.

The plot shows that Sassy is consistently ~4-8x faster than Edlib, and that both are relatively slow for short sequences and converge only for sequences of length >8kbp or so.

Ragnar {Groot Koerkamp}@curiouscoding· Mar 6

Are people actually using A*PA? It suddenly got 4 stars in the last 2 days and appears to be growing steadily either way, but I'm getting issues maybe twice a year, which can either mean nobody uses it, or that it just works (but the UX is really not so great).

start-history.com graph of A*PA github stars, with bumps at time of preprint and publication, and steady growth since then.

Ragnar {Groot Koerkamp}@curiouscoding· Mar 2

Some notes on Nicola Prezza and group's work on Wheeler DFAs after finally properly understanding them at DSB. It's pretty cool how the LF-mapping of the FM-index translates nearly directly to Wheeler DFAs. curiouscoding.nl/posts/wheele...