Data Engineering Communities. ssp.sh/brain/data-e... Do you participate in any others that I should add?

Need a new book to read? Updated my book recommendations for 2026. www.ssp.sh/books/ Also, check out recommendations from 2018 onwards, if you haven't already. What are your thoughts so far?

And a visual log of more #todaysoffice setups. www.ssp.sh/brain/todays...

And hardware-wise, desk setup. www.ssp.sh/brain/comput...

How do you use your writing tool? Here's how I write articles in @obsidian.md: 1. current note 2. outline 3. related notes through smart connections 4. initial blog interview 5. connected note I have written 6. more related notes And you see word counts on the lower right and vim motions status.

I tried 'Kintsugi' (sort of) on my broken outdoor chair. It was originally made of some ropes that broke quickly in the sun. Now with lots of patience and almost meditative work, I fixed the chair with new colored clothesline (only the red 😉), and they are beautiful, and as comfy to sit on again.

Everything I know, in one search. ssp.sh/search

I've also added a full-text search page at ssp.sh/search for a more convenient search. Check it out.

Don't overrate how you do it; it matters more what you do, especially with LLM models. It's just a tool.

Automating my newsletter with the latest updates from my second brain, new essays, books I'm reading, notes, and latest social media post (pulled with DuckDB from Bluesky). No Gen-AI, all direct fetched from local MD files or Firehose via Python (linked), integrated with Listmonk and Neomd (email).

Also @cubedev.bsky.social sent me one, they are also pretty fast, but not as easily distinguishable with the colors (very similar) :)

Great article, for me, it's like Jason Fried said, it's about learning, not "producing" content.

I just bought a new toy on vacation - it's a glass fountain pen from Venice. Really enjoying the writing process. Each pen is a unique creation of Murano glass rods, worked by hand with a lamp by a local craftsman. A perfect fit for a distraction-free writing session 😍.

The article was on top of the HN front page. In case you missed it, a reflection post on creation and creativity.

Interesting, sqlglot is more often downloaded than duckdb 😮

How to use this guide: Each section contains curated resources (articles, videos, tutorials) for that topic. Click on the links that interest you most. It's meant as a guided roadmap to learn the fundamentals of a "full stack" data engineer. Here's the guide: www.ssp.sh/blog/data-e...

Learn Data Engineering this summer break with a guided roadmap. A three-week plan with links and topics to learn the basics to be a data engineer. It's very ambitious, but you can schedule at your own pace 👇.

Also, check `curl sub.ssp.sh/why ` to see why you should subscribe. Can we make it to 4000 (currently at 3853 - check with `/count`)?

You can now curl to subscribe to my newsletter. Try it out: ``` curl sub.ssp.sh/name@domain.... ``` I'd be happy to count you in. I don't send often, only if there's a new blog article, or I have something to share.

A great example of this is the Blast radius. Blast radius refers to the potential extent of damage. Check the applied-to-DE example in the linked article, which shows how to change a column and get a clear report of what will break and be affected by mapping the dependencies, rather than guessing.

This doesn't need to be a physical one, but a tooling like a compiler that calls a function to prove it against the parsed AST and the schema, just as a type checker proves a program compiles rather than guessing. Additionally, the compiler approach also helps improve token efficiency.

A probabilistic agent with the LLM that does the creative work of reading intent, picking a strategy, drafting SQL, summarizing results, and recovering when something goes wrong, and a layer that checks the output deterministically with a deterministic harness. Potentially a correction layer?

I found that there are essentially three main levels where agents can help the data engineer: 1️⃣ The initial chat-phase 2️⃣ The autonomous approach 3️⃣ Dedicated agents But is that enough?

I've asked myself where AI agents belong in data engineering work, specifically. Turns out there are 3 levels of "help" (chat, autonomous, dedicated tools)—but the real unlock is a 4th layer: a deterministic core underneath the LLM that proves correctness. Here's the breakdown.

Xorq, the tool featured, supports the Ibis grammar and its executable memory system for tabular data, and provides a horizontally scalable data architecture that works across our data stack. Supporting everything from discovery with a catalog to defining transformation logic to modeling.

Grammars compress complex systems into a few simple rules. As a Vim motions and declarative enthusiast, I wanted to explore whether we could apply the same structured approach to data pipelines and ensure we could deterministically reproduce and apply that exact definition.

Ever thought about a way to express, verify, and run your data pipelines regardless of the underlying compute engine? Imagine declaring your transformations agnostically, using a universal grammar for data engineering with distinct nouns and verbs.

I also have 656 public notes SB, totaling 320'879 words. And an additional 3'660'253 words in private notes (collected for 17 years). Plus a work-in-progress book with 157'086 words. Quite astonishing, how much I've written over the years. The first one in 2015, more than 11 years ago. ssp.sh

Time to celebrate, 100 blog posts written and more importantly, published. These are a total of 323'789 words. This is the amount of 1-2 full books. Sometimes I forget to celebrate the small wins, but it's important to keep going. I'm thankful for all you readers. To the next 100 articles. 🥂

I've added a new full-text search that searches both the second brain and blogs in one singular search. Plus, you can filter by dates, e.g., `This year` or `Last 2 years`, and toggle between Blog or Brain. Try it, hit `ctrl+k` (or cmd+k) and search ssp.sh.