Leon Derczynski
@leonderczynski
LLMs & Security at NVIDIA Prof in CS/NLP at IT University of Copenhagen garak guy, garak.ai "berømt skikkelse" "like a gazelle" Copenhagen/Seattle
Huge success and part of how I chose who to approach when moving to industry. Open source is the way. Looking at "repositories with meaningful traction", AI world say "NVIDIA is now the largest contributor, with more than 600 repositories in the past year" 🎉 🧙 💚 aiworld.eu/story/the-ne...
Anonymised analysis of the openai model 'breaching' hugging face: > report doesn't say what sandbox sol broke out of?? > a docker container running as root > Plot twist there was no sandbox at all > many use "sandbox" and "container with host access" interchangeably ymmv, use critical thinking
generally, everyone outside of the upper ~quarter of incomes is much better off under heavy economic redistribution (i.e. left-wing econonmic policy) this is uncontroversial unfortunately the mathematical literacy to see this is not universal fortunately money isn't the most important factor
Solid US-origin open model, congratulations Thinking Machines * context window of 1M * available on hugging face now * between opus 4.6 and gpt 5.6 on a web dev benchmark * token efficient * 41B active params of 975B total www.wired.com/story/thinki...
The whole digital economy runs on open source. Banning open models is bad for almost everything if it lands - business and research and consumers. And they present no risk. What's going on? https://thehill.com/policy/technology/5952253-trump-administration-ai-restrictions-opens/
@jonchristian.net love seeing your stories pumped straight to my lock screen notifications
Come to LLMSEC at ACL & hear Niloofar's keynote "What does it mean for agentic AI to preserve privacy?" - Niloofar Mireshghallah, Meta/CMU (Friday 1st Aug, 11.00; Austria Center Vienna Hall B) See you there! #acl2025 #acl2025nlp
Here's my "Most Inappropriate Demo" trophy at NVIDIA, 2024. For garak's "atkgen.Tox" probe, an unfettered LLM used to goad other LLMs into being toxic.
you know the field has changed when the foreign event you were speaking at is on the tv news on the bus home
Will be representing NVIDIA at the EU AI Summit in Paris. I'll be talking about how we build & help others build safe, secure AI systems. On 11.2 you can see me at: * AI Assurance and Testing: Global Perspectives * Building trustworthy AI: balancing innovation, responsibility, and democratization
Are there people who don't make the sponge cake rice cooker recipe asap?!
Good Christmas times, finally the elephant has come to our house!
see you at #chr2024! (i swore off hashtags but i guess sometimes it's hard to stay on the wagon)
Did not expect Russian Ryanair to get this excited about Black Friday
i suppose the fact that my reaction to this at 7.something a.m. is one of mild panic and dejection is a clue that i should not work in product or sales
and the tests work in practice. ANSI terminal control sequence attack success rates on <a recent model> - way into the double digits!
it turns out LLMs can output the control codes needed for ANSI control codes to run - so the entirety of those command sets is available usable through LLM output. why are these things in the tokeniser lol
STÖK did a cool talk (www.youtube.com/watch?v=3T2A...) showing how viewing e.g. a log entry can cause code to be run on your computer without your intervention, great for hacking. all of OSC8 + OSC52 is available without user intervention. but wait, there's more!
just found a hilarious llm vuln. you know those ANSI escape codes for colouring your terminal? they can also move the cursor, hide text, execute commands, etc. through just viewing a piece of text
> trying to persuade my partner to get onto bsky after they already left twitter > they can't find a pfp they want > "just take a selfie now" > "in the bathroom? no. never. bathroom selfies are utter trash. the worst" > "lol 🫠"
Here's the French minister of AI, the convening Columbia prof, and the president of Mozilla, standing on an office table in a bar on a swaying boat
Back in snowy Denmark. I always forget how good the showers tend to be here - in heat, control, water quantity, and bathroom layout.