Lars Grammel
@lars
Working on software factories.
exactly the same problem with ai sdk prs imo (and most likely true for many open source projects) "Most of the work of reviewing is not simply catching bugs. A great deal of it is deciding whether this direction is a good approach, whether the PR is a good idea at all."
Put Sol to the test and created a desktop app for managing the AI SDK software factory today.
Use local docker sandboxes with AI SDK on Mac You can run tools against the sandbox, manipulate it directly, or use the AI SDK harness package to run e.g. Codex or Claude in the sandbox
Added step performance stats to AI SDK 7 canary: - step time - llm response time - time to first token - tool execution times - tokens per second (several variants)
i created a llama.cpp provider for vercel ai sdk that runs directly in the Node process (no separate server required, it uses llama.cpp bindings) the llama.cpp provider supports reasoning, tool calling, image inputs, prompt caching (for a single conversation)
At React Universe Conference Wroclaw today - would love to chat with AI SDK users. If you are around and want to give feedback or learn more about the AI SDK send me a DM.
AI SDK 5 prototype: custom UI data parts You can define custom ui data parts and stream updates to them from the server:
AI SDK 5 preview: ChatStore & ChatTransport ChatStore synchronizes chat write operations and manages streaming state. You can use it directly or through framework integrations like useChat. ChatTransport makes backend integrations more flexible, allowing for client-only usage.
AI SDK 5 preview: UI message persistance We recommend storing UI messages, not model messages, if your application has a useChat component. This ensures that the UI state can be correctly restored and helps integrating backends other than streamText (e.g. Langchain).
AI SDK 5 preview: UI message metadata UI messages in AI SDK 5 will have a generic metadata property (instead of specific properties like createdAt). This lets you send and show the message metadata that's important in your application:
UI messages != model messages What you display to the user (ui messages) is different from what you want to send to the LLM (model messages). UI messages contain additional information such as custom app data and metadata. Tools calls might be omitted. etc.
In AI SDK 5, UI messages will be composed of parts. This is my current vision of how UI messages could look like - feedback welcome.
Combining the provider registry with custom providers and middleware is extremely powerful for setting up your models in one place:
The AI SDK now supports LLMs that generate file outputs, first example are images:
The granular useChat status messages make it easy to show a loading spinner (when status=submitted):
AI SDK 4.0.6 🧪 structured outputs with tool calling using generateText (experimental) You can create structured outputs with generateText using the experimental_output setting. The object is available in the experimental_output result property.
Idea code: adding support for structured outputs to generateText Would love feedback!
🧩AI SDK provider: TogetherAI Using TogetherAI models has become even easier with the new provider for the AI SDK: