Developer Tools
4 in the last 7 daysArticles
15Sandboxing Model-Generated Code: Limits on Resources, Network, and Filesystem Before You Execute Anything
If your agent can run code it wrote itself, the sandbox is the real security boundary. Here is what to constrain, and why a prompt instruction is not a control.
Temperature Zero Is Not Determinism: What You Can and Can't Reproduce in LLM Calls
Setting temperature to zero narrows output variation but doesn't guarantee identical responses. Here is where the remaining variation comes from and how to build around it.
Record and Replay LLM Calls: Offline Tests That Don't Burn Tokens or Flake
A recorded-response fixture layer lets most of your test suite run without a network, a bill, or nondeterminism. The trick is knowing what to record, how to key it, and when to throw recordings away.
Write Tool Errors for the Model: Error Messages an Agent Can Actually Recover From
A tool that returns 'Error: failed' leaves the agent guessing. A good error message names the problem, the cause, and the next valid move.
Golden-File Tests for Rendered Prompts: Catching Template Changes Before the Model Does
Your prompt is assembled from code, and the code can change it without anyone noticing. Snapshotting the rendered text makes those changes visible in review.
Tokenization Surprises: Why Character Counts, Whitespace, and Numbers Break Your Assumptions
String length is a poor proxy for tokens, and tokenizers change between models. Here is how to count correctly and avoid silent budget overruns.
Write a Prompt Linter: Cheap Static Checks Before Any Model Call
A handful of deterministic checks on placeholders, size, and format promises can catch real prompt defects in CI before they reach users.
Building a Human Review Queue for AI Outputs: Triage, Escalation, and Closing the Feedback Loop
A review queue that just lists outputs for a human to approve or reject is a bottleneck waiting to happen. The queues that hold up under volume are designed around triage, escalation, and turning every decision back into signal.
Shadow-Mode Deployments for LLM Features: Validating a New Pipeline Without Showing It to Anyone
Before a new prompt, model, or retrieval pipeline ever reaches a real user, it can run in parallel with production and get judged against real traffic. Here's how to build that harness without it turning into a second production system you have to maintain forever.
Tool Overload: When More Functions Make an Agent Worse, Not Better
Every tool you add competes for the same attention budget the model uses to pick the right one — past a certain roster size, that competition starts costing you accuracy.
The Hidden Cost of Retries: Why "Just Retry the LLM Call" Isn't Free
Retries add load, multiply across layers, and re-bill tokens. Here is a retry policy built on AWS guidance and adapted to LLM calls with side effects.
Batch APIs and Async Processing: When Not to Call the Model in Real Time
Much LLM work has no human waiting. Here is how batch APIs cut cost by half, what limits and outcomes to plan for, and how to audit your own call sites.
Designing the Handoff Contract Between Agents: What Actually Has to Transfer When Work Changes Hands
Splitting work across agents is easy to diagram and hard to make reliable — most of the reliability problems live in the handoff, not the split.
Mocking LLM Calls Without Lying to Your Test Suite
Unit tests that stub an LLM response usually end up testing the stub, not the system — here is how to keep them honest.
Multi-Agent Systems: When Splitting Work Across Agents Actually Helps
Splitting work across agents can win big, at several times the token cost. Here's what Anthropic's own multi-agent system shows about when it pays off.
Signal
No signals yet
Signals related to this topic will appear here as new intelligence is published.