Agent Know-How

Google Cloud made token-based cost management generally available for BigQuery's LLM functions (AI.GENERATE, AI.CLASSIFY, AI.IF, AI.SCORE, AI.AGG). Four daily quotas run through IAM: input and output tokens, each per project and per user.

• Defaults: 200B input and 20B output tokens per project; 150B and 15B per user.
• Quotas are global across regions, and output quotas count thinking tokens.
• Lowering a quota applies immediately; raising one is a request, often auto-approved.
• Embedding functions are exempt.

For builders: the cap sits at the platform layer, so one runaway query cannot spend past it.

Google Cloud became the third major vendor to ship FinOps for agents, and goes further on flexibility:

• Billing: a pay-as-you-go Gemini Enterprise tier with no upfront commitment beside per-seat plans; Flexible Savings Plans give 10% (one year) or 20% (three years) off with no minimum commitment.
• Anomaly detection names a spike's root cause and its top three SKUs.
• Project-level spend caps pause API calls at a hard monthly limit, with alerts at 50/80/100%.
• Deferred execution (coming) runs eligible agent workloads in off-peak windows for up to 50% off inference.

Anomaly root-causing and off-peak scheduling are two levers the OpenAI and Anthropic consoles do not offer yet.

Claude Code v2.1.239 folds the 1.1x US-only-inference premium that data-residency workspaces pay into /cost, the status line, and --max-budget-usd.

Before the fix, a team on a residency-locked workspace budgeted against the base rate and paid 10% more. A budget cap is only as good as the estimate it compares against, so the estimate needs the same scrutiny as the cap.

Agentsview, an open-source tool, lets a developer browse, search, and track costs across all of their AI coding agents in one place.

Each vendor's dashboard shows only its own agent. A developer or team running several agents needs a cross-agent view to see total spend, and third-party tooling is filling that gap.

AWS released FinOps Agent in public preview. It investigates cost anomalies, correlates spend changes with AWS activity data, and routes findings to resource owners through Slack and Jira.

Anomaly triage becomes continuous and queryable instead of a manual monthly dig through the bill. Cost analysis is itself becoming an agentic product, though it still depends on the tagging and attribution a team has in place.

Prtokens, an open-source tool, attributes agent token cost to an individual pull request, so cost becomes a number on the artifact rather than an aggregate on the monthly bill.

Per-unit-of-work attribution is what lets a team ask whether a given change was worth what the agent spent on it, which is the starting point for any cost-per-task or ROI argument.

Simon Willison argues that pay-by-usage services and APIs need default hard budget caps: "after $X/month, cut this thing off," not an alert that arrives after the bill. Agents can burn metered usage faster than a human notices.

A Pi-based coding agent shown on Hacker News ships the same limit inside the agent itself, with hard budget caps as a product feature.

Both treat the cap as the backstop under visibility and alerts. The open design question is the default: off until configured, or on until raised.

TokenOps wraps every model call and checks a shared run budget before execution. It sits in the execution path, not beside it like a tracing dashboard. Multiple agent processes read and write one SQLite ledger, so a budget can span a distributed multi-agent workflow.

Its ten policies go beyond a hard stop: throttle, log, mutate the prompt, or inject cost-reduction instructions. The project reports cutting wasted agent spend by up to 65%, but publishes no methodology or independent benchmark behind that figure.

Mole is an open-source deep-research agent for the terminal. Its author built it because research agents "blow way past budget, jumble the sources," and still sound confident. Mole never spends more than its budget (measured overshoot is 0%), attaches a source to every claim, and keeps local data on the machine.

It shows runaway spend is still the default without an enforced ceiling, told from the tool builder's side. The fix is a cap the agent cannot exceed, not an alert after the run.

Claude Code v2.1.237 fixed prompt caching for sessions using an LLM gateway or custom base URL. Before the fix, caching could stop firing on those sessions without a visible signal.

A caching default is infrastructure with its own failure mode. Teams routing through a gateway should monitor cache-hit rate, not assume it once the feature is on.

A Google Dataflow pattern combines Apache Beam streaming with the Agent Development Kit. The pipeline escalates an event to a full gen-AI agent, with database lookups and email tools, only when the event needs that judgment, such as an angry customer message. Everything else stays on the static path.

It controls cost at the entry point rather than inside the agent loop: the cheapest model call is the one a pre-filter decides not to make.

LangChain's Deep Agents uses provider prompt caching to cut LLM token costs by up to 80% across every major provider, with no extra config.

An agent loop re-sends a large, stable prefix every turn: system prompt, tool schemas, prior steps. That is exactly the input a provider prompt cache discounts. Caching the stable prefix becomes a default the framework owns, not a knob each team has to discover.