"Just use the frontier model" is a lazy default, not a quality decision
Inference costs and energy consumption are the same variable measured in different units. Every token you don't generate is compute a GPU doesn't draw, cooling a data center doesn't run, and carbon a grid doesn't emit. The sloppiest pattern in the industry, defaulting to the largest available model for everything, wastes both.
The reasoning behind that default is understandable: the big model is the best model, quality matters, so use the big model everywhere. But most production AI work is not frontier work. Classifying, extracting, reformatting, handling short conversational exchanges: none of it needs the largest model any more than hanging a picture needs a sledgehammer. Frontier models cost more per token and burn more compute per token for capability the task never exercises. The default isn't a quality bar. It's an unexamined habit with a smokestack attached.
What routing looks like in practice
TailorFit runs a deliberately boring version of this. Our assistant Tai handles conversation, logging, and extraction on a small, fast model, because "figure out which application the user means and log a touchpoint" is a small-model task. The small model is also simply better here: faster responses make a chat assistant feel alive. The heavyweight model is reserved for work that earns it, like tailoring a full resume against a job description, where output quality is the product. One product, two models, each task on the cheapest model that meets its quality bar. That last clause is the whole methodology, and it generalizes to any stack.
The discipline this requires is measurement, not cleverness. We log token counts and prompt versions on every AI call, so when we consider moving a task down a tier, we test it against real traffic. Teams that don't measure route by superstition, and superstition always routes up.
The rest of the toolbox
Routing is the biggest lever, but the same cost-equals-energy logic runs through the whole inference path.
Prompt caching means the static preamble of your prompts, including instructions, schemas, and reference content, is processed once and reused instead of recomputed on every call. For chat products that resend context each turn, this eliminates a large fraction of total compute.
Lean prompts matter for the same reason. Every boilerplate paragraph in your system prompt is paid for on every request, forever. Prompt bloat is a standing energy tax.
Response caching, output length caps, and batching round out the list. Cache whole responses where inputs repeat. Cap output lengths to what your UI actually displays. Batch offline work. None of this is exotic. It's the ordinary engineering discipline of not computing things twice, applied to the most compute-heavy dependency most products have ever had.
Why the energy framing deserves the meeting
A cost argument gets efficiency prioritized when margins pinch and deprioritized the moment growth covers the bill. The energy reality doesn't fluctuate with your pricing. Aggregate AI inference demand is now large enough to shape data center buildouts, grid planning, and water draw in real places. Every product team's routing decisions are a small vote in that total.
Small companies can't build clean grids. You can refuse to burn compute on capability your tasks don't use. And unlike most climate-adjacent decisions, this one pays you to make it. Route deliberately, cache aggressively, measure everything. Your bill and your footprint are the same graph, and both should slope down.