On-Prem Inference

Validating, because the position has been tested against real numbers three times and held each time. The pressure came from two clients — a bank and a state government agency — who wanted to know whether to buy a GPU cluster. Our model puts the cost crossover for a 70B-class open-weight model at roughly 55–65% sustained utilisation on H100-class hardware at AU power and colocation prices, and almost no enterprise workload we have measured runs above 40%. Below the crossover, an AU-region hosted service with contractual data residency is cheaper and carries the same sovereignty story for every regulator we have checked. The exceptions are real: air-gapped defence and some health workloads, and any client with a GPU allocation they already own. Our own 18-month payback claim for an H100 cluster is in the graveyard. GPU supply eased in 2026 and hosted prices fell about 40% in twelve months, which moved the crossover further from most clients, not closer.

This is the most expensive single decision a client can make in AI infrastructure and the one where vendor pressure is strongest. A bank that buys a $12M cluster at 30% utilisation has bought sovereignty at three times the hosted price and will run it for five years to justify it. Quantium gets asked directly and answers with a standing answer that has to be right. It also sets the floor for TEE and Secure Inference: if hosted AU-region inference is the default, confidential computing is what makes it acceptable for regulated data, and the two fields move together.

Owned inference hardware pays back only above a sustained utilisation floor that most enterprises never reach; below it, sovereignty and residency are better bought as a hosted AU-region service than as a GPU cluster, and the crossover point moves against on-prem every time a hosted price drops. That is the lab's one-line position on it, which is not the same as an explanation.

The shape the field is converging on, from the most authoritative source in it: AU regulators accept contractual residency for the data classes clients run inference on.signal

This is the section a page most needs a person for, and the one composition is worst at. Nobody has written the plain-language version — what the idea is, in words that assume nothing — and it is the first thing a reader who has never met the term needs.

The lab opened this field on 2025-11-04, and it has not reached mainstream awareness yet. Everything below is what has moved since.

What shipped: Two AU-region hosted inference providers cut 70B-class prices for the third time in a year (Provider pricing pages, 2026-08-04) and Two frontier labs announce AU-region inference with contractual residency (Lab announcements, 2026-06-24). Tooling arriving is what moves a field from argument to something a team could try.signalsignal

And the rule moved: OAIC and DTA guidance: contractual AU-region residency satisfies APP 8 and hosting policy for PROTECTED and below In a regulated vertical that usually decides the timing more than the technology does.signal

Demand is rising on it rather than steady — “Can this run entirely on-shore, and can you prove it?” — which is the difference between a field worth watching and one worth doing something about.demand

What changes, concretely: A 70B open-weight model on an 8×H100 node at AU colocation and power prices costs more per million tokens than the cheapest AU-region hosted equivalent until sustained utilisation passes about 60% (validation run 4, re-baselined).signal

There is already a shipped default — inference.host = hosted-au-region · exceptions: owned-allocation | air-gapped | util>60% · standing answer: sa-onprem-when — so a team adopting this is changing a setting rather than starting a project.

The binding constraint is economics: it works and does not yet pay. Everything upstream of that is solved and everything downstream of it is waiting.

Workforce readiness is high: The position is a spreadsheet and a set of exceptions; any sector owner can deliver it. Agent-estimated. A recommendation needing skills the firm does not hold is an aspiration rather than an action, and it routes to the enablement agenda instead of the delivery one.

Something here has already been killed: On-prem H100 cluster pays back inside 18 months, on two au-region hosted price cuts in q1 and q2 moved the breakeven to over four years before we finished the model; the standing answer now carries the calculation and its refresh date.graveyard

The argued case against it is the red team's, further down this page, and it is deliberately one-sided — this section is what stands in the way mechanically, not what somebody thinks of it.