A GPU (graphics processing unit) runs the parallel arithmetic that AI models depend on. TPUs (tensor processing units) and other ASICs (application-specific integrated circuits) are chips designed for the same work. They can be faster or cheaper for particular workloads, at the cost of a narrower software ecosystem and, often, tighter ties to one supplier.
For an enterprise, the economics depend less on the chip than on how it is bought and used:
- Through an API, the accelerator is inside the token price. The provider carries the utilisation risk, and the buyer pays per token.
- Reserved or committed capacity trades a lower rate for a commitment that is paid whether or not it is used. The buyer now carries the utilisation risk.
- Owned or privately hosted hardware adds power, facilities, staff and refresh cycles, and makes utilisation the deciding variable.
Three questions decide whether accelerator capacity pays:
- Utilisation. What share of paid hours does useful work? Idle capacity is the most common hidden cost in self-hosted AI.
- Fit. Does the workload need this class of chip, or would a smaller model on cheaper hardware clear the quality floor?
- Exit. How long is the commitment, and what does it cost to move if prices fall or the workload changes?
For suppliers, the same hardware is a factory whose output is tokens, and its economics are measured in tokens per watt. For buyers it is a cost to set against outcomes. See Rent, Reserve or Own Intelligence? for the capacity decision and the AI TCO Framework for where infrastructure sits in total cost.
