# Systems and Compiler Depth The engineering skill of making a given amount of hardware do more work. It runs through the whole stack the article lists: kernels, optimisers, serving, and chips. ## The layers - **Kernels.** Custom GPU functions tuned to a specific model and chip. See [[Below CUDA - GPU Kernels, PTX and Streaming Multiprocessors]]. - **Compilers.** Software that turns model code into efficient hardware instructions, fusing operations and planning memory. Examples include Triton, XLA and TVM. - **Optimisers.** The algorithm that updates weights during training (Adam, AdamW, newer variants like Muon, which Moonshot used for Kimi K2). A better optimiser reaches the same quality in fewer steps, which directly saves compute. - **Numerical precision.** Training or serving in 8-bit or lower instead of 16-bit roughly halves memory and bandwidth. - **Serving systems.** The software that runs a model for users: batching requests, managing the KV cache, speculative decoding, splitting prefill and decode across machines. This determines cost per token ([[Token Economics of Inference]], [[AI Inference Infrastructure]]). - **Chips.** Designing silicon, or porting the stack to domestic accelerators, is the deepest layer. ## Why it compounds Compute is consumed: a GPU-hour spent is gone. A systems improvement is kept: a faster kernel or better optimiser lowers the cost of every future run. Labs that cannot buy more compute are forced to invest in the second kind, and over several years that produces a lasting efficiency advantage. > [!important] Investment lens > Capex buys a position. Systems depth buys a rate of change. When assessing a lab or an infrastructure company, look at how much of its advantage survives if its hardware budget is cut. Related: [[AI Chip Export Controls]], [[Nvidia H800]], [[Model Compression & Edge AI MOC]], [[The Cross-Pacific AI Chessboard]]