I find it frankly weird that the same AI build can be a structural growth story for the accelerator, the network and the power plant, then become a brief commodity boom when we reach the memory. The machine doesn't make that distinction.
A neural network learns by changing the parameters it retains. Those weights encode learned structure; they aren't simply a filing cabinet of training examples. During execution, the model also needs somewhere to hold the information it is working on. Memory is fundamental to both what the model has learned and what it can do with it.
Associative memory was central to Hopfield's work decades ago: retrieve a relevant pattern from an incomplete cue. Modern Hopfield theory connects a particular associative-memory update directly to transformer attention. That doesn't make intelligence a lookup table. It does make treating memory as an incidental component a rather odd starting point.
At a trillion parameters, 16-bit weights alone occupy 2TB. Training also needs gradients, optimiser state and activations. Distributing these across chips, avoiding redundant copies and deciding what to recompute are central to making the training possible. The training plan is already a memory and communication plan before anyone orders the rack.
Sparse models make the distinction particularly clear. DeepSeek-V4-Pro has 1.6 trillion parameters, with about 49 billion active per token. The other experts haven't ceased to exist. Different tokens need different experts; the serving system still has to store them and make them accessible. Sparse computation doesn't mean the rest of the model evaporates.
DeepSeek's newer V4.1-Flash makes the connection unusually explicit. Alongside its 552bn backbone parameters sit 196bn parameters of conditional memory, called Engram, with lookups prefetched from host memory. Some things are cheaper to remember than to work out again. Its engineers also report substantial reductions in the context cache. Both developments matter: less waste, and a different way to use memory to make the model capable and affordable.
The transformer has another kind of memory in use: the context it is currently attending to. Retaining previously computed keys and values avoids recomputation. Using that state still consumes bandwidth. Capacity, bandwidth and placement determine how much context can be used, at what latency, and by how many requests together. FlashAttention's advance was partly about avoiding unnecessary movement between on-chip SRAM and HBM. Faster exact attention was also better memory engineering.
Agents extend this across time. An agent waiting for a tool can stop consuming execution time while its workspace and useful context persist. CPU utilisation can fall without the corresponding memory becoming free. Some context is worth keeping close; some can move to DRAM or flash; some is cheaper to reconstruct. The practical question is what must remain available, for how long, and how quickly it must be restored. Counting agents or tokens alone won't answer it.
There is a further distinction I think gets lost. The model's weights, its temporary context cache, and the agent's durable record of what worked are different things. Google's ReasoningBank extracts lessons from successful and failed attempts for later tasks to retrieve. An agent can give the next task a better starting point without retraining the model. Keeping everything would be wasteful. Keeping the right things lets useful work accumulate. That is a much more interesting source of demand than another chatbot remembering your coffee order.
Follow those requirements into the machine and the familiar component boundaries start to look inadequate. SRAM close to arithmetic. HBM around accelerators. CPU-attached DDR or LPDDR for working environments. Flash for context, checkpoints and durable records. Fabrics moving the right state between chips and racks. These tiers aren't interchangeable. Their arrangement determines which work the factory can finish economically.
Google's TPU 8i is a lovely concrete example: 33% more HBM and three times the SRAM of TPU 8t, despite about 20% less peak FP4 arithmetic. Different jobs, deliberately different machines. NVIDIA's shared context-memory tier makes another choice: retain reusable context beyond the GPU and bring it back when needed. You cannot derive either design from a FLOPS forecast alone.
This is also why I find the reflex that efficiency must kill demand so unconvincing. Quantisation, sharing and better caching really do save resources. They also change which jobs are worth commissioning. Coding already shows what happens when people can afford to delegate more ambitious work. Jevons isn't a magic formula; we still need to measure use. But holding the amount of useful work fixed while improving the technology that makes it affordable is a very strong assumption masquerading as prudence.
Micron's CTO made a useful distinction on the callback: different DRAM products increasingly require different wafer processes, even on the same technology node. A supply forecast expressed in undifferentiated bits can be as misleading as a demand forecast expressed only in FLOPS.
And the customers are doing rather more than expressing enthusiasm. In its September 30 results, Micron reported 26 take-or-pay strategic agreements, roughly $150bn of remaining obligations at minimum contracted volumes and prices, and some agreements extending into 2031. These don't fix every future selling price. They do put obligations behind a dependency. Customers also deposited $12.3bn during the quarter to secure future supply. Their forecast comes with money attached.
That is the connection I think finance still underweights. What a model has learned, what it is working on, and what the next agent should retain all have physical requirements. Customers are now committing years of money to meeting them. The duration is being negotiated by businesses whose own growth depends on delivery.
When I say there is no memory cycle, I mean the independent PC/phone boom-and-bust template being imposed on this system. Prices and margins still need analysing, product by product and contract by contract. A historical average doesn't do that work for us.
There is no AI, let alone superintelligence, without memory. An AI deployment plan is also a memory deployment plan. If your forecast gives the factory years of growth but sends its memory earnings off a cliff, show me the qualified supply, substitution, pricing or contract change that gets us there.
Don't just show me 2022.