In 2013 deep learning speech models halved the error rate but were expensive. Jeff Dean calculated that if users spoke to their phones for three minutes a day Google would need to double its fleet, so it built a chip for low-precision dense linear algebra that was 30 to 80 times more energy efficient.

Moving data from memory into the processor costs about a thousand times more energy than the computation itself. That is why training and serving use batching, to amortise the data movement, and why very low latency inference is hard. Dean wants inference hardware that minimises data movement and uses very low precision.

Dean suggests using models and harnesses to solve real problems, watching where they fail, and then writing better guidelines and skills that teach the model how to use tools for that class of problem. He and Sanjay wrote a skill for benchmark-driven performance optimisation.

Dean says agents degrade once a task drifts off the distribution of what the model was trained on. Skills and hints keep it on the well-lit path, and multi-agent systems where another agent evaluates competing approaches use inference-time search to make long-running flows more reliable.

Pick something you are excited about, then test general models on it. If they fail almost completely, 0% or 1% of the time, that is a good sign; if they succeed 20% of the time the capability is emerging and will likely improve. Private data or a niche specialised model can also give an edge.