Inference, Diffusion, World Models, and More | YC Paper Club

The inaugural YC Paper Club session opens at Pioneer, the Woodside campus where Sam Altman, Andrej Karpathy, and Greg Brockman once sat alongside early-stage AI founders brainstorming what would become OpenAI. The organizer frames two missions: create a community of top founders and researchers, and revitalize a campus that has sat underutilized despite its legacy. A show-of-hands poll reveals the room's caliber: multiple attendees have ten-thousand-plus citations; several have raised $50 million or more.

The hidden geographic thesis is that half the Bay Area's AI talent lives in San Francisco (Anthropic, OpenAI, Cursor), but the other half — Google DeepMind, Tesla, xAI, Thinking Machines — works in Palo Alto and rarely makes the trek north. Pioneer sits at the nexus, and the club aims to pull both hemispheres together. Five papers anchor the session, spanning speculative decoding, diffusion-based control, world models, generalization theory, and data-constrained scaling.

Model predictive control (MPC) factors agent design into an action-proposal module and a dynamics model (world model). Diffusion MPC (DMPC) applies diffusion models to both, reducing compounding error and simplifying planning. The speaker demonstrates on locomotion tasks: a quadruped trained only on «run forward» and «jump» can exhibit novel gaits at test time by swapping in a new reward function. When dynamics shift — for example, a walker with a broken left ankle — the factorized design allows re-training only the dynamics model on a handful of samples, preserving the action prior and recovering performance.

DMPC outperforms prior MPC and model-free baselines on 2D tasks (Push-T) and remains competitive on 3D (Push Cube), though DINO World Model wins in 3D thanks to its large foundational vision backbone. Crucially, DMPC runs 50× faster than alternatives because all work happens in a learned latent space, fitting on a single GPU with under 24 GB VRAM and only 15 million parameters. The work predates the current wave of «robot foundation models» but shares the same core bet: factored, multi-step models generalize better and adapt faster than end-to-end policies.

The current explanation for why scaling works is that «it just does» — a dissatisfying answer when generalization underpins every capability gain in modern AI. Andrew Gordon Wilson's paper argues that classical PAC-Bayes bounds, long dismissed as vacuous for overparameterized models, actually explain deep learning when applied correctly. PAC-Bayes bounds test loss with training loss plus a compression term. Historically, the compression term dominated and bounds became loose. Wilson shows that larger models find more compressible solutions: the volume of flat minima in parameter space grows exponentially with parameter count, and flat minima compress better than sharp ones.

The paper also resolves «benign overfitting» — the mystery of how networks fit random noise yet generalize on structured data. A regularized polynomial offers intuition: enough parameters fit noise, but regularization biases the model toward low-order terms that capture structure. Deep nets are expressive hypothesis spaces with soft inductive biases. The takeaway: if we identify the right inductive biases and optimize for them (e.g., compressibility, flatness), we may unlock massive sample-efficiency gains. The no-free-lunch theorem guarantees that all learning efficiency comes from inductive bias, and humans remain orders of magnitude more sample-efficient than models.