> *The most underrated frontier in AI isn't another chatbot or image generator. It's AI that discovers mathematics humans haven't figured out yet.* - [[#The Breakthrough That Changed Everything|The Breakthrough That Changed Everything]] - [[#The Breakthrough That Changed Everything#Why One Multiplication Matters|Why One Multiplication Matters]] - [[#The Sparse Data Problem|The Sparse Data Problem]] - [[#The Sparse Data Problem#The Navier Stokes Example|The Navier Stokes Example]] - [[#Physics Informed Neural Networks: Encoding Physical Laws|Physics Informed Neural Networks: Encoding Physical Laws]] - [[#Physics Informed Neural Networks: Encoding Physical Laws#How PINNs Work|How PINNs Work]] - [[#Physics Informed Neural Networks: Encoding Physical Laws#Advantages in Sparse Data Regimes|Advantages in Sparse Data Regimes]] - [[#Physics Informed Neural Networks: Encoding Physical Laws#Current Limitations|Current Limitations]] - [[#Tensor Networks: Structure from Quantum Mechanics|Tensor Networks: Structure from Quantum Mechanics]] - [[#Tensor Networks: Structure from Quantum Mechanics#Why Tensor Networks Matter for Machine Learning|Why Tensor Networks Matter for Machine Learning]] - [[#Tensor Networks: Structure from Quantum Mechanics#Matrix Product States and Beyond|Matrix Product States and Beyond]] - [[#The Quantum Bridge: Hybrid Quantum Machine Learning|The Quantum Bridge: Hybrid Quantum Machine Learning]] - [[#The Quantum Bridge: Hybrid Quantum Machine Learning#Variational Quantum Circuits|Variational Quantum Circuits]] - [[#The Quantum Bridge: Hybrid Quantum Machine Learning#Hybrid Quantum PINNs for CFD|Hybrid Quantum PINNs for CFD]] - [[#The Quantum Bridge: Hybrid Quantum Machine Learning#Why Quantum for PDEs?|Why Quantum for PDEs?]] - [[#The Verification Bottleneck|The Verification Bottleneck]] - [[#The Verification Bottleneck#The Inevitability of Hallucination|The Inevitability of Hallucination]] - [[#The Verification Bottleneck#The Formal Verification Solution|The Formal Verification Solution]] - [[#The Synthesis: Scientific Discovery Engines|The Synthesis: Scientific Discovery Engines]] - [[#The Synthesis: Scientific Discovery Engines#Near Term Applications|Near Term Applications]] - [[#The Synthesis: Scientific Discovery Engines#Longer Term Horizons|Longer Term Horizons]] - [[#The Race Worth Watching|The Race Worth Watching]] - [[#Key References|Key References]] - [[#Key References#AI for Mathematical Discovery|AI for Mathematical Discovery]] - [[#Key References#Physics Informed Neural Networks|Physics Informed Neural Networks]] - [[#Key References#Hybrid Quantum PINNs|Hybrid Quantum PINNs]] - [[#Key References#Tensor Networks for ML|Tensor Networks for ML]] - [[#Key References#Hallucination and Verification|Hallucination and Verification]] --- ## The Breakthrough That Changed Everything In May 2025, Google DeepMind's AlphaEvolve accomplished something extraordinary: it discovered a new algorithm for multiplying 4×4 complex valued matrices using only 48 scalar multiplications, breaking a record that had stood for 56 years since Volker Strassen's algorithm in 1969. The AI didn't just optimize existing code. It found a fundamentally more efficient mathematical structure that generations of brilliant computer scientists had missed. What's remarkable is that AlphaEvolve used complex numbers in a counterintuitive way, creating "magical cancellations" that no human thought to try. PhD dissertations were written attempting to improve on Strassen's method. Academic careers were built on the pursuit. Yet it took an AI system to finally break through. Ref: [[Future of Foundational Models#What are the computational challenges?]] ### Why One Multiplication Matters While reducing 49 to 48 might seem trivial, the implications compound dramatically at scale. When you recursively apply this to larger matrices, instead of 49×49 operations for an 8×8 matrix, you now need 48×48. The gap grows exponentially. In a world where AI systems perform billions of matrix multiplications every second during training and inference, this translates to massive savings in energy and computational costs. AlphaEvolve has since been deployed across Google's computing ecosystem: - A scheduling heuristic for Borg (Google's cluster management) that continuously recovers 0.7% of worldwide compute resources - A 23% speedup in matrix multiplication kernels used for Gemini training, leading to 1% reduction in overall training time - Optimizations to Tensor Processing Unit (TPU) arithmetic circuits that will ship in upcoming hardware [[Future of Foundational Models#New Hardware Innovators]] ## The Sparse Data Problem The natural question: why not aim this capability at problems with massive economic stakes? Fluid dynamics, protein folding, climate modeling, battery chemistry optimization. Every field with equations we can't solve analytically is a field waiting for this. But here's the challenge: these domains are often sparse data environments. You can't just throw more training data at turbulent flow or quantum chemistry. The experiments are expensive. The parameter spaces are enormous. And the phenomena being modeled often exist in regimes where empirical data is fundamentally limited. ### The Navier Stokes Example Consider the Navier Stokes equations, the fundamental partial differential equations governing fluid dynamics. They describe the physics of weather, ocean currents, water flow in pipes, air flow around wings, blood flow, power station design, and pollution dispersal. The problem: **no analytical solutions exist to the Navier Stokes equations in their most general form.** This is so fundamental that the Clay Mathematics Institute has designated it as one of the seven Millennium Problems, offering $1 million for a proof of existence and smoothness of solutions. In practice, this means: - Engineers must rely on computational fluid dynamics (CFD) simulations - These require expensive hardware, substantial computational time, and manual effort for mesh definition - Any change in geometric parameters or boundary conditions requires complete re simulation - Direct numerical simulation (DNS) of turbulent flows requires mesh resolutions so fine that computational costs exceed even today's supercomputers for most industrial Reynolds numbers The economic implications are staggering. Every aerospace company, every automotive manufacturer, every chemical plant, every pharmaceutical company with flow processes spends millions on simulations because Navier Stokes has no general analytical solution. [[interesting AI areas]] ## Physics Informed Neural Networks: Encoding Physical Laws This is where physics informed neural networks (PINNs) become critical. PINNs are neural networks that embed knowledge of physical laws, described by partial differential equations, directly into the learning process. The key insight: by encoding governing equations into the loss function, you can train models that respect physical laws even with limited data. The network doesn't just fit curves. It learns solutions that satisfy conservation of mass, momentum, and energy by construction. [[Physics Informed Neural Operators]] ### How PINNs Work Traditional neural networks rely on large datasets of input output pairs. PINNs take a fundamentally different approach: 1. **Physics as regularization**: The prior knowledge of physical laws acts as a regularization agent, limiting the space of admissible solutions and increasing generalizability even with minimal training data. 2. **Unsupervised learning**: PINNs can be trained without labeled simulation data, using only the PDE residuals and boundary conditions as the loss function. 3. **Continuous representation**: Unlike mesh based methods that discretize space, PINNs learn continuous solutions across the entire domain. 4. **Inverse problems**: PINNs can estimate unknown model parameters (like material properties or unknown coefficients) from sparse observations. ### Advantages in Sparse Data Regimes The PINN framework offers several critical advantages: - **Reduced data dependency**: Physical laws provide a priori knowledge that guides learning, enabling accurate solutions with far less training data than purely data driven approaches - **Robustness to noise**: The physics constraints help filter spurious signals in noisy measurements - **Generalization**: Solutions are guaranteed to respect conservation laws even in regions without data - **Mesh free computation**: No need for expensive mesh generation or re meshing when geometry changes ### Current Limitations PINNs are not without challenges: - **Strong nonlinearities**: PINNs struggle with PDEs that have sharp gradients or discontinuities (like shocks in compressible flow) - **High frequency features**: Spectral bias causes slower convergence for high frequency components - **Curse of dimensionality**: While better than traditional methods, very high dimensional PDEs remain challenging - **Multi scale coupling**: Problems with vastly different length or time scales require careful architecture design [[How AI Is Rethinking the Way We Reason#Key Points]] ## Tensor Networks: Structure from Quantum Mechanics Tensor networks originated in quantum many body physics as a way to efficiently represent high dimensional quantum states. The core insight: a large tensor that would be computationally intractable can often be decomposed into a network of smaller, connected tensors with polynomial complexity. ### Why Tensor Networks Matter for Machine Learning 1. **Interpretability**: Unlike black box neural networks, tensor network structure has solid theoretical foundations from quantum information theory. 2. **Efficiency**: By exploiting low rank structure, tensor networks can compress models dramatically while maintaining precision. 3. **Natural interface with quantum**: Tensor networks are "quantum inspired" algorithms that can run on classical hardware but map naturally to quantum circuits. 4. **Expressiveness control**: The bond dimension (connectivity between tensors) provides a tunable knob for model expressiveness, helping avoid both underfitting and overfitting. ### Matrix Product States and Beyond The most common tensor network structure is the Matrix Product State (MPS), also known as tensor train decomposition. For a system with N sites: - Classical full tensor: exponential storage (d^N parameters) - MPS representation: polynomial storage (N × d × χ² parameters, where χ is bond dimension) More sophisticated structures include: - **PEPS** (Projected Entangled Pair States): extends to 2D systems - **TTN** (Tree Tensor Networks): hierarchical structure with logarithmic depth - **MERA** (Multi scale Entanglement Renormalization Ansatz): captures multi scale correlations --- ## The Quantum Bridge: Hybrid Quantum Machine Learning When you hybridize physics informed neural networks with quantum circuits, you unlock additional expressibility that may be essential for certain problem classes. ### Variational Quantum Circuits Variational quantum algorithms (VQAs) use parameterized quantum circuits whose parameters are optimized by classical routines. The key advantages: 1. **Exponential state space**: A quantum system of n qubits can represent 2^n complex amplitudes simultaneously through superposition and entanglement. 2. **Natural function classes**: Certain function classes can be represented exponentially more efficiently on quantum hardware than on classical computers. 3. **Quantum feature maps**: Data can be encoded into quantum states in ways that create exponentially large feature spaces, potentially enabling kernel methods impossible classically. ### Hybrid Quantum PINNs for CFD Our work on hybrid quantum physics informed neural networks for computational fluid dynamics demonstrated concrete advantages: - **21% higher accuracy** compared to purely classical neural networks in complex 3D geometries - The quantum layers provide additional expressibility for capturing nonlinear dynamics of fluid flow - Transfer learning across different geometric configurations becomes more efficient The hybrid approach works by: 1. Encoding spatial coordinates through a classical preprocessing network 2. Passing features through parameterized quantum circuits that add expressibility 3. Combining quantum measurements with classical post processing 4. Optimizing the combined loss function including PDE residuals ### Why Quantum for PDEs? For high dimensional PDEs, the kind that govern real world physics, quantum approaches may offer fundamental advantages: - **Curse of dimensionality**: Classical methods scale exponentially with dimension; quantum methods can sometimes maintain polynomial scaling - **Entanglement structure**: Many physical systems have entanglement patterns that tensor networks and quantum circuits naturally capture - **Efficient gradients**: Quantum systems allow certain gradient computations that would be expensive classically Current challenges include: - **Noise**: NISQ (Noisy Intermediate Scale Quantum) devices introduce errors - **Barren plateaus**: Random initialization often leads to vanishing gradients in variational circuits - **Circuit depth**: Quantum coherence limits how deep circuits can be before noise dominates --- ## The Verification Bottleneck The most significant barrier to AI driven mathematical discovery isn't compute. It's that current foundation models hallucinate when you need rigor, and cannot reliably distinguish a valid proof from a plausible sounding one. ### The Inevitability of Hallucination Recent theoretical work has established that hallucination in large language models is not a bug to be engineered away, but a fundamental structural property: - **Gödel's incompleteness**: Any sufficiently powerful formal system contains true statements that cannot be proven within that system. LLMs, as complex formal systems, inherit this limitation. - **Computational undecidability**: The halting problem and related undecidable problems arise in LLM training and operation. - **Learning theory bounds**: LLMs cannot learn all computable functions and will therefore inevitably hallucinate if used as general problem solvers. Studies show LLMs hallucinate between 2.5% and 15% of the time depending on the model and domain. In specialized domains like law, hallucination rates can exceed 80% for specific query types. ### The Formal Verification Solution The path forward is formal mathematical reasoning with automated verification. This is exactly what DeepMind's AlphaProof achieves: - Operates entirely in the formal Lean language - Proofs are automatically verifiable by the proof assistant - Eliminates the possibility of hallucinated reasoning steps - Achieved silver medal performance at IMO 2024, solving P6 (the hardest problem, solved by only 5 of 609 human participants) AlphaProof combines: - **Neural theorem proving**: Deep learning models suggest proof tactics at each step - **Symbolic proof search**: Best first and MCTS algorithms assemble candidate proofs - **Reinforcement learning**: Errors detected by the formal system are incorporated into training - **Autoformalization**: Translation of informal mathematical statements into formal representations [[AI Verification - The Real Bottleneck in Enterprise AI Adoption]] ## The Synthesis: Scientific Discovery Engines The convergence of these technologies points toward something transformative: general purpose scientific discovery engines that can: 1. **Operate in sparse data regimes** through physics informed architectures that encode known laws 2. **Leverage quantum expressibility** for high dimensional PDEs and many body systems 3. **Guarantee correctness** through formal verification that eliminates hallucination 4. **Discover novel mathematics** by exploring spaces humans cannot search exhaustively ### Near Term Applications Already tractable: - **Shape optimization**: CFD simulations for industrial mixer design, aerodynamics - **Materials discovery**: Property prediction with physics constraints - **Drug design**: Molecular dynamics with quantum chemistry priors - **Climate modeling**: Physics informed emulators for faster simulation [[Themes shaping 2025#6.2 Software-Powered Drug Discovery: A Golden Age]] ### Longer Term Horizons With continued progress: - **Analytical approximations**: Deriving closed form solutions for PDEs that currently require simulation - **Theorem discovery**: AI suggesting conjectures that human mathematicians prove - **Algorithm design**: Finding fundamentally new computational approaches (as AlphaEvolve demonstrated) [[Themes shaping 2025#6.2 Software-Powered Drug Discovery: A Golden Age]] ## The Race Worth Watching Whoever cracks **formal reasoning at scale with verification**, while leveraging physics informed architectures for sparse data regimes, unlocks something closer to a general purpose scientific discovery engine than anything we've built so far. > The economic implications are measured in trillions. Every field with equations we can't solve analytically, every simulation that takes days instead of seconds, every design iteration that requires expensive physical prototypes, these are all opportunities waiting for AI systems that can reason rigorously about mathematics and physics. Google's AlphaEvolve breaking the Strassen barrier was a proof of concept. The next decade will determine whether we can generalize this capability across the scientific frontier. [[Seedstrapping vs Bootstrapping#**Why AI Has Made Seed-Strapping the Ultimate Model**]] ## Key References **AI for Mathematical Discovery** - AlphaEvolve (Google DeepMind, May 2025): Gemini powered coding agent that evolved algorithms for matrix multiplication, breaking 56 year record - AlphaProof (Google DeepMind, 2024): Formal reasoning system achieving IMO silver medal performance - AlphaTensor (Google DeepMind, 2022): Discovered 47 multiplication algorithm for 4×4 matrices over finite fields **Physics Informed Neural Networks** - Raissi, Perdikaris, Karniadakis (2019): "Physics informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations" *Journal of Computational Physics* - Physics informed neural operator (PINO): Combines training data with PDE constraints for operator learning **Hybrid Quantum PINNs** - Sedykh, Podapaka, Sagingalieva, **Pinto**, Pflitsch, Melnikov (2024): "Hybrid quantum physics informed neural networks for simulating computational fluid dynamics in complex shapes" *Machine Learning: Science and Technology* - Demonstrated 21% accuracy improvement over classical PINNs for 3D Y shaped mixer simulations **Tensor Networks for ML** - Tensor networks for quantum machine learning (Royal Society, 2023): Comprehensive review of QTN architectures - TensorKrowch: PyTorch library for tensor network based machine learning **Hallucination and Verification** - "LLMs Will Always Hallucinate, and We Need to Live With This" (Banerjee et al., 2024): Mathematical proof of structural hallucination - "Hallucination is Inevitable: An Innate Limitation of Large Language Models" (Xu et al., 2024): Learning theoretic analysis ---