I’ll present our work M Adam: Low-Precision Training via Mantissa–Exponent Optimization on Wednesday, a collaboration with Xiaoyuan Liang, @mabeto5p and @AnimaAnandkumar icml.cc/virtual/2026/poster/…

A principled construction for turning CNNs and transformers into resolution-agnostic neural operators Train a U-Net or a ViT at one grid resolution, then run it at another, and performance usually falls apart. The receptive field and the tokenization are tied to pixel indices rather than physical coordinates, so changing the grid spacing quietly changes the operation the layer performs. Fine for vision, a real problem for scientific ML. Many physical systems (fluid flow, heat transfer, electromagnetism) are governed by PDEs and live on continuous function spaces. The object you want to learn is an operator: a map from one function, say a forcing term or coefficient field, to another, the solution. But standard networks consume finite vectors, and simulation data arrive on meshes and point clouds whose resolution varies from run to run. Julius Berner and coauthors distil clear principles for neural operators, models that map between function spaces and stay consistent across discretizations. The central move: figure out which continuous operator a layer is secretly approximating, then discretize it so it respects coordinates and quadrature weights instead of array indices. Sums over indices become integrals over the domain, and latent interfaces stay independent of input resolution. The construction converts familiar blocks almost one to one: fully connected layers become integral transforms, convolutions become spectral operators (the FNO), attention becomes a global integral operator. On a Navier-Stokes benchmark, models trained only at resolution 128 tell the story: FNO and the OFormer transformer generalize from 64 up to 1024, while U-Net and ViT degrade sharply off the training grid. Quadrature weights matter too: drop them and aggregating irregularly spaced values overweights dense regions, so the output never converges. For pipelines in drug discovery, materials development and energy modeling, the payoff is training once and deploying across resolutions: learn on cheap low-resolution simulations, reserve a few expensive high-resolution runs, and query the trained model at whatever fidelity the task needs. Paper: Berner et al., Nature Machine Intelligence (2026), CC BY 4.0 | doi.org/10.1038/s42256-026-0…

🔬 Weekly Science Long Read 🌍 🧪 Can AI truly accelerate scientific discovery? In a new Dædalus article, Board member @AnimaAnandkumar argues that AI for science must understand the physical world, not just generate ideas. Article: tinyurl.com/2edbu24k

My talk will be soon live-streamed. See you there!

Thank you @CSP_live and @ScienceBoard_UN It was wonderful to meet all the members for the first time in person in Turin since I joined this March. This is a pivotal time for #ai and we need to ground it in a scientific way. Fear mongering about #ai is a great disservice especially to young people. We need to maximize the benefits of #ai for scientific discovery.

Science x AI Summit (UC Riverside) is next Monday, June 29, 9 AM PT. AI for analyzing scientific data, model synthesis, and theory-building. Terence Tao, Barry Barish, Anima Anandkumar (@AnimaAnandkumar), Rana Adhikari (@RanaXAdhikari), and more. Live on X, Part I (morning): x.com/i/broadcasts/1RJjppeXw…

🎉Congrats to @CaltechLCSSP's @rmichaelalvarez & @RKocielnik, @pengrui_han, Peiyang Song, Myrl Marmarelis, Ramit Debnath, & @Caltech's Dean Mobbs & @AnimaAnandkumar on their @icmlconf 2026 paper becoming an oral presentation! arxiv.org/abs/2606.12730

Our paper, “Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior,” has been selected for Oral Presentation at CTB @icmlconf * Paper: arxiv.org/abs/2606.12730 * Website: psychology-of-ai.github.io/p… * Code: github.com/psychology-of-AI/… A central question in AI evaluation is whether we can use low-cost self-report probes to anticipate how LLMs will actually behave in tasks. In our earlier work, “The Personality Illusion,” we found that LLMs can give coherent personality self-reports that do not reliably predict behavior. This paper asks a follow-up question: When do self-reports actually track behavior, and what are the failure modes where they don't? Across 11 LLMs, 4 behavioral tasks, and a 2 × 2 × 2 experimental design, we find that self-report–behavior coherence exists, but it is selective: 1) The instrument matters. Broad Big Five personality traits do not predict task behavior well. But a more behavior-specific framework, the Theory of Planned Behavior, can recover much stronger coherence under favorable conditions. 2) Context matters. When self-reports and behavior happen in the same conversation, coherence can reach human-level intention–behavior baselines. But when they happen in separate conversations, coherence often collapses. 3) The task matters. Coherence survives better for behaviors anchored outside the immediate prompt, such as implicit bias and aspects of honesty. It collapses for behaviors strongly shaped by the local context, such as sycophancy. 4) Personas are not a fix. Persona prompting makes models’ self-reports more stable across conversations, but it does not reliably bring behavior into alignment. This is especially important for persona-customized AI systems: changing what a model says about itself does not necessarily change what it does. The takeaway: LLM self-reports should not be treated as context-free behavioral diagnostics. If we want to use psychometric probes for AI safety, deployment, or model evaluation, we need task-specific instruments, behaviorally grounded validation, and careful separation between what a model says and what it actually does. Huge thanks to my co-authors @RKocielnik Pengrui Han, Peiyang Song, Myrl G. Marmarelis, Ramit Debnath, Dean Mobbs, and R. Michael Alvarez, and to the @Caltech Linde Center for Science, Society, and Policy @CaltechLCSSP

Check out our work on end-to-end ultrasound using neural operator for lung aeration pmc.ncbi.nlm.nih.gov/article… We directly reconstructs lung aeration maps from RF data, bypassing the need for traditional beamformers and indirect interpretation of B-mode images.

This is something I have been emphasizing since we started our work on Neural Operators. We very quickly went from simple fluid dynamics benchmarks to hard problems like building the first high-resolution AI-weather model, FourCastNet, and modeling turbulence in nuclear fusion. For those applications, we got speedup of 10,000 - million times. Simple benchmarks are great to test new architecture/algorithms work, but not the end.

Nice work studying zero shot super resolution in neural operators.

Great to see extrapolation success with FNOs.

I am thrilled that my article in @americanacad Daedalus special issue on AI & Science: What Is the Future of Discovery? edited by James Manyika. amacad.org/daedalus/ai-scien… I talk about : How Do We Build AI to Push the Frontiers of Scientific Discovery? Scientific progress is limited not by a lack of new ideas but by the time and cost involved in physical experimentation. Scientific discovery is a needle in the haystack problem: it does not help if AI gives you a vastly bigger haystack. Without knowing if any of the ideas work, an AI system that designs experiments just increases the effort required, since performing the experiments to validate the ideas is the real bottleneck. In my view, AI’s most transformative impact in enabling scientific discoveries lies in reducing the need for such experiments. To get there, we need to build AI models that are able to granularly simulate and understand physics at all scales, rather than just abstractly reason in the textual domain. I explore what methods like Neural Operators have already helped achieve, what still needs to be done, and what lies ahead.

We introduce Sparse Autoencoder Neural Operators (SAE-NOs), a functional framework for representation learning and mechanistic interpretability that treats data as samples from underlying continuous functions and learns mappings between function spaces. Standard SAEs (SAE-MLP) represent each concept with a scalar activation and a vector-valued dictionary atom, limiting their ability to capture how and where a concept is expressed across structured domains. SAE-FNO introduces feature-map representations with both concept sparsity and domain sparsity, allowing the model to capture not only which concepts are active, but also where and how they are expressed across the domain. This is a joint collaboration, between @UAlberta/@AmiiThinks and @Caltech, with Ailsa Shen and @AnimaAnandkumar. 1/ arXiv: arxiv.org/abs/2509.03738

Very excited to finally release TorchLean publicly! I also wrote a longer blog on why I think this matters: robertj1.com/torchlean_verif… Thread below :)

TorchLean codebase is now available! TorchLean is a Lean 4 framework for verified neural-network software. It supports typed tensors, runnable training, graph IRs, verified autograd, Float32/IEEE semantics, CROWN / IBP-style verification, certificate checking, PyTorch interop, and CUDA/GPU execution. After feedback and comments on our original post, we expanded TorchLean substantially: neural operators/FNOs, diffusion models, GPT-style text models, GPT-2-style runs, Mamba/state-space models, RL, 3D vision certificates, Bug Zoo case studies, PyTorch interop, and more. Project page: lean-dojo.github.io/TorchLea… Codebase: github.com/lean-dojo/TorchLe… @Robertljg, Jennifer Cruden, Will Adkisson, Xiangru Zhong, @huan_zhang12 @caltech #MachineLearning #ScientificComputing #Lean #FormalVerification

Nominations are now open for the Pritzker Prize for AI in Science Research Excellence! This prize honors outstanding researchers advancing both AI and the natural sciences or engineering. Nominate someone today! 🔗 datascience.uchicago.edu/res…

Accurate and scalable deep Maxwell solvers Maxwell's equations are the bedrock of photonic device design, from metalenses to chip-scale wavelength multiplexers. Solving them over realistic device sizes (hundreds of wavelengths, with subwavelength dielectric features) is computationally brutal. Neural network surrogates have been promising on toy problems but rarely scale: fixed domain sizes, narrow parameter ranges, no general boundary conditions, accuracy that degrades as the problem grows. Chenkai Mao and Jonathan Fan at Stanford propose a different recipe. Instead of training a network to solve the full problem, they train a neural operator on subdomains and plug it into classical iterative methods. The subdomain network is a modified Fourier neural operator that takes arbitrary Robin-type boundary conditions as inputs, used as a flexible preconditioner inside F-GMRES. It gives bounded-accuracy subdomain solutions, and reaches double precision at inference despite single-precision training. The interesting move is at the global scale. They wrap the subdomain solver in an overlapping Schwarz domain decomposition loop, and use the same network to cheaply solve the subdomain eigenvalue problems that build a coarse space for two-level Schwarz. That coarse correction gives near-optimal scaling, where iteration counts stay roughly constant as the global problem grows. A single network handles different sizes, resolutions, wavelengths and dielectric distributions, with 20 to 50x fewer iterations than CPU GMRES or BiCGSTAB. They benchmark up to ~3000x3000 grids and 200 wavelengths, then plug the solver into adjoint-based optimization to inverse-design freeform devices: a wavelength division multiplexer, a near-infrared metalens, and a volumetric coupler. Trajectories track ground-truth FDFD almost exactly. For photonics, semiconductors and optical communications, this makes neural surrogates operationally useful for real device design. Training only a subdomain model and letting iterative methods handle global scaling is a reusable pattern across PDE problems in heat transfer, acoustics and mechanics. Paper: Mao & Fan, Proc. of the National Academy of Sciences (2026) | journal license doi.org/10.1073/pnas.2530330…

Excited to share that The Personality Illusion has been accepted to ICML 2026 🥂 We show that LLMs' self-reported personalities are systematically dissociated from their actual behavior ：） Huge thanks to my amazing collaborators and advisors! @RKocielnik @p_song1 Ramit Debnath, Dean Mobbs @AnimaAnandkumar @rmichaelalvarez #ICML #Caltech #LLM

Geometric operator learning is challenging because high-quality simulations on complex geometries are expensive. In GeoPT, we pretrain on low-cost graphics datasets augmented with simple dynamics, showing promising scaling behavior.