Speakers

In the first part of the seminar I will introduce shallow neural-networks from a statistical-mechanics perspective, focusing on simple cases and on a naive scenario where information to be learnt is structureless. Then, inspired by biological information-processing, I will enrich the framework and make the network able to successfully and cheaply handle structured datasets. In particular, I will recast reinforcement and remotion mechanisms occurring in mammal’s brain during sleep into suitable machine-learning hyperparameters. Results presented are both analytical and numerical.

The familiar second derivative test for convexity is shown to yield a useful tool also in the context of matrix-valued functions. We demonstrate its applicability on a number of theorems in this field. These include convexity principles which play an essential role in the Lieb-Ruskai proof of the strong subadditivity of quantum entropy. (Joint work with Giorgio Cipolloni.)

I will present recent results concerning the Bayesian estimation of low-rank matrices corrupted by structured noise, namely rotational invariant noise with generic spectrum. Using the replica method we derive the optimal performance limit. This is possible by exploiting the low-rank structure of the matrix signal implying that we can reduce the model to an effective quadratic model of the Ising type. Secondly, we show that the Approximate Message Passing (AMP) algorithm currently proposed in the literature for Bayesian estimation is sub-optimal. Exploiting the theory of Adaptive Thouless-Anderson-Palmer equations by Opper et al. we explain the reason for this sub-optimality and as a consequence we deduce an optimal Bayesian AMP algorithm with a rigorous state evolution matching the replica prediction.

In this talk I try to show how three main milestones reached in the past two centuries regarding our comprehension of information processing systems, namely Pavlov's Classical Conditioning, Hebb's Prescription "neurons that fire together wire together" and Hinton's "Contrastive-Divergence-like" learning rules, can be understood as essentially three different manifestations of the same phenomenon that is the intrinsic glassy nature of Computation. The underlying methodological approach of the talk will be the statistical mechanics of spin glasses and neural networks. Minimal Biblio: [1] Agliari, Elena, et al. "Multitasking associative networks." Physical review letters 109.26 (2012): 268101. [2] Alemanno, Francesco, et al. "Supervised hebbian learning." Europhysics Letters 141 (2022): 11001. [3] Agliari, Elena, et al. "From Pavlov Conditioning to Hebb Learning." Neural Computation 35.5 (2023): 930-957.

We present the TAP approach for deriving the free energy (i.e. the Gardner formula) for perceptron models. This is based on a joint paper with Shuta Nakajima, Nike Sun, and Changji Xu. We also discuss some open problems around the perceptron: For instance the possibility of a Sanov type theorem for the perceptron. An old joint paper with Nicola Kistler introduced and analyzed a perceptron version of the GREM. This suggested the possibility of a new version of the cavity method, which until now did not materialize.

Matrix factorization is an important and challenging mathematical problem encountered in the context of dictionary learning, recommendation systems and machine learning. The study of its Bayes-optimal limits, namely the insurmountable bounds provided by information theory, presents several obstacles that are still hard to overcome. In this talk, I will abandon Bayes-optimality, in favor of an alternative procedure, called “decimation". Decimation is shown to map matrix factorization into a sequence of neural network models of associative memory, of which the Hopfield model is a celebrated example. Each of these networks turn out to depend on the order parameters of the previous ones, that are in turn linked to their retrieval performances. Although sub-optimal in general, decimation has the benefit of completely analyzable performances. Finally, I will exhibit an “oracle” algorithm based on the ground-state search of a neural network, which shows performances that match the theoretical prediction. Based on: Camilli, Francesco, and Marc Mézard. "Matrix factorization with neural networks", Physical Review E (2023, to appear), arXiv:2212.02105.

Restricted Boltzmann Machines (RBM) can be interpreted as generalised Hopfield models, in which patterns are learned from the data and the energy of a configuration is a non-quadratic function of its projections along the patterns. In my talk I will focus on the problem of sampling transition paths in a RBM model. I will present both a Monte-Carlo algorithm to sample transition paths and a mean-field approach to compute optimal transition paths and other statistical physics properties, such as the probability to escape from a local minimum and the path entropy. I will then consider application to fitness landscapes inferred from in silico and real protein sequence data, more precisely, transition paths allowing for switching the binding specificity of a small protein binding domain. If time permits I will also introduce a second way to sample paths that switch specificity by controlling the representations learned by the RBM [3]. References [1] E. Mauri, S. Cocco, R. Monasson. Mutational paths with sequence-based models of proteins: from sampling to mean-field characterization. Phys. Rev. Lett. 130, 158402 (2023) [2] E. Mauri, S. Cocco, R. Monasson Direct vs. global transition paths in Potts-like energy landscapes, arXiv:2304.03128 (2023). [3] J. Fernandez-de-Cossio-Diaz, S. Cocco, and R. Monasson. "Disentangling representations in restricted Boltzmann machines without adversaries." Physical Review X 13.2 (2023): 021003.

The definition and manipulation of Langevin equations with multiplicative white noise need special care: one has to specify the time discretisation and a stochastic chain rule must be used to perform non-linear changes of variables. However, while discretisation-scheme transformations and non-linear changes of variable can be safely performed at the level of the Langevin equation, these same transformations lead to inconsistencies in the path-integral representations that are commonly used in the literature. I will describe how to cure these problems in stochastic systems described by one degree of freedom. Concretely, I will present the construction of a path integration through a new time-discretisation procedure that is free of this long standing problem. Time permitting, I will give some hints on the generalisation to problems with more than one degree of freedom. This talk is based on the papers: Path integrals and stochastic calculus Thibaut Arnoulx de Pirey, Leticia F. Cugliandolo, Vivien Lecomte, Frédéric van Wijland arXiv:2211.09470, Adv. in Phys. (2023) Building a path-integral calculus: a covariant discretization approach Leticia F. Cugliandolo, Vivien Lecomte, Frédéric Van Wijland arXiv:1806.09486 J. Phys. A: Math. Theor. 52 50LT01 (2019)

We propose a generalization of the Hamming distance to qubits and a generalization of the Lipschitz constant to quantum observables. We apply the quantum Hamming distance as cost function of the quantum version of Generative Adversarial Networks (GANs). Quantum GANs provide an algorithm to learn an unknown target quantum state and constitute one of the most promising applications of near-term quantum computers. The quantum Hamming distance makes learning more stable and efficient, and the proposed quantum GANs can learn a broad class of quantum data with remarkable improvements over the previous proposals. Furthermore, we apply the quantum Hamming distance to prove extremely tight limitation bounds for variational quantum algorithms for combinatorial optimization problems, which consist in finding the optimal element within a finite set of possible choices and have an extremely wide application spectrum. Our bounds prove that many of such problems cannot be solved by quantum circuits with constant-depth.

Spin glasses with continuous variables, provide models for low temperature excitations in glasses. Unlike the SK model $O(m)$ long range vector spin glasses present a zero temperature spin glass transition at a finite field if $m>2$. This transition reflects the properties of the ground state of the system as a function of the magnetic field. The spectrum of the Hessian is gapless both in the paramagnetic and in the spin glass phase, with a pseudo-gap behaving as $\lambda^{m-1}$ in the paramagnetic phase and as $\lambda^{1/2}$ in the spin glass phase. Despite the long-range nature of the model, the eigenstates close to the edge of the spectrum display quasi-localization properties. We show that the paramagnetic to spin glass transition corresponds to delocalization of the edge eigenvectors. In the p-spin version of the model the situation is different. Typical glassy minima display $\lambda^{m-1}$ pseudo-gap and quasi-localized modes. One also finds rare ultra-stable minima with a gap in the spectrum of excitations. I will briefly discuss how the results are modified in finite connectivity models.

We recall the main features of the replica symmetry breaking method in the frame of the interpolation perspective. In particular, we show how the functional order parameter is emerging in the case of the Sherrington-Kirkpatrick spin glass model, and the way how symmetry breaking is originated, by extending on features of the explicitly solvable Derrida Random Energy Model.

Under the same assumptions and level of rigor as the previous work of Franz, Mézard, Parisi and Peliti, one can show that the ultrametric solution for the equilibrium measure holds if and only if the system's dynamics spontaneously split into widely separated timescales with only one temperature per timescale for all observables.

This talk provides a brief introduction to quantum computation, and shows how quantum information processing can be used to analyze the generation of complexity in the universe. We begin with how quantum fluctuations provide the seeds for large scale cosmological structure, discuss how the quantum structure of chemistry forms the basis for biological information processing, and end with how quantum computing provides novel methods for machine learning and artificial intelligence.

This work is a new approach to analyze the gradient flow for a positive definite matrix denoising problem in an extensive-rank and high-dimensional regime. Recent linear pencil techniques of random matrix theory are used to derive fixed point equations which track the complete time evolution of the matrix-mean-square-error of the problem. A few predictions of the formalism will be illustrated by way of examples, and in particular continuous phase transitions in the extensive-rank and high-dimensional regime, which connect to the classical phase transitions of the low-rank problem in the appropriate limit. This is joint work with Antoine Bodin.

We consider a class of mean-field disordered models characterized by the Nishimori property. These models describe high dimensional inference problems in the Bayesian optimal setting. It is well known that the Nishimori property forces the overlap to be self averaging. We study the fluctuations of the overlap around its mean value proving that the joint distribution of the overlap array satisfies a multivariate central limit theorem. Based on a joint work with F. Camilli and P. Contucci, preprint arXiv:2305.19943.

Spin glass models with orthogonally invariant coupling matrices were introduced as tractable proxies for certain deterministic systems exhibiting spinglass behavior. This family includes the Sherrington-Kirkpatrick model and the Gaussian Hopfield model as special cases. The static behavior of this general family is still not well understood. We will first discuss some recent progress in understanding the behavior of these models at high-temperature. Second, we will turn to the question of universality i.e., whether these random models accurately track the behavior of the aforementioned deterministic systems. In particular, we will discuss universality of a class of Approximate Message Passing algorithms, used to solve the TAP equations in these models. Time permitting, we will also discuss connections with high-dimensional statistics and machine learning.

I will present a new approach toward understanding the spin glass phase at zero and low temperature by studying the stability of a spin glass ground state against perturbations of a single coupling. Four proposed scenarios for the low-temperature spin glass phase --- replica symmetry breaking, scaling-droplet, TNT and chaotic pairs --- will be analyzed through the lens of the predictions of each scenario for the lowest energy large-lengthscale excitations above the ground state. Using a new concept called sigma-criticality, which quantifies the sensitivity of ground states to single-bond coupling variations, each of these four pictures can be identified with different critical droplet geometries and energies. If time permits, I will also present necessary and sufficient conditions for the existence of multiple incongruent ground states. (This work was done in collaboration with Chuck Newman.)

We define a generalization of the Wasserstein distance of order 1, also known as the earth mover's distance, for quantum spin systems on infinite lattices. We call this metric the "specific quantum W1 distance" and it is based on a previously defined W1 distance for qudits. We show that the specific quantum W1 distance and a generalization of the Lipschitz constant for quantum observables are dual. We derive a continuity bound for the von Neumann entropy in terms of the quantum W1 distance. Finally, we prove that local commuting quantum interactions above a critical temperature satisfy a transportation-cost inequality, implying the uniqueness of their Gibbs states. Based on joint work with G. De Palma (arXiv:2210.11446).

We present a variational expression for the free energy of mean-field spin glasses such as the Sherrington-Kirkpatrick model in a transversal magnet field. This includes a clarification of the order parameter of such quantum glasses. (Based on joint work with Chokri Manai)

In this talk I will outline a novel approach to quantum mereology based on minimal information scrambling. Generalized quantum subsystems are defined by pairs of von Neumann algebras and their commutant and their scrambling in terms of a novel Algebraic Out of Time Order Correlation (A-OTOC) function. The short time expansion of the A-OTOC allows one to define a notion of Gaussian Scrambling rate. The latter has a simple geometrical interpretation, and its local minima provide an operational criterion for the selection of emergent subsystems. Examples of factors and maximal abelian algebras will be discussed and shown that in these key cases the formalism leads to physically meaningful connections to operator entanglement and to coherence generating power respectively. References: P. Zanardi, E. Dallas, S. Lloyd, Operational Quantum Mereology and Minimal Scrambling, arXiv:2212.14340 P. Zanardi, Quantum scrambling of observable algebras, Quantum 6, 666 (2022),