Symposium EQ04—Machine Learning on Experimental Data for Emergent Quantum Materials

Understanding symmetry’s role in the physical sciences is critical for choosing an appropriate machine learning method. For example, coordinates used to describe positions of atoms in a material are traditionally a challenging data type to use for machine learning -- coordinates and coordinate systems are sensitive to the symmetries of 3D space: 3D rotations, translations, and inversion. One of the motivations for incorporating symmetry into machine learning models on 3D data is that it eliminates the need for data augmentation -- the 500-fold increase in brute-force training necessary for a model to learn 3D patterns in arbitrary orientations. Additionally, many features of physical systems are consequences of symmetry -- geometric tensor properties, point groups, and space groups, degeneracy, phase transitions, atomic orbitals, etc. By incorporating symmetry into a model by construction (rather than by training), these consequences arise naturally in the behavior of the model (even in untrained models). There are two general types of symmetry-aware models: invariant and equivariant. Invariant models get rid of coordinate systems by only dealing with quantities that are invariant to the choice of coordinate system (scalars), while equivariant models preserve how quantities predictably change under coordinate transformations. While invariant models are the most prevalent symmetry-aware models because they are mathematically simpler, equivariant models more faithfully represent the complexity of physical interactions. In this tutorial we will discuss how symmetry emerges when representing physical systems and strategies for accommodating these symmetries when building machine learning algorithms. As an example, we will give an overview of some of the properties of Euclidean neural networks, a general neural network framework that fully treats the equivariance of physical systems and naturally handles 3D geometry and operates on the scalar, vector, and tensor fields that characterize them. Later lectures will dive into how these and related methods are implemented and applied to real-world materials challenges.

In this tutorial we will introduce useful concepts group theory for creating symmetry invariant and
equivariant algorithms for materials: representation of groups and the vector spaces they act on, including irreducible representations, and how to interact and combine group representations, e.g. tensor products.

We will connect this theory to practical examples in material science, for example we will give examples of tensor properties of materials and how to express them in terms of Cartesian tensors and irreducible representations.

Euclidean neural networks use these group theoretic principles to achieve global and local
equivariance to 3D rotations, translations, and inversion at every layer. We will go through concrete coding examples for how to build these models using e3nn: a modular open-source PyTorch framework for Euclidean Neural Networks (https://e3nn.org). In e3nn, the group theoretic tools discussed above are implemented as practical PyTorch modules. We will cover the Irreps class and operations such as
TensorProduct. We will also touch upon topics such as nonlinearities and how these differ from traditional neural networks. The e3nn package will continue to be used throughout the morning and afternoon tutorials to demonstrate core concepts in using invariant and equivariant algorithms for practical materials applications.

Some of the most challenging aspects of applying machine learning to atomic configurations have to do with interfacing atomic configurations (point clouds and species labels) with machine learning methods in a way that 1) respects geometric symmetry, 2) is flexible with respect to the number of atoms in a configuration, and 3) respects indistinguishability of atoms. e3nn handles each of these issues naturally, making it an ideal framework for machine learning on atomic configurations.
In this hands-on lecture we will use e3nn to 1) demonstrate how to project atomic environments onto basis function expansions (using the SphericalTensor and FourierTensor objects in e3nn), 2) use those expansions to calculate invariant representations of the atomic configurations, and 3) show how to use optimization techniques and the automatic differentiation capabilities of e3nn to recover geometry from these representations.

Machine-Learning Interatomic Potentials have over the past decade emerged as a powerful tool for increasing time- and length-scales of molecular dynamics simulations while retaining the high accuracy of the reference calculations they were trained on. Recently, the NequIP potential, an equivariant neural network-based interatomic potential has been demonstrated to obtain an unprecedented level of accuracy and sample efficiency, allowing ML Interatomic Potentials to be trained with up to 1000x fewer training data than competing methods.

In this tutorial, we will demonstrate how to train and subsequently deploy NequIP interatomic potentials in molecular dynamics simulations. We will outline best practices for data set selection and training of the NequIP model as well as how to successfully run efficient simulations. We will give an introduction to the NequIP software code and detail how to work with its interface to the LAMMPS molecular dynamics software suite, a highly efficient code for large-scale molecular dynamics simulations, enabling the efficient and accurate study of a diverse range of materials.

One of the fundamental challenges of atomic-scale science for the 21st century is the ability to simulate large molecules with quantum mechanical accuracy. What makes this problem so challenging is the poor scaling of solving the equations of quantum mechanics on a classical computer. Even the most efficient quantum chemistry programs, such as Density Functional Theory (DFT), scale with the cube of the system size. This makes accurate simulations of large molecules with these methods fundamentally impossible.

One proposed method of breaking this “quantum scaling limit” is to use machine learning (ML) to learn quantum mechanics. In this tutorial, we will demonstrate how Euclidean Neural Networks in general, and e3nn in particular, are well-suited for this task. As an instructive example, we will show how to predict the electron density of clusters of water molecules and examine how that model can transfer to larger systems.

Phonon density of states (DoS) is a key determinant of many properties of crystalline solids. However, acquisition of phonon DoS is experimentally nontrivial due to limited inelastic scattering facility resources, and at the same time computationally expensive for complex materials, particularly disordered systems. These challenges further lead to a scarcity of phonon DoS data to perform data-driven studies.

e3nn enables efficient and direct prediction of phonon DoS using easily accessible input information about atomic structures, specifically atom types, masses, and positions. By directly incorporating crystallographic and 3D spatial symmetry constraints, e3nn can achieve good performance even when trained on a modest phonon DoS dataset of ~1000 examples. In this lecture, we will demonstrate how to predict phonon DoS from crystal structures using e3nn, as well as introduce the natural extension of predicting the DoS of alloy materials.

Quantum material has become a new research frontier and is widely believed to be the cornerstone of the next technological revolution. To initiate revolutionary innovations, the precise control of the electronic properties in quantum materials is on demand. The ultrafast laser pump becomes a promising control approach due to the rich degrees of freedom and the extra time axis. However, it also places an urgent requirement for the ultrafast identification of the materials' property.

In this talk, I will briefly introduce the new ultrafast pump-probe technique, time-resolved resonant inelastic x-ray scattering, from a theoretical and computational perspective. Then I will use two typical classes of quantum materials, the topological material and correlated magnets, as examples to sketch the novel applications. As a nonlinear pump-probe technique, its intermediate state selectivity provides a clean characterization of topological edge states. Its sensitivity to multiple excitations and the broad momentum coverage gives a unique opportunity to Floquet-engineer spin excitations in correlated materials.

The ultimate goal of any atomistic design process, be it for materials or molecules, is to start with one or more target properties and predict atomic geometries that are statistically likely to possess these properties. In this talk, I will discuss how to construct complete, symmetry invariant descriptions of atomic environments that can be inverted to recover the original environment. Using this representation I will show that it is possible to build a model that can predict the conductivity of ionic liquids, an important potential electrolyte for future battery technologies. By training on a dataset of experimental results the model learns how to attribute contributions to the overall conductivity to fragments of the anion and cation. This allows us to virtually screen thousands of anion/cation combinations to pinpoint those with high conductivities. Furthermore, it opens the door to designing entirely new molecules by combining fragments to achieve properties that go beyond those that are possible using current ionic liquids.

Phonon transport at the nanoscale intimately relates to various crucial applications in the modern society. However, our understandings about frequency-dependent transport mechanisms, especially across an interface, are scarce due to the fundamental gap between observables and computables. In this talk, we introduce a framework that combines the scientific machine learning and the Boltzmann transport equation (BTE) to unravel microscopic phonon transport information inside heterostructures from the ultrafast electron diffraction (UED). We show that the time- and momentum-resolved diffraction patterns can be reliably interpreted by our approach to reconstruct frequency-dependent properties such as transmission coefficients and relaxation times, as well as other general properties like material-air emissivities, from which real-time and real-space phonon dynamics can be further recovered. We validate our approach through several numerical scenarios and apply it to analyze experiment measurements of an Au/Si heterostructure. Our work opens a new avenue to reveal phonon transport mechanisms across interfaces at the nanoscale.

The study of topological band structures is an active area of research in condensed matter physics and beyond. Here, we combine recent progress in this field with developments in machine learning, another rising topic of interest. Specifically, we introduce an unsupervised machine learning approach that searches for and retrieves paths of adiabatic deformations between Hamiltonians, thereby clustering them according to their topological properties. The algorithm is general, as it does not rely on a specific parametrization of the Hamiltonian and is readily applicable to any symmetry class. We demonstrate the approach using several different models in both one and two spatial dimensions and for different symmetry classes with and without crystalline symmetries. Accordingly, it is also shown how trivial and topological phases can be diagnosed upon comparing with a generally designated set of trivial atomic insulators.

Superhard materials exhibit a Vickers hardness H > 40 GPa, and they have extensive industrial and technological applications. Due to the huge search space of possible element combinations, it is challenging to explore new superhard ternary or quaternary materials. Here, I will first discuss machine learning (ML) discovery of new superhard B-C-N and B-N-O compounds. The ML results are validated by evolutionary structure prediction and density functional theory (DFT). In particular, the proposed BC₁₀N has a low formation energy and a high hardness H ~ 86 GPa only next to diamond. I will next discuss the calculations of entropy formation ability (EFA) using both DFT and ML approaches for 5-metal hexagonal high-entropy borides. The results indicate that EFA serves as a good descriptor for the synthesizability of high-entropy materials, some of which have superior mechanical properties promising for extreme-environment applications.

Recently a relationship between the Debye temperature and the superconducting transition temperature T_c of conventional superconductors has been proposed. The relationship indicates that the maximum T_c for phonon-mediated BCS superconductors is at most ~0.1×Debye temperature. In order to verify this bound and develop tools for predicting the Debye temperature we’ve trained machine learning models on over 10000 compounds with just chemical formula and crystal symmetry group as features. By examining over 9000 known superconducting compounds in the NIMS SuperCon database, our predictions show that conventional superconductors in the database follow the previously proposed bound of T_c versus Debye temperature. We will also use density-functional theory calculations to examine compounds very close and very far away from this bound, and discuss possible strategies of increasing T_c by manipulating the electron-phonon coupling in these systems.

Machine learning applications in materials science are often hampered by shortage of experimental data. Integration with external datasets from past experiments is a viable way to solve the problem. But complex calibration is often necessary to use the data obtained under different conditions. In this paper, we present a novel calibration-free strategy to enhance the performance of Bayesian optimization with preference learning. The entire learning process is solely based on pairwise comparison of quantities (i.e., higher or lower) in the same dataset, and experimental design can be done without comparing quantities in different datasets. We demonstrate that Bayesian optimization is significantly enhanced via data integration for organic molecules and inorganic solid-state materials. Our method increases the chance that public datasets are reused and may encourage data sharing in various fields of physics.

Spectral images obtained with Electron Energy-Loss Spectroscopy (EELS) are two-dimensional sets of spectra where each pixel provide information on a localised region of the analysed sample. Each of these spectra contain a plethora of information related to the physical and electronic properties of the material. Exploiting this information requires reliable access to the low-loss region where the zero-loss peak (ZLP) overwhelms the inelastic scattering contributions of the specimen. Here we present a novel approach to process and analyse EEL spectral images in an automated manner based on machine learning. By clustering the constituent spectra into groups associated to sample regions with similar profiles using unsupervised learning, we can then subtract the ZLP in a model-independent manner using deep neural networks. The resulting processed spectral images can then be used to identify spatially-resolved local electronic properties such as the bandgap (and determine whether it is a direct or indirect bandgap) and the optical dielectric function across the sample. Furthermore, we correlate them to the thickness and other structural properties of the sample. In addition, our approach also allows us to identify interesting features in the low-loss region and determine how these features vary in the different regions of the sample. This strategy has been performed on both Tungsten Disulfide and Indium Selenide nanostructures obtaining results that are in agreement with the literature. Our novel method can be extended into higher-dimensional datasets in a straightforward manner and is implemented into a new release of the open source python package EELSfitter.

I will discuss different approaches to improve the accuracy of materials property predictions, combining density-functional theory calculations, different machine learning methods and experimental datasets.

I will present different examples of applications and discuss their performance for the determination of thermodynamic stability, geometry optimization and evaluation of the electronic band gap.

Advancements in microscopy techniques have made materials investigations at nanometer-picosecond (nm-ps) spatio-temporal resolutions possible. This work details the use of a machine learning based approach to extract quantitative information, in terms of optical flow, pertaining to the motion of contrast bands as captured by an ultrafast electron microscope (UEM). UEM is an emerging technique that uses pulsed electron beams to image structural dynamics at nm-ps resolutions. This spatio-temporal characteristic of a UEM dataset is one of the main challenges encountered during its analysis. Classical image processing techniques for optical flow, such as functions that are part of the OpenCV library, are parametric and hence require manual supervision. In this work, a U-net type convolutional neural network (CNN) is designed to take a pair of UEM images at different times as input and generate the optical flow at each pixel as output. A custom loss function is defined, consisting of a photometric loss term and a gradient loss term. The loss is regularized with the output from an adversarial model which is simultaneously trained to distinguish between image pairs with and without spatial displacement of objects. In order to overcome the lack of ground truth data for optical flow, the model is trained in an unsupervised manner. The performance of an OpenCV function for dense optical flow is chosen to be the baseline, and the optimal values for the parameters are selected through a grid search on candidate values. The model proposed in the current work outperforms the baseline method on the photometric loss metric. We detail the approach to study the effect of experiment variables, such as temperature, on the phonon dynamics of 2D semiconductor materials.

Today, experimental data sets for various applications are almost exclusively curated manually because data is scattered throughout the research literature. In this talk, I will present progress towards using natural language processing (NLP) algorithms to automatically collect and organize data on a research topic. Such methods are particularly valuable in fields like quantum materials where simulation is difficult or time consuming, necessitating efficient use of experimental data where it is available. In particular, I will present our work on detecting named entities within research texts and connecting these entities into organized data sets. For example, we have collected data pertaining to how various materials are doped by applying NLP to a large corpus of published abstracts and full texts, and connected doping information with applications. Finally, I will discuss how such methods might be applied to a variety of quantum materials applications and how such methods may complement other sources of data such as theory and simulation.

Inelastic and diffuse neutron scattering hold crucial information on the underlying physical states of materials and models that describe them. The difficulty of handling the large amounts of data and simulations involved as well as challenges of experiment planning and interpretation can be addressed using machine learning. We have been applying these approaches to topological flat band materials of importance as hosts to fractional quasiparticles with anyonic statistics as well as competing disordered phases. Complex behavior due to subtle competing interactions prove to be vital and machine learning provides an effective approach to handling the myriad of phases and behaviors involved. I show outcomes of our machine learning based experiments on quantum and classical spin liquids.

Designing nanostructures with desired optical properties is critical to the field of nanophotonics. Traditionally, numerical methods and theoretical predictions are used to inform on design parameters of nanophotonic elements. On the other hand, having knowledge of the structure-property relationships of a given system provides another data-driven approach to design nanostructures. To pave the way towards stochastic design of nanoplasmonic structures, we establish the correlative relationship between local nanoparticle geometries and their plasmonic responses. By using an encoder-decoder neural networks within the framework of PyTorch, we demonstrate the predictions of spectra from local geometries in the im2spec network, as well as the predictions of local geometry from spectral inputs in the spec2im network [1].
By utilizing scanning electron transmission electron microscopy (STEM), we acquire an electron energy loss (EEL) spectrum image and its associated acquired high angle annular dark field (HAADF) image of a self-assembled monolayer of fluorine and tin doped indium oxide nanocrystal arrays [2]. While the particles mainly undergo self-assembly, many defects and irregularities are observed within and nearby the arrays – for example, missing particles, small clusters of particles, etc. With the knowledge that the plasmonic behavior at a position in space depends primarily on local geometry at that position, subimage-spectrum pairs are generated at each spatial pixel that are used to teach the network the correlative relationship between them. The network encodes the images (spectra) to a low-dimensional feature space consisting of a small number of latent variables, then decodes the latent variables back into spectra (images). The reduced descriptions contained in the so-called latent space can yield a surprising insight into the generative mechanisms of complicated plasmonic responses in nanoparticle arrays [3].
Equipped with the knowledge of correlative structure property relationships gained by the autoencoder networks, we predict the plasmonic response given an unseen nanoparticle geometry. Conversely, we predict nanoparticle geometry given a plasmonic spectrum. This method ultimately enhances the design capabilities for plasmonic systems, and may even permit a route to solving the inverse design challenge in nanophotonics.

[1] S. V. Kalinin et al., Adv. Optical Mater. 2021, 2001808.
[2] S. H. Cho et al., J., Chem. Mater. 2019 31, 2661.
[3] K. M. Roccapriore et al., Small 2021 17, 2100181.

This effort (ML and STEM) is based upon work supported by the U.S. Department of Energy (DOE), Office of Science, Basic Energy Sciences (BES), Materials Sciences and Engineering Division (K.M.R., S.V.K.) and was performed and partially supported (J.A.H., M.Z.) at the Oak Ridge National Laboratory’s Center for Nanophase Materials Sciences (CNMS), a U.S. Department of Energy, Office of Science User Facility. S.H.C acknowledges (NSF, CHE-19052631609656, CBET-1704634, NASCENT, an NSF ERC EEC-1160494, and CDCM, an NSF MRSEC DMR-1720595), the Welch Foundation (F-1848), and the Fulbright Program (IIE-15151071).

Scanning transmission electron microscopy (STEM) has become an important tool for studying atomic structures of complex materials with picometer precision and the associated order parameter fields. The latter includes polarization that is classically quantified by preprocessing images to remove noise and instrumental distortions, finding positions of all the observed atomic columns corresponding and calculating atomic displacements. This process is not only highly time consuming and laborious but also may lead to systematic errors in the polarization values.
In this work, we have proposed an alternate approach of utilizing deep convolution neural networks (DCNNs) for direct polarization mapping from STEM data. The DCNN is trained on the manually-labeled part of the image and is subsequently applied to other images. The training data is constructed using image patches (parts of original images cropped for a specific window size) generated with and without information about atomic positions. The patches are used as the (local) feature set and the corresponding polarization values become the target set. We show that this framework allows predicting polarization maps without atom-finding and compare the results with those obtained using the traditional approach.
We have demonstrated the workflow for STEM images with different Sm dopant concentrations in BiFeO₃, such that the explored feature space encompasses both ferroelectric and weakly-ferroelectric systems. We further intend to apply this technique to Scanning Tunneling Microscopy (STM/S) datasets of quantum materials to study other functionalities such as superconducting gap or the intensity of selected quasiparticle interference spots.

This effort (STEM) is based upon work supported by the U.S. Department of Energy (DOE), Office of Science, Basic Energy Sciences (BES), Materials Sciences and Engineering Division (S.V.K., C.T.N.) and was performed and partially supported (M.Z.) at Oak Ridge National Laboratory’s Center for Nanophase Materials Sciences (CNMS), a U.S. DOE, Office of Science User Facility. This effort (ML) is based upon work supported by the U.S. DOE, Office of Science, Office of Basic Energy Sciences Data, Artificial Intelligence and Machine Learning at DOE Scientific User Facilities (A.G.). The work at the University of Maryland was supported in part by the National Institute of Standards and Technology Cooperative Agreement 70NANB17H301 and the Center for Spintronic Materials in Advanced Information Technologies (SMART) one of the centers in nCORE, a Semiconductor Research Corporation (SRC) program sponsored by NSF and NIST.

Topological insulators and topological semimetals are solid-state systems which display unique macroscopic quantum effects due to the topology of their electronic wavefunction, such as protected edge or surface modes and quantized response functions, and have potential applications in quantum technology. Recently introduced theoretical methods have allowed for their large-scale discovery: databases compiled using ab initio calculations (density functional theory) now contain tens of thousands of topological materials. This makes the application of modern machine learning methods possible. Using gradient boosted trees, we show how to construct a machine learning model which can predict the topology of a given existent material with an accuracy of 90%. Such predictions are orders of magnitude faster and less expensive than actual ab initio calculations. We use machine learning models to probe how different material properties affect topological features. Notably, we observe that topology is mostly determined by the “coarse-grained” chemical composition and crystal symmetry and depends little on the particular positions of atoms in the crystal lattice. We identify the sources of our model’s errors, discuss approaches to overcome them and create a simplified human-readable classification tree. In this talk, we will put a particular empahsis on methodological questions concerning machine learning as well as lessons learned and pitfalls encountered during the research process, in the hope that others looking to apply machine learning to material science may profit.

A system’s spectral emissivity determines how it will radiatively exchange energy with its environment and other interacting systems. This governs many natural phenomena and technologies such as optical sensors, energy harvesting and conversion, targeted drug delivery, sooty combustion, and global warming. Calculating radiative properties from known geometry and designs can be computationally expensive, and trying to invert the problem to come up with designs specific to desired radiative properties is even more challenging (typically one-to-many and nonlinear). In this talk, I will first discuss a machine-learning-based method for both the forward and inverse problem for dielectric and metallic particles. Our decision-tree-based model is able to provide explicit design rules for inverse problems. Furthermore, we can use the same trained model for both the forward and the inverse problem, which greatly simplifies the computation. Second, I will discuss our more recent work using generative models and latent space sampling to prescribe how to nanotexture a surface of a single material in order to produce a variety of desired optical properties through plasmon resonance and photon wave interference effects. These designs are experimentally realized by rapid laser machining of different substrates. Our methodologies show the promise of augmenting optical system optimizations by providing interpretable and actionable learned models for rapidly finding approximate solutions to the inverse design problem.

Machine learning (ML) has been all the rage in materials science in the past 5 years. While there have been promises of transformative acceleration of analysis capabilities, new insights, and even closed-loop control, it has yet to live up to the hype. The core challenge is that while ML models are fast to deploy they require arduous training of ensembles of models, hyperparameter tuning, and validation. In turn, ML ends up taking nearly the same or many times more person-hours as standard analysis protocols. While many ML efforts have focused on unraveling hidden insight in data, ML can also be a powerful tool to accelerate conventional analysis pipelines. There has been a boon in multimodal spectroscopy techniques that collect data with increasing volume, velocity, veracity, and variety. The use of these techniques has become limited not by their capability, but the time required to manage and process this data. Additionally, this imposes challenges to the experimentalist who cannot conduct even cursory on-the-fly analysis to guide subsequent experiments.
Here, we show how the confluence and codesign of physics-informed machine learning models and edge computing hardware can enable real-time streaming processing of band-excitation piezoresponse force switching spectroscopy without distributing computing. We acquire frequency-dependent cantilever resonance responses during ferroelectric switching at a frequency of 1 kHz. We use initial data to train a 1D-convolutional autoencoder model with a latent space that defines the amplitude, phase, quality factor, and resonance frequency. By constraining the decoder to be the empirical function of a simple harmonic oscillator, we can train the model fully unsupervised by minimizing the mean squared reconstruction loss. Training this model from scratch takes less than 5 minutes on 3M examples on a Graphical Processing Unit (GPU, Google Collab Free-Tier). Once trained, the model can fit the resonance curves; however, the necessary throughput of 1 kHz is only achievable through batching as GPU initialization takes ~1 ms. To avoid latencies associated with data movement, we deployed the model on a field-programmable gate array (FPGA). To do this, we pruned and quantized the model to fit the FPGA resources. Using a package hls4ml, we converted our model from TensorFlow to high-level synthesis (HLS). The HLS can then be compiled using vivadoHLS and was ultimately deployed on a Xilinx KU060 coprocessor in a Labview PXI chassis. By deploying this model on an FPGA, we could achieve streaming inference with latencies of <2 μs, orders of magnitude faster than is required.
Furthermore, because the fits are conducted more holistically, the fits are more robust to initialization and noise. We will also discuss how a similar methodology can be applied to fit piezoelectric hysteresis loops. The ability to conduct empirical fitting with sub-ms latency provides a pathway toward real-time control of dynamics processing in materials at beyond-human response times. We will provide additional insight into other characterization modalities, for example 4-dimensional transmission electron microscopy, where a similar methodologies can be applied.