Supervised and Unsupervised Machine Learning Applied to Challenging and Rapid Diffraction and Structural Problems

Development of next generation materials for applications in sustainable energy and beyond require us to study the structure of real materials in real devices even as they operate: for example, putting operating batteries in the beam, studying spatially resolved labs-on-chip, doing real-time autonomous experiments and using computed tomography to see diffraction from cross-sections of bulk samples. These developments, powered by wonderful synchrotron and neutron source and detector developments, present major challenges on the data analysis side. Now we are putting heterogeneous devices in the beam and getting signals from different parts of them. We have bad powder averages (spotty data) because we can't spin the battery, and single crystal spots coming from some component in the setup that happens to be in the way of the beam. We have unknown and unexpected phases coming and going, and want to extract tiny signals from large backgrounds. I will present some of the data analysis, algorithmic and computational developments that are helping us to overcome these challenging situations and not only recovering from 'bad data', but also turning bad data into good data. Spotty powder patterns have more information in them than smooth powder rings. I will describe some new approaches, algorithmic, statistical, machine learning and otherwise, that are helping us move the goalposts in this domain, which can open up new opportunities for studying complex heterogeneous samples with hard x-rays.