Current and future strategies for end-to-end data analysis at LCLS
Fred Poitevin, Richard O. Claus, D. S. Damiani, Christopher Ford, Mikhail Dubrovin, W. Kroeger, Stefano Marchesini, Valerio Mariani, Riccardo Melchiorri, Silke Nelson, C. P. O’Grady, Julieth Otero, Omar Quijano, Murali Shankar, Monarin Uervirojnangkoorn, Matthew Weaver, Seshu Yamajala, Cong Wang, Jana Thayer · Structural Dynamics · 2025
The efficiency, and sometimes success, of experiments carried out at LCLS often depends on getting timely information about the past and current state of the experiment, in order to inform decisions for the reminder of the beamtime. Timely information can mean different things along the steps leading from raw data to final results. Real-time feedback can be obtained from very early steps to quickly guide the instrument - for example, during experiments delivering sample with jets, the relative position between the sample injector and the point of interaction between the liquid jet and X-ray beam is constantly adjusted to maximize hit rate. Decisions that need to be made on slightly longer timescales can be informed by intermediate processing steps; for example in serial crystallography (SFX), the position of the detector or dilution of the sample can be optimized against the crystal indexing rate. Finally, decisions that need to be made on even longer timescales benefit from efficient end-to-end processing, typically done “offline”, yielding statistical results about the quality of the experiment; for example, merging statistics in SFX can be monitored to decide when to switch sample or measurement conditions. In this talk, I will present the infrastructure that has been built in the LCLS Data Systems to facilitate the latter type of decisions. I will attempt to describe the complexity of this infrastructure, which needs to robustly span and handle the whole data life-cycle from the hutch to supercomputers, while highlighting efforts made in hiding the complexity to help users focus on the science. In particular, I will describe the data processing workflows that can be operated from the experiment interface in the browser. With the coming online of LCLS-II in 2023, and its hard X-ray upgrade expected in 2026, a step change in data generation rates will exert a stress on the ability to efficiently process and inform experiments. I will give a brief overview of the data reduction pipeline (DRP) that has been built within LCLS Data Systems to handle order of magnitudes more data than LCLS-I ever produced. The DRP is a hardware infrastructure neatly putting together compressor nodes, monitoring nodes and event building nodes between hutches and disks which together make possible drastic reduction in storage needs while allowing to preserve information. To guarantee that this is true, effort is underway to devise generic compression algorithms that efficiently run in the DRP. With this background in mind, the remainder of the talk will sketch a possible way forward to bringing the current “offline” end-to-end processing directly into the DRP, adding experiment specific algorithms to the existing generic ones. In particular, I will discuss how work pioneered in cryoEM leveraging differentiable simulators could be extended to the realm of X-ray diffractive imaging and mapped onto the LCLS-II Data Systems.