Automatic parameter selection for electron ptychography via Bayesian optimization

www.nature.com/scientificreports OPEN Automatic parameter selection for electron ptychography via Bayesian optimization Michael C. Cao1, Zhen Chen2, Yi Jiang3* & Yimo Han1* Electron ptychography provides new opportunities to resolve atomic structures with deep subangstrom spatial resolution and to study electron-beam sensitive materials with high dose efficiency. In practice, obtaining accurate ptychography images requires simultaneously optimizing multiple parameters that are often selected based on trial-and-error, resulting in low-throughput experiments and preventing wider adoption. Here, we develop an automatic parameter selection framework to circumvent this problem using Bayesian optimization with Gaussian processes. With minimal prior knowledge, the workflow efficiently produces ptychographic reconstructions that are superior to those processed by experienced experts. The method also facilitates better experimental designs by exploring optimized experimental parameters from simulated data. Ptychography is a computational imaging method that has gained great interests in the electron microscopy community1–4. The technique was first proposed by Hoppe in 1 9695 and re-invigorated in recent years with the developments of fast electron d etectors6–11 that can rapidly collect thousands of diffraction patterns per second. Various iterative reconstruction algorithms have been developed to retrieve the scattering potentials of the sample and the wave function of the illumination from intensity m easurements12–14. It has been demonstrated that electron ptychography can break the Abbe diffraction limit of imaging s ystems15 and set a new world record in spatial resolution (0.39 Å) in atomically thin two-dimensional (2D) m aterials3. As one of the phase-contrast imaging techniques, electron ptychography also has high dose efficiency for low-dose imaging ranging from lowdimensional nanomaterials16,17 to biological specimens18,19. An even more critical breakthrough is that electron ptychography can inversely solve the long-standing problem of multiple scattering in thick (> 20 nm) samples and enables a lattice-vibration-limited resolution (0.2 Å)4, as well as three-dimensional depth s ectioning4,20. Despite its great success in achieving record-breaking resolution, ptychography remains a niche technique in electron microscopy due to many practical challenges in both experimental setup and data analysis. In particular, there exist many types of parameters that significantly influence image quality and need to be carefully selected for different data or applications. For example, physical parameters that describe processes such as noise generation, partial coherence, and probe vibration can be modeled in an iterative ptychographic reconstruction, which essentially solves a non-convex optimization problem. Choosing appropriate parameters to account for these practical errors is paramount to achieving solutions that are close to the real object. Other parameters, including the number of iterations, update step size, and initial probe, also influence reconstructions by controlling the convergence process. For simplicity, in the work, we categorize all parameters described above as reconstruction parameters. In addition, experimental parameters, such as scan step size, probe defocus, and camera length also need to be determined before measurement and often limit the best image quality of a given data. Due to virtually infinite possibilities and complex trade-offs between various parameters, it is practically impossible to design and optimize ptychography experiments by searching the entire parameter space. In practical a pplications3,4,16,17, scientists often select parameters manually based on their experiences with the sample or instrument. This can potentially introduce biases to scientific conclusions drawn from the results. Although a few key parameters were systematically studied in previous literature18,21,22, exploring multiple parameters greatly reduces the overall throughput and creates a high barrier for general researchers to adopt the technique. Here we present a general framework for fully automatic parameter tuning in electron ptychography by leveraging Bayesian optimization (BO) with Gaussian processes23—a popular strategy for global optimization of unknown functions. Using experimental ptychography data and state-of-the-art reconstruction algorithms, we demonstrated that our approach can automatically produce high-resolution images after exploring only 1% 1 Department of Materials Science and NanoEngineering, Rice University, Houston, TX 77005, USA. 2School of Materials Science and Engineering, Tsinghua University, Beijing 100084, China. 3Advanced Photon Source, Argonne National Laboratory, Lemont, IL 60439, USA. *email: ; Scientific Reports | (2022) 12:12284 | https://doi.org/10.1038/s41598-022-16041-5 1 Vol.:(0123456789) www.nature.com/scientificreports/ Figure 1.  Schematic of automatic reconstruction tuning with Bayesian optimization. (a) The process aims to find the best ptychographic reconstruction by optimizing an unknown quality function that is data-dependent in general. (b) Bayesian optimization loop strategically determines the next point (indicated in orange) to sample, performs ptychographic reconstruction, and then updates the surrogate model based on the image quality. As the number of iterations increases, the surrogate model becomes closer to the true quality function and more points around the optimum are exploited. (c) The image with the best quality during BO is retrieved as the final reconstruction. of the discretized reconstruction parameter space. We also optimized experimental parameters for ultra-low electron dose levels, providing insights for more robust experimental designs that further to enhance ptychography’s usability. Instead of relying on human intuition and judgment, automatic parameter selection promotes objective and reproducible protocols, paving the way for fully autonomous experiments and data processing for ptychography applications. Results Bayesian optimization with Gaussian process. Bayesian optimization with Gaussian process is frequently used to find global maxima and minima of a black-box function that is unknown and expensive to evaluate. The technique has been used in a wide variety of applications in machine learning24,25, Monte Carlo simulation26, and autonomous controls in microscopy experiments27–29. In general, BO consists of three steps: (1) compute a surrogate function that models the true objective function based on sampled points, (2) determine the next point(s) to be sampled based on an acquisition function, (3) evaluate the objective function at the corresponding points. The surrogate function is described by kernel functions, which affect the periodicity, smoothness, and length scales of the objective function. It also predicts values and their standard deviations at unsampled points, w (...truncated)