Unsupervised deep denoising for four-dimensional scanning transmission electron microscopy

npj | computational materials Article Published in partnership with the Shanghai Institute of Ceramics of the Chinese Academy of Sciences https://doi.org/10.1038/s41524-024-01428-x Unsupervised deep denoising for fourdimensional scanning transmission electron microscopy Check for updates 1 2 1 2 1234567890():,; 1234567890():,; Alireza Sadri , Timothy C. Petersen , Emmanuel W. C. Terzoudis-Lumsden , Bryan D. Esser , Joanne Etheridge 1,2 & Scott D. Findlay 1 By simultaneously achieving high spatial and angular sampling resolution, four dimensional scanning transmission electron microscopy (4D STEM) is enabling analysis techniques that provide great insight into the atomic structure of materials. Applying these techniques to scientifically and technologically significant beam-sensitive materials remains challenging because the low doses needed to minimise beam damage lead to noisy data. We demonstrate an unsupervised deep learning model that leverages the continuity and coupling between the probe position and the electron scattering distribution to denoise 4D STEM data. By restricting the network complexity it can learn the geometric flow present but not the noise. Through experimental and simulated case studies, we demonstrate that denoising as a preprocessing step enables 4D STEM analysis techniques to succeed at lower doses, broadening the range of materials that can be studied using these powerful structure characterization techniques. Scanning transmission electron microscopy (STEM) has proven highly successful and versatile for characterising material structure from the micron down to the atomic scale. For every byte of data generated by traditional monolithic STEM detectors, developments in fast readout electron pixel detectors (e.g. refs. 1,2) now mean we can collect millions of bytes with the added benefit of momentum resolution. Recording the detailed electron scattering distribution has enabled a range of imaging strategies. In four-dimensional (4D) STEM, at each probe position in a two-dimensional scan across a sample a two-dimensional convergent beam electron diffraction (CBED) pattern is acquired (Fig. 1a). Described by the umbrella term of 4D STEM, these imaging strategies include mapping crystal orientation, strain and electromagnetic fields, and phase contrast techniques for determining the crystal structure (see Ref. 3 for a review). While in some cases the known scattering physics leads to constructive processing algorithms to determine the quantity of interest4–6, in many cases machine learning, optimisation and other statistical methods are used to efficiently identify patterns or determine quantities of interest from the data7–13. In most cases, the reliability of the analysis depends on the quality of the data. In contrast to traditional STEM imaging modes such as annular dark field imaging that use only a fraction of the electrons incident upon the sample or recorded by the detector, 4D STEM modes are often considered dose efficient because they make use of the majority of such electrons14,15. Nevertheless, spreading the dose over many detector pixels reduces the signal-to-noise ratio per pixel. The CBED patterns may therefore appear noisy, which can impact forms of 4D STEM analysis that depend on detailed structure in the patterns. In principle, signal-to-noise can only be improved by increasing the number of counts in each pattern, by counting longer (dwell time) and/or increasing count rate (beam current), both of which may be undesirable in practice. First, increasing dwell time increases susceptibility to timedependent variables like sample drift, heating and electric charging, while also increasing dose (number of electrons per probe position). Second, many materials of interest for high resolution characterisation are electron-beamsensitive, for which the dose must necessarily be kept very low16. This then poses the challenge of whether and which 4D STEM methods remain reliable for analyzing noisy data obtained at low dose for beam sensitive applications17,18. While some materials admit material-specific strategies (such as using assumed periodicity to distribute dose widely and solve for an average structure19), this work will focus on denoising as a preprocessing step as a more general strategy. Various denoising strategies have been applied to 2D STEM images20–23 and might be adapted for application to individual CBED patterns, but are not designed to take advantage of the connection between probe position and diffraction coordinate inherent in 4D STEM. One approach that uses the 4D nature of the dataset is the tensor-SVD (TSVD) method of Zhang et al.24. Linear models used in TSVD (also used during compressive 1 School of Physics and Astronomy, Monash University, Melbourne, VIC, Australia. 2Monash Centre for Electron Microscopy, Monash University, Melbourne, VIC, e-mail: Australia. npj Computational Materials | (2024)10:243 1 Article https://doi.org/10.1038/s41524-024-01428-x Fig. 1 | Overview of 4D STEM. a Schematic 4D STEM set-up. At each probe position a convergent electron beam scatters through a material and forms a convergent beam electron diffraction (CBED) pattern. b Experimental CBED patterns from aluminium ([001] zone axis, 300 keV electrons, 15 mrad convergence semiangle, 0.2 Å probe spacing and 8 × 106 e−. Å−2 dose). Visual features in the patterns change smoothly in position, intensity and shape across adjacent probe positions, a kind of geometric flow, with prominent examples indicated by white and orange arrows. c STEM imaging of the total number of electrons in each pattern in the 4D STEM data. The red box indicates the region from which the patterns in (b) are taken. d The probe-position averaged CBED (PACBED) pattern for data in (c), with the bright and dark field regions indicated. sensing25) are efficient and, depending as they do on repetitions in diffraction patterns to provide reliable statistics, are particularly effective for periodic samples. However, many samples of interest are not perfectly periodic, and to handle this larger class of materials we take a different approach. Figure 1b shows several adjacent experimental CBED patterns, a subset of a larger 4D STEM dataset taken from the region indicated by the red box in the STEM image in Fig. 1c. At this 0.2 Å probe spacing, the position, intensity and shape of visual features in the CBED patterns change smoothly between adjacent probe positions, with specific examples indicated by the white and orange arrows. We will use this geometric flow26 connection between adjacent CBED patterns as the basis for denoising 4D STEM data. A useful analogy for our general strategy comes from language models: having learned relationships between words as they occur in the context of sentences27, a language model can predict a missing word in an incomplete sentence28. Similarly, in the 4D STEM example in Fig. 1b, if one can learn the 2D language, as it were, of the g (...truncated)