Pseudospectral Shattering, the Sign Function, and Diagonalization in Nearly Matrix Multiplication Time

Foundations of Computational Mathematics, Aug 2022

We exhibit a randomized algorithm which, given a square matrix $$A\in \mathbb {C}^{n\times n}$$ with $$\Vert A\Vert \le 1$$ and $$\delta >0$$ , computes with high probability an invertible V and diagonal D such that $$ \Vert A-VDV^{-1}\Vert \le \delta $$ using $$O(T_\mathsf {MM}(n)\log ^2(n/\delta ))$$ arithmetic operations, in finite arithmetic with $$O(\log ^4(n/\delta )\log n)$$ bits of precision. The computed similarity V additionally satisfies $$\Vert V\Vert \Vert V^{-1}\Vert \le O(n^{2.5}/\delta )$$ . Here $$T_\mathsf {MM}(n)$$ is the number of arithmetic operations required to multiply two $$n\times n$$ complex matrices numerically stably, known to satisfy $$T_\mathsf {MM}(n)=O(n^{\omega +\eta })$$ for every $$\eta >0$$ where $$\omega $$ is the exponent of matrix multiplication (Demmel et al. in Numer Math 108(1):59–91, 2007). The algorithm is a variant of the spectral bisection algorithm in numerical linear algebra (Beavers Jr. and Denman in Numer Math 21(1-2):143–169, 1974) with a crucial Gaussian perturbation preprocessing step. Our result significantly improves the previously best-known provable running times of $$O(n^{10}/\delta ^2)$$ arithmetic operations for diagonalization of general matrices (Armentano et al. in J Eur Math Soc 20(6):1375–1437, 2018) and (with regard to the dependence on n) $$O(n^3)$$ arithmetic operations for Hermitian matrices (Dekker and Traub in Linear Algebra Appl 4:137–154, 1971). It is the first algorithm to achieve nearly matrix multiplication time for diagonalization in any model of computation (real arithmetic, rational arithmetic, or finite arithmetic), thereby matching the complexity of other dense linear algebra operations such as inversion and QR factorization up to polylogarithmic factors. The proof rests on two new ingredients. (1) We show that adding a small complex Gaussian perturbation to any matrix splits its pseudospectrum into n small well-separated components. In particular, this implies that the eigenvalues of the perturbed matrix have a large minimum gap, a property of independent interest in random matrix theory. (2) We give a rigorous analysis of Roberts’ Newton iteration method (Roberts in Int J Control 32(4):677–687, 1980) for computing the sign function of a matrix in finite arithmetic, itself an open problem in numerical analysis since at least 1986.

Article PDF cannot be displayed. You can download it here:

https://link.springer.com/content/pdf/10.1007/s10208-022-09577-5.pdf

Pseudospectral Shattering, the Sign Function, and Diagonalization in Nearly Matrix Multiplication Time

Foundations of Computational Mathematics https://doi.org/10.1007/s10208-022-09577-5 Pseudospectral Shattering, the Sign Function, and Diagonalization in Nearly Matrix Multiplication Time Jess Banks1 · Jorge Garza-Vargas1 · Archit Kulkarni1 · Nikhil Srivastava1 Received: 11 February 2021 / Revised: 1 February 2022 / Accepted: 4 March 2022 © The Author(s) 2022 Abstract We exhibit a randomized algorithm which, given a square matrix A ∈ Cn×n with A ≤ 1 and δ > 0, computes with high probability an invertible V and diagonal D such that A − V DV −1  ≤ δ using O(TMM (n) log2 (n/δ)) arithmetic operations, in finite arithmetic with O(log4 (n/δ) log n) bits of precision. The computed similarity V additionally satisfies V V −1  ≤ O(n 2.5 /δ). Here TMM (n) is the number of arithmetic operations required to multiply two n × n complex matrices numerically stably, known to satisfy TMM (n) = O(n ω+η ) for every η > 0 where ω is the exponent of matrix multiplication (Demmel et al. in Numer Math 108(1):59–91, 2007). The algorithm is a variant of the spectral bisection algorithm in numerical linear algebra (Beavers Jr. and Denman in Numer Math 21(1-2):143–169, 1974) with a crucial Gaussian perturbation preprocessing step. Our result significantly improves the previously best-known provable running times of O(n 10 /δ 2 ) arithmetic operations for diagonalization of general matrices (Armentano et al. in J Eur Math Soc 20(6):1375–1437, 2018) and (with regard to the dependence on n) O(n 3 ) arithmetic operations for Hermitian matrices (Dekker and Traub in Linear Algebra Appl 4:137–154, 1971). It is the first algorithm to achieve nearly matrix multiplication time for diagonalization in Communicated by Peter Bürgisser. Jess Banks supported by the NSF Graduate Research Fellowship Program under Grant DGE-1752814. Nikhil Srivastava supported by NSF Grant CCF-1553751. B Nikhil Srivastava Jess Banks Jorge Garza-Vargas Archit Kulkarni 1 Department of Mathematics, UC Berkeley, Berkeley 94720, CA, USA 123 Foundations of Computational Mathematics any model of computation (real arithmetic, rational arithmetic, or finite arithmetic), thereby matching the complexity of other dense linear algebra operations such as inversion and Q R factorization up to polylogarithmic factors. The proof rests on two new ingredients. (1) We show that adding a small complex Gaussian perturbation to any matrix splits its pseudospectrum into n small well-separated components. In particular, this implies that the eigenvalues of the perturbed matrix have a large minimum gap, a property of independent interest in random matrix theory. (2) We give a rigorous analysis of Roberts’ Newton iteration method (Roberts in Int J Control 32(4):677–687, 1980) for computing the sign function of a matrix in finite arithmetic, itself an open problem in numerical analysis since at least 1986. Keywords Linear algebra · Random matrix theory · Numerical analysis · Computational complexity Mathematics Subject Classification 60B20 · 65F15 · 68Q25 Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Accuracy and Conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.2 Models of Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Results and Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Spectral Projectors and Holomorphic Functional Calculus . . . . . . . . . . . . . . . . . . . . . 2.2 Pseudospectrum and Spectral Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Finite-Precision Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Sampling Gaussians in Finite Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Black-box Error Assumptions for Multiplication, Inversion, and QR . . . . . . . . . . . . . . . 3 Pseudospectral Shattering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Smoothed Analysis of Gap and Eigenvector Condition Number . . . . . . . . . . . . . . . . . . 3.2 Shattering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Matrix Sign Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Circles of Apollonius . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Exact Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Finite Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Spectral Bisection Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Proof of Theorem 5.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Conclusion and Open Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Deferred Proofs from Sect. 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B Analysis of SPLIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C Analysis of DEFLATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.1 Smallest Singular Value of the Corner of a Haar Unitary . . . . . . . . . . . . . . . . . . . . . . C.2 Sampling Haar Unitaries in Finite Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.3 Preliminaries of RURV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.4 Exact Arithmetic Analysis of DEFLATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.5 Finite Arithmetic Analysis of DEFLATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D Alternate Proofs of Shattering and Davies’ Conjecture . . . . . . . . . . . . . . . . . . . . . . . . . D.1 Davies’ conjecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2 Smoothed analysis of gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Foundations of Computational Mathematics 1 Introduction We study the algorithmic problem of approximately finding all of the eigenvalues and eigenvectors of a given arbitrary n × n complex matrix. While this problem is quite well-understood in the special case of Hermitian matrices (see, e.g., [52]), the general non-Hermitian case has remained mysterious fr (...truncated)


This is a preview of a remote PDF: https://link.springer.com/content/pdf/10.1007/s10208-022-09577-5.pdf
Article home page: https://link.springer.com/article/10.1007/s10208-022-09577-5

Banks, Jess, Garza-Vargas, Jorge, Kulkarni, Archit, Srivastava, Nikhil. Pseudospectral Shattering, the Sign Function, and Diagonalization in Nearly Matrix Multiplication Time, Foundations of Computational Mathematics, 2022, pp. 1-89, DOI: 10.1007/s10208-022-09577-5