Asymptotics and concentration bounds for bilinear forms of spectral projectors of sample covariance

Let X,X1,. ., Xn be i.i.d. Gaussian random variables with zero mean and covariance operator ∑ = E(X ⊗ X) taking values in a separable Hilbert space H. Let r(∑) := tr(∑) ||∑||∞ be the effective rank of ∑, tr(∑) being the trace of ∑ and ||∑||∞ being its operator norm. Let ∑ n := n -1 ∑n j=1 (Xj ⊗Xj ) be the sample (empirical) covariance operator based on (X1,. ., Xn). The paper deals with a problem of estimation of spectral projectors of the covariance operator ∑ by their empirical counterparts, the spectral projectors of ∑ n (empirical spectral projectors). The focus is on the problems where both the sample size n and the effective rank r(∑) are large. This framework includes and generalizes well known high-dimensional spiked covariance models. Given a spectral projector Pr corresponding to an eigenvalue μr of covariance operator ∑ and its empirical counterpart Pr, we derive sharp concentration bounds for bilinear forms of empirical spectral projector Pr in terms of sample size n and effective dimension r(∑). Building upon these concentration bounds, we prove the asymptotic normality of bilinear forms of random operators Pr -E Pr under the assumptions that n→∞and r(∑) = o(n). In a special case of eigenvalues of multiplicity one, these results are rephrased as concentration bounds and asymptotic normality for linear forms of empirical eigenvectors. Other results include bounds on the bias E Pr - Pr and a method of bias reduction as well as a discussion of possible applications to statistical inference in high-dimensional Principal Component Analysis.