Statistics Theory
See recent articles
Showing new listings for Monday, 16 November 2026
- [1] arXiv:2611.17744 [pdf, html, other]
-
Title: Minimaxity in Robust Nonparametric Regression: from Heavy-Tailed to Sub-Gaussian AssumptionsComments: 50 pagesSubjects: Statistics Theory (math.ST)
We study nonparametric regression over star-shaped function classes under strong adversarial contamination. The clean (original) observations $(\tilde{X} _{i},\tilde{Y} _{i})_{i=1} ^{n}$ are i.i.d. and satisfy $\tilde{Y} _{i}=f ^{*}(\tilde{X} _{i})+\xi _{i}$, where $f ^{*}\in \mathcal{F}$, $\tilde{X}$ is distributed as $\mathbb{P} _{X}$, and the noise is conditionally centered. An adversary may inspect the clean sample and replace at most $\lfloor \epsilon n\rfloor$ observation pairs. We assume that $\mathcal{F}\subset L _{2}(\mathbb{P} _{X})$ is star-shaped with finite diameter $d$.
Under a Bernstein-type moment condition on function increments and a bounded conditional second moment for the noise, we establish the matching lower and upper bounds of order $\left((\eta ^{*}) ^{2} \vee \epsilon \right)\wedge d ^{2}$, uniformly over admissible star-shaped function classes, therefore characterizing the minimax rate of estimation error, where \begin{equation*}
\eta ^{*}:=\sup\left\{\eta >0: n\eta^2\leq\log \mathcal M _{\mathcal{F}}^{\text{loc}}(\eta,c)\right\}. \end{equation*} Here $\log \mathcal{M} _{\mathcal{F}} ^{\text{loc}}(\eta,c)$ denotes the local metric entropy of $\mathcal{F}$ in $L _2(\mathbb{P} _{X})$, and $c$ is a sufficiently large constant. Under a uniform relative sub-Gaussian condition on function increments and noise, the lower bound is adjusted to $\left((\eta ^{*})^{2}\vee \epsilon ^{2}\right)\wedge d ^{2}$ while the upper bound improves to $\left((\eta ^{*})^{2}\vee \epsilon ^{2}\log ^{2}\left(\frac{e}{\epsilon }\right)\right)\wedge d ^{2}$, yielding minimax optimality up to logarithmic factors. - [2] arXiv:2611.17923 [pdf, html, other]
-
Title: High-Dimensional Gaussian and Bootstrap Approximation: Near $1/\sqrt{n}$ Rates with General CovarianceComments: 42 pagesSubjects: Statistics Theory (math.ST); Probability (math.PR)
We consider Berry-Esseen bounds over rectangles for sums of $n$ random vectors in $\mathbb{R}^p$ that are i.i.d. A longstanding open question in high-dimensional probability and statistics is whether bounds of order $1/\sqrt n$ are attainable up to factors of $\log(pn)$ when the population covariance matrix is unrestricted. We answer this question affirmatively, provided that the random vectors have sub-Gaussian entries with non-zero variance. In addition, we provide analogous results for bootstrap approximation and Gaussian comparison, which make use of a special discrepancy measure between covariance matrices that enables improvement with respect to prior work. Taking an approach based on interpolation, the novel aspects of the proofs deal with bounding the interpolation derivatives in a way that is not sensitive to the covariance structure.
- [3] arXiv:2611.17934 [pdf, html, other]
-
Title: Statistical guarantees for signature kernel ridge regressionComments: Keywords: Path space regression, Signature transform, Rough paths, Functional data analysis, Kernel methodsSubjects: Statistics Theory (math.ST); Probability (math.PR)
Path signatures are a standard tool for learning functionals on path space. The main underpinnings of their application are universal approximation results. For a wide range of function classes on path space, these results state that any function in such a class can be approximated arbitrarily well by a linear functional of the path signature. Hence, linear functionals of the signature are in principle expressive enough for learning functions on path space. Nonetheless, these results do not provide insights into the statistical question of how much data is needed to identify a suitable signature functional. In this work, we study this question in the framework of kernel ridge regression with the signature kernel. The target is the conditional expectation $\E[Y|X]$, where $X$ is the observed path and $Y$ is a real-valued response. Under different assumptions on the moments of the signature kernel and the response, we provide finite-sample and consistency guarantees for the method. We supplement our theoretical analysis with numerical experiments that illustrate our results.
- [4] arXiv:2611.18141 [pdf, html, other]
-
Title: Characterizing M-estimators using canonical divergences from information geometryComments: 30 pages, 1 figure. Supplementary material includedSubjects: Statistics Theory (math.ST); Methodology (stat.ME)
On Euclidean space, the losses that produce consistent M-estimators of a functional are its strictly consistent scoring functions (Dimitriadis, Fissler and Ziegel, 2024), and Osband's principle characterizes these through identification functions, which define method-of-moments estimators. We extend this characterization to the probability simplex. We define the residual, which on Euclidean space is a subtraction, as the inverse exponential map of an affine connection. It identifies the exponential barycenter of the connection: the arithmetic, normalized geometric and Fréchet means for the mixture, exponential and Levi-Civita connections. Its integral along geodesics is the canonical divergence of Ay and Amari (2015). A geometric form of Osband's principle decides which losses each connection admits. For the flat connections every strictly convex generator yields a consistent Bregman loss. For the curved $\alpha$-connections, half the squared Fisher-Rao distance is (up to scale) the only consistent loss for the Fréchet mean, and for $\alpha \notin \{-1, 0, 1\}$ no such loss exists and only the method of moments remains. The residual is integrable only at $\alpha = 0$, sharpening the integrability claim of Ay and Amari. The characterization yields the estimating equations of regression on the simplex and the calibration decomposition of compositional forecasts.
- [5] arXiv:2611.18225 [pdf, html, other]
-
Title: Causal and conditional tail coefficients: asymptotics, inference and graph recoverySubjects: Statistics Theory (math.ST); Methodology (stat.ME)
The causal tail coefficient (CTC) measures the influence of an extreme value of one variable on another variable and has been proposed as a tool for causal discovery in heavy-tailed models. Despite its popularity, the asymptotic behavior of its empirical counterpart has not been well understood. We develop an asymptotic theory for the rank-based CTC estimator with an intermediate threshold under nonparametric conditions that are satisfied by all standard copula models. A key observation is that the CTC is determined by the mean of a conditional tail limit distribution, which clarifies its interpretation and determines the asymptotic variance of the estimator. We establish joint asymptotic normality for the entire CTC matrix and prove consistency of a multiplier bootstrap for simultaneous inference. For heavy-tailed linear and recursive max-linear structural equation models, we show that the CTC equals one if and only if the first variable is an ancestor of the second, which leads to confidence graphs for the ancestral graph with asymptotic coverage guarantees. All results extend to general conditional tail coefficients, defined as further functionals of the conditional tail limit distribution. A simulation study and an application to dietary intake data illustrate the methods.
- [6] arXiv:2611.18348 [pdf, html, other]
-
Title: Quantifying Dependence Effects on $k$-out-of-$n$ Reliability Relative to the Independent BenchmarkSubjects: Statistics Theory (math.ST); Probability (math.PR); Methodology (stat.ME)
Understanding how dependence among component lifetimes affects the reliability of redundant systems is a central problem in reliability engineering. In this paper, we develop a framework for quantifying the impact of multivariate dependence on $k$-out-of-$n$ system reliability relative to the independent benchmark. Using an orthogonal Legendre expansion of the copula density, we derive an exact decomposition that separates the dependence structure from the system architecture. This representation identifies how individual dependence contributions are transmitted to system reliability through explicit structural weights. An application to multivariate Farlie-Gumbel-Morgenstern copulas demonstrates that higher-order dependence can produce distinct reliability effects even when all bivariate copulas remain unchanged. Our findings provide a systematic approach to quantifying and interpreting the effects of multivariate dependence on the reliability of redundant systems.
New submissions (showing 6 of 6 entries)
- [7] arXiv:2611.17609 (cross-list from stat.ME) [pdf, html, other]
-
Title: Homogeneous pursuit in $β$ modelSubjects: Methodology (stat.ME); Statistics Theory (math.ST); Machine Learning (stat.ML)
Data in forms of networks has been encountered in various prevalent fields at the arrival of big data era. To unveil and recover the true model with core units with numerous links, the $\beta$-model under sparsity constraint is proposed to meet the expectation of statisticians and engineers. Not only does this network model inherit spectacular property from the Erdős-Rényi model, but also exhibits degree heterogeneity through tuning sparsity parameter $s$. In this paper, we propose Homogeneous Pursuit (HP), an $\ell_0$-constrained maximum likelihood estimation method implemented through a primal-dual algorithm. To ease the harshness of computation, we find the surrogate function of the corresponding objective function to derive Majorization Minimization algorithm through the primal-dual process. Moreover, after meticulous evaluation, the algorithm brought about captures the property of fine convergence rate in a contraction mapping way. Under some mild conditions, the proposed generalized information criterion (GIC$^*$) for selecting the sparsity level is consistent to find the truth. Combining HP with GIC$^*$ gives the adaptive procedure HP-GIC$^*$; HP-BIC denotes the same estimation method with sparsity selected by BIC. Finally, simulation studies show that GIC$^*$ improves support recovery relative to BIC in the reported comparisons, while HP achieves lower computational cost while maintaining estimation accuracy comparable to the sparse benchmark methods. A microfinance application illustrates the use of HP-GIC$^*$ for studying the association between network prominence and participation, with improved fit in the reported village-level regressions.
- [8] arXiv:2611.18095 (cross-list from stat.ML) [pdf, html, other]
-
Title: Training-point consistency of kernel ridge regression with varying correlation scalesSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
The scale parameter in kernel ridge regression plays a pivotal role in modeling the correlation structure of data. In modern applications, sampling density and the underlying correlation range often change simultaneously, making a fixed-scale assumption highly restrictive. However, establishing asymptotic theories under varying scale parameters is fundamentally challenging. To address this gap, this paper develops a unified theoretical framework for kernel ridge regression under varying scale parameters. We establish that, on uniform tensor grids with product exponential kernels, the consistency of the training-point risk holds if and only if the effective local sample size diverges to infinity. This sharp criterion universally covers all regimes along arbitrary deterministic scale sequences, seamlessly unifying the long-range, fixed-scale, and short-range. Furthermore, we provide an explicit, signal-independent construction of a deterministic regularization penalty that guarantees this consistency. Numerical experiments illustrate the criterion and the corresponding penalty across all the three regimes.
Cross submissions (showing 2 of 2 entries)
- [9] arXiv:2303.02510 (replaced) [pdf, html, other]
-
Title: Testing the equality of the copulas of two multivariate populations with Bernstein polynomialsSubjects: Statistics Theory (math.ST)
The empirical Bernstein copula process is utilized to conduct tests for the equal- ity of the dependence structures between two samples. To this end, three tests statistics are proposed, and their asymptotic properties are established under mild regularity conditions. Furthermore, in order to obtain a valid p-value, sub- sampling and multiplier bootstrap methods are introduced and investigated for the Bernstein based-test statistics. Through a simulation study, it is shown that Bernstein-based tests significantly outperform tests based on the empirical copula.
- [10] arXiv:2409.06275 (replaced) [pdf, other]
-
Title: On Sparsity and Sub-Gaussianity in the Johnson-Lindenstrauss LemmaAur{é}lien Garivier (UMPA-ENSL, MC2), Emmanuel Pilliat (UMPA-ENSL)Journal-ref: Transactions on Machine Learning Research 2025Subjects: Statistics Theory (math.ST)
We provide a simple proof of the Johnson-Lindenstrauss lemma for sub-Gaussian variables. We extend the analysis to identify how sparse projections can be, and what the cost of sparsity is on the target dimension. The Johnson-Lindenstrauss lemma is the theoretical core of the dimensionality reduction methods based on random projections. While its original formulation involves matrices with Gaussian entries, the computational cost of random projections can be drastically reduced by the use of simpler variables, especially if they vanish with a high probability. In this paper, we propose a simple and elementary analysis of random projections under classical assumptions that emphasizes the key role of sub-Gaussianity. Furthermore, we show how to extend it to sparse projections, emphasizing the limits induced by the sparsity of the data itself.
- [11] arXiv:2512.01817 (replaced) [pdf, html, other]
-
Title: Self-Normalized Bounds for Time Series with Sample Variance Only and Their Application to ForecastingSubjects: Statistics Theory (math.ST)
Evaluating a forecasting rule on a time series requires a risk bound that can be computed from the observed losses. Such generalization bounds rest on empirical Bernstein (EB) and self-normalized inequalities, which replace the unknown variance by a sample counterpart. These tools, however, have been developed almost exclusively for independent observations and martingale differences, and, for dependent data, the natural setting of forecasting, direct results are scarce. They require $\beta$-mixing and knowledge of the mixing coefficients, and their leading terms are not sharp. This paper fills this gap. We first establish, for martingale differences, an EB inequality in closed form that uses only the realized quadratic variation, holds with probability at least $1-\alpha$ for every sample size, and is sharp under heteroscedasticity, whereas the sharpness of existing EB inequalities is established for a constant conditional variance. Building on this, we derive EB and self-normalized inequalities for the marginal mean of $\tilde{\phi}$- and $\tau$-mixing processes, a class that contains processes that are not even strongly mixing, such as autoregressions with Bernoulli innovations. The radii use nothing but a block sample variance: no variance proxy has to be designed or tuned, and, for $\tau$-mixing processes, no dependence coefficient enters the radius. Under summability-type conditions on the dependence coefficients, their leading terms carry the constant of the oracle Bernstein inequality with the long-run variance, so, to our knowledge, these are the first sharp self-normalized inequalities under non-strong mixing conditions. As an application, we obtain a sharp upper bound on the risk of a time-series forecasting rule that is computed from its observed losses with a single block length and contains no dependence coefficient.
- [12] arXiv:2605.03300 (replaced) [pdf, html, other]
-
Title: Smoothed estimation of Wasserstein barycentersSubjects: Statistics Theory (math.ST)
This paper studies the statistical estimation of exact Wasserstein barycenters. Existing non-asymptotic results for empirical barycenters exhibit a severe curse of dimensionality. Motivated by the semi-dual formulation of the barycenter problem and its associated Sobolev optimization geometry, we develop a smoothness-aware approach that combines density estimation with Sobolev geometric structure to estimate the population barycenter. We establish nonparametric convergence rates for estimating both the barycenter functional and its minimizer, demonstrating how smoothness can substantially improve statistical performance.
- [13] arXiv:2609.30106 (replaced) [pdf, html, other]
-
Title: Stopping models closed under pgf composition, and the stability of randomly stopped model extensionsComments: 45 pagesSubjects: Statistics Theory (math.ST); Probability (math.PR)
Statistical model transformations based on randomly stopped sums, maxima, and minima are widely used to extend statistical models by introducing an additional parameter. We characterize the finite-dimensional positive stopping models for which these randomly stopped sum and extreme transformations act as statistically stable (idempotent) model extensions. Stability requires the underlying stopping model to be closed under pgf composition. We prove that any finite-dimensional, connected, positive stopping model closed under pgf composition necessarily forms a family of random variables whose pgfs commute. Using the associated Koenigs function, we establish that these models form a statistical manifold admitting a global, one-dimensional parametrization $\theta = \Pr(N=1) \in (0, \theta_*]$, in which the probability mass at $i$ is a polynomial in $\theta$ of degree at most $i$. Within this class, we further establish a duality between those models containing the identity variable (the ones yielding statistically stable extensions) and the set of all probability distributions supported on the positive integers. By providing numerous examples of stopping models closed under pgf composition, we clarify the assertion that, under certain conditions, the only stopping model that is so is geometric.
- [14] arXiv:2611.07219 (replaced) [pdf, html, other]
-
Title: Neyman Orthogonalized Specification Tests for Conditional Tail Index ModelsSubjects: Statistics Theory (math.ST); Methodology (stat.ME)
We develop specification tests for parametric conditional tail index models of heavy-tailed distributions. The tests are based on a projected empirical process that removes the first-order effect of estimating the null parameter. Within the conditional Hall class, we identify conditions under which projection also eliminates the leading Pareto approximation bias in the test process, permitting lower thresholds while preserving asymptotically correct size. We establish the validity of the tests via multiplier bootstrap and characterize power against fixed and local alternatives. For log-linear models, bootstrap validity extends to settings in which covariates of observations above a common threshold concentrate and the smallest eigenvalue of the information matrix tends to zero. We further propose a novel threshold selection rule based on an estimate of the Pareto approximation bias after projection. The rule uses the estimated bias to determine whether a reference threshold can be lowered to retain more observations, with potential gains in power. The finite sample performance of the tests is examined through simulations and a real data application.
- [15] arXiv:2507.18520 (replaced) [pdf, html, other]
-
Title: Recovering the Euclidean Geometry of High-Dimensional Data Under Heterogeneous NoiseSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
Pairwise Euclidean distances provide a simple geometric representation of high-dimensional data and underlie many data analysis methods, including clustering, visualization, and manifold learning. However, in real-world applications, these distances can be severely corrupted by heterogeneous noise, where different data points are subject to different, and potentially large, noise levels. Such noise distorts the distances inhomogeneously, leading to a misrepresentation of the underlying data geometry. In this work, we ask whether the underlying clean Euclidean geometry can nevertheless be recovered from the corrupted pairwise distances. We answer this question affirmatively, showing that, perhaps surprisingly, such recovery is possible across a broad class of high-dimensional settings with underlying low-dimensional geometry, even when the noise magnitudes vary substantially across observations. We develop a principled approach for recovering the clean pairwise distances, without prior knowledge of either the clean data structure or the noise distribution, and establish probabilistic recovery guarantees. In particular, the recovery error converges to zero at polynomial rates as the feature dimension and dataset size increase simultaneously. Experiments on synthetic datasets demonstrate accurate recovery of the Euclidean geometry in challenging regimes and improved robustness of downstream methods that rely on pairwise distances. When applied to real single-cell RNA sequencing data, our method reliably recovers local neighborhood structure, enabling accurate nearest-neighbor identification used in many downstream analyses.
- [16] arXiv:2510.07559 (replaced) [pdf, html, other]
-
Title: General coupling-based f-divergence diagnostics for Markov chain Monte CarloComments: Significantly revised manuscript including new nonasymptotic boundsSubjects: Computation (stat.CO); Probability (math.PR); Statistics Theory (math.ST); Methodology (stat.ME)
A long-standing gap exists between the theoretical analysis of Markov chain Monte Carlo convergence, which is often based on statistical divergences, and the diagnostics used in practice. We introduce the first general convergence diagnostics for Markov chain Monte Carlo based on a broad class of f-divergence, allowing users to directly monitor, among others, the Kullback-Leibler and the chi-squared divergences as well as the total variation distance. Our approach rests on a coupling-based 'weight harmonization' scheme that produces direct, computable, and consistent importance weights for interacting Markov chains with respect to their target distribution. Beyond their use as convergence diagnostics, these weights give unbiased estimates of the Radon-Nikodym derivative between the un-normalized target and the distribution of the current iterate, a richer object than the convergence bounds alone, with natural applications to importance-weighted inference. We show how such weightings can provide asymptotic upper bounds to any f-divergence, prove that these bounds converge to zero as the chains approach stationarity, and demonstrate that, while more conservative than existing coupling-based total variation estimators, our method remains a practical and broadly applicable diagnostic tool.
- [17] arXiv:2606.10944 (replaced) [pdf, html, other]
-
Title: Express Language ModelingSubjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Statistics Theory (math.ST); Methodology (stat.ME); Machine Learning (stat.ML)
We introduce a new tool, Express, for converting a non-causal attention approximation into a causal approximation with matching approximation guarantees. When combined with the state-of-the-art Thinformer approximation, Express improves upon the best known causal attention guarantees, delivering $\log^{3/2}(n)/s$ approximation error with only $O(s)$ memory and $O(s^2 \log^2(n))$ compression overhead for a sequence of length $n$. We pair these developments with an efficient I/O-aware Triton implementation, demonstrate substantial speedups over FlashAttention 2, and use Express to overcome four resource bottlenecks in the language modeling pipeline: long-context prefill, KV cache compression, long-form memory-constrained decoding, and long-form compute-constrained decoding.
- [18] arXiv:2608.05434 (replaced) [pdf, html, other]
-
Title: Risk-Aware Quantile Learning for Personalized Dynamic Treatment RegimesComments: An earlier version of this work was accepted at STAI-X 2026Subjects: Methodology (stat.ME); Statistics Theory (math.ST); Machine Learning (stat.ML)
Sequential clinical decision-making often involves more than maximizing average efficacy. Clinicians may need to simultaneously optimize clinically relevant tails of the outcome distribution, control treatment-related risk, and choose among multiple treatment options. Existing quantile dynamic treatment regime (DTR) methods capture distributional features of treatment outcomes but remain largely restricted to efficacy-only objectives and binary treatments. To address these limitations, we propose Risk-Aware Quantile Dynamic Treatment Regimes (RQDTR), a unified framework that optimizes a prespecified quantile of the cumulative potential outcome while explicitly incorporating treatment-related risk. We also develop an angle-based formulation for jointly learning decision rules across multiple treatment categories. Our framework includes three interpretable subclasses: efficacy-only quantile learning, constraint-based learning with population-level risk control, and utility-based learning through a composite benefit-risk utility. Theoretically, we establish identification and oracle equivalence, surrogate and estimator consistency under increasing quadratic penalties, asymptotic risk feasibility, and finite-sample performance error bounds. Extensive simulation studies and applications to All of Us major depressive disorder and MIMIC-III sepsis data demonstrate that RQDTR improves tail-oriented efficacy and achieves more favorable benefit-risk trade-offs than existing quantile DTR methods.
- [19] arXiv:2610.28301 (replaced) [pdf, html, other]
-
Title: Gaussian Fluctuations of the Gradient of the Fréchet Function and Nonstandard Asymptotics of Sample Fréchet MeansComments: 25 pagesSubjects: Probability (math.PR); Statistics Theory (math.ST)
We establish a central limit theorem for the gradient of the Fréchet function in local Euclidean coordinates on a metric space. Assuming a unique population mean, a consistent measurable sequence of sample means, semiconcavity or semiconvexity and moment bounds for the squared distance functions, and local $C^{1,\alpha}$ regularity of the population Fréchet function, we prove an asymptotic linear representation for its gradient at the sample mean. The leading term is the negative empirical average of the gradients of the squared distance functions at the population mean; these gradients exist almost surely under our assumptions. On complete connected Riemannian manifolds, uniqueness of the population mean, existence of a $(2+\eta)$-moment, and local $C^{1,\alpha}$ regularity suffice, and the leading term is expressed through the logarithm map. We also give a sufficient condition for the regularity of the population Fréchet function through probability bounds for neighborhoods of cut loci. We then show how the local behavior of the population Fréchet function transforms these Gaussian fluctuations into limit distributions for suitably rescaled sample Fréchet means, allowing for anisotropic scaling and non-Gaussian limits. A flat-torus example exhibits anisotropic scaling, and two examples on the circle illustrate logarithmic smeariness and the failure of any deterministic linear scaling to yield a nondegenerate limit, respectively.