Statistics
See recent articles
Showing new listings for Monday, 16 November 2026
- [1] arXiv:2611.17390 [pdf, html, other]
-
Title: Enumerating Posterior-Supported Priors: First-Order Error of Finite-Integral Cross-Categorical Measurement MorphismsSubjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
In generative modeling, standard measure theory adopts the a.e. convention, which legitimately ignores any error component carried by a null-support set: a single null element can hide out-of-domain error inside null support and never be explicitly accounted for. The contribution of this paper is not a new distributional distance, but rather a verifiable structure that writes down which underlying predicate supports a high-level object, when it is invertible, and at which layer the error resides, from which we propose the enumerating posterior-supported prior framework. The framework operates on a three-layer categorical chain Quot->MorTop->Set: the first two layers posit no point elements and take binary-predicate quotient partitions as objects; the replica-index counting measure successively discriminates one-to-many collections into injections, lifting object collection from surjection to a quotient-layer bijection; the mutual exclusion of preimages in the shared measure space is resolved by an invertible MPT. Four results consolidate into Thm. 7: binary quotient-partition meta-categorical objects support the set categorical functional along categorical support via measure descent--recovery, enumerating the morphism itself rather than its preimage and object; their multiplicity is normalized through replica-index injection and accounted as an explicitly first-order error. The contribution of this paper is a transformation from nontrivial-input construction -> engineering-computable first-order output: the binary quotient-partition meta-categorical objects, supported via categorical support measure descent--recovery, constitute a substantive nontrivial input---precisely because they lift the out-of-domain error that hid inside null support into a first-order falsifiable coefficient mu_err, a finitely obtainable finite integral first-order error.
- [2] arXiv:2611.17448 [pdf, html, other]
-
Title: Envelope Model for GTEx DataSubjects: Applications (stat.AP); Methodology (stat.ME)
This article studies the envelope model for enhanced gene expression analysis using GTEx data. The study compares this advanced statistical approach with standard multivariate linear regression, demonstrating improved capacity for identifying significant genes and enhanced estimation efficiency. By partitioning response variation into material and immaterial components, the envelope model reduces estimation noise, leading to stronger statistical signals and smaller standard errors. These findings, as demonstrated in our tissue-specific and gene-group anal- yses, show that the envelope model can pick up on important biological signals that traditional methods tend to miss. This makes it a useful tool for genomic studies and GTEx data analysis.
- [3] arXiv:2611.17503 [pdf, other]
-
Title: Multifidelity Operator Inference for Dynamical SystemsComments: 19 pages, AIAA SciTech Forum 2027 (Uncertainty Quantification Conference)Subjects: Machine Learning (stat.ML); Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG); Numerical Analysis (math.NA)
This paper presents a multifidelity (MF) formulation of Operator Inference (OpInf) for learning non-intrusive, physics-based reduced-order models (ROMs) of nonlinear dynamical systems. OpInf learns reduced operators from projections of snapshot data by solving a least squares regression problem. However, generating high-fidelity snapshots can be computationally expensive, limiting parameter coverage and the temporal extent of training trajectories for many aerospace applications, which can lead to inaccurate or unstable ROM predictions. This work proposes MF-OpInf, which combines limited high-fidelity snapshots with more plentiful low-fidelity snapshots through a control-variates-inspired MF least squares formulation to learn a single set of reduced operators across multiple trajectories. Low-fidelity trajectories provide additional parameter coverage and extend beyond the high-fidelity training time horizon. The low-fidelity snapshots are also used to construct an improved reduced basis through weighted proper orthogonal decomposition. We demonstrate MF-OpInf on a nonlinear reaction-diffusion problem with parametrized sinusoidal forcing and a two-dimensional incompressible flow past a cylinder with a parametrized inlet pulse. In these experiments, under an equal training simulation budget, MF-OpInf reduces the mean relative prediction error at unseen parameters by up to $87\%$ for the reaction-diffusion problem and $28\%$ for the cylinder flow problem over the test time regime, which extends beyond the high-fidelity training time horizon. These results demonstrate that MF-OpInf can improve temporal extrapolation and prediction at unseen parameters without increasing the training simulation budget.
- [4] arXiv:2611.17507 [pdf, html, other]
-
Title: Parametric Modal SARIMA Models with Skewed DistributionsComments: 21 pages, 5 figures, 11 tables. Supplementary tables and figures are provided as an ancillary fileSubjects: Methodology (stat.ME)
Seasonal autoregressive integrated moving-average (SARIMA) models typically target the conditional mean of a time series under a Gaussian error assumption. This approach may be inadequate when the innovations are asymmetric because the conditional mean no longer represents the most likely outcome. We propose the Modal SARIMA$(p,d,q)\times(P,D,Q)_s$ model, which links the conditional mode, rather than the conditional mean, to a multiplicative seasonal ARIMA structure. The innovations follow the Fernandez-Steel class of skewed distributions (SKD), whose location parameter is the mode, and the non-seasonal Modal ARIMA$(p,d,q)$ model is the special case without seasonal terms. We give the conditional log-likelihood for the Skew-Normal, Skewed Student-$t$, and Skewed Laplace members, establish the stationarity and invertibility conditions on the product polynomials, show that the recursive modal forecast is the joint mode of the future trajectory, and characterize the correction that yields the marginal $h$-step conditional mode. Monte Carlo experiments show that conditional maximum likelihood and its Wald inference are reliable when the seasonal factors are well separated, that the test of symmetry holds its level from about 200 observations, and that the modal estimator of the dynamic coefficients is as efficient as Gaussian least squares under symmetry and 1.2 to 2.6 times more efficient under asymmetry or heavy tails. We also show, analytically and by simulation, that nearly cancelling seasonal factors amplify the effect of the starting values, and that beyond one step the recursive forecast must be corrected to remain a forecast of the most probable value. In an application to the Canadian lynx trappings, a seasonal component for the ten-year cycle improves the non-seasonal fit for every member of the SKD family. The methods are implemented in the R package ModalForecast.
- [5] arXiv:2611.17518 [pdf, html, other]
-
Title: Bayesian Semiparametric Regression for Count Data Using Zero Altered Moment ModelsSubjects: Methodology (stat.ME)
We develop a Bayesian semiparametric regression framework for count data that can accommodate both underdispersion and overdispersion without specifying a full parametric probability model. The model is built from the first two conditional factorial moments of a zero altered distribution. Covariates may enter the conditional mean and the conditional dispersion separately, thereby allowing the dispersion to vary across observations. We turn the moment restrictions into an adjusted exponentially tilted empirical likelihood (AETEL) criterion and combine this criterion with Gaussian priors to form a Bayesian empirical-likelihood pseudo-posterior. The adjustment improves numerical feasibility and a two-block Metropolis delayed acceptance sampler allows for faster runs of the MCMC chains. This reduces unnecessary exact evaluations without changing the AETEL target. Using existing Bayesian exponentially tilted empirical likelihood (ETEL) theory, we state a first-order posterior normal approximation and verify the moment, smoothness, and Jacobian rank conditions needed for the asymptotic inference. Simulations under zero inflated Poisson and Conway Maxwell Poisson data exhibit small bias, accurate uncertainty estimates, and near nominal interval coverage for both overdispersed and underdispersed counts. Applications to various datasets illustrate how the Bayesian analysis can identify covariate dependent dispersion using finite sample posterior inference.
- [6] arXiv:2611.17523 [pdf, html, other]
-
Title: A Spatial Analysis of Proximity to the Built Environment and Wildfire Ignition Cause in the United StatesComments: 17 pages, 5 figures. Under review at the International Journal of Wildland FireSubjects: Applications (stat.AP)
Background. Human activities in areas with burnable land cover can lead to wildfire ignitions, and higher levels of human presence increase the likelihood of such fires. Building locations provide a widely available representation of human presence. Aims. This study examines how accurately proximity to buildings predicts wildfire ignition cause across different land-cover types. Methods. Supervised classification methods were applied to building, land-cover, and ignition data for the contiguous United States to estimate the conditional probability that an ignition is human-caused. Proximity to buildings was represented using either building density within surrounding zones or distance to nearby areas with high building density. Key results. Both methods, particularly the distance-based approach, showed that proximity to buildings is a strong predictor of ignition cause: approximately three quarters of both human- and lightning-caused ignitions were correctly classified. Retraining the models separately for individual states did not substantially improve classification performance relative to the national model. Conclusions. Building proximity is an effective predictor of wildfire ignition cause, and the relationship between ignition cause and building-related predictors is similar across many regions of the contiguous United States. Implications. Building-proximity variables can be incorporated into wildfire forecasting and risk assessment models to improve prediction of human-caused wildfire ignitions.
- [7] arXiv:2611.17528 [pdf, html, other]
-
Title: Regularized parametric modal regression for positive responses with high-dimensional covariatesComments: 15 pages, 7 tables. R package ModalNet: this https URLSubjects: Methodology (stat.ME)
Modal regression models the conditional mode rather than the conditional mean, which is attractive for positive and right-skewed responses whose mean is distorted by the upper tail. We propose a regularized version of parametric modal regression for positive responses with high-dimensional covariates, in which the density parameters of the Gamma, Weibull and Lognormal distributions are written as functions of the conditional mode and the log-likelihood is penalized by the Lasso, adaptive Lasso or elastic-net penalties. The estimates are computed by a proximal Newton algorithm, equivalently an iteratively reweighted penalized least-squares scheme, whose inner problem is solved exactly by the coordinate-descent routines of glmnet, and the regularization parameter is selected by the extended Bayesian information criterion. We show that the Lognormal estimator coincides with the Gaussian Lasso applied to the log-transformed response, that under constant dispersion the conditional mean and the conditional mode have the same slopes, and that the estimation error attains the rate $O(\sqrt{s}\lambda_n)$ with no smoothing bias. Simulation studies with up to $p = 1000$ covariates show that the method improves on the Lasso applied to the log-transformed response for Gamma and Weibull responses, and two applications illustrate its use. The method is implemented in the R package ModalNet.
- [8] arXiv:2611.17545 [pdf, html, other]
-
Title: Identifiability and Spectral Estimation of Mixed Memberships in Heterogeneous Multilayer NetworksComments: 80 pages, 5 figures, 4 tablesSubjects: Methodology (stat.ME)
Multilayer network data, arising from multiple sources, repeated measurements, or multi-view interactions, are increasingly common, highlighting the need for statistical methods that jointly leverage information across layers while allowing for layer heterogeneity and computational scalability. Here, we study a multilayer degree-corrected mixed membership model (ML-DCMM), in which nodes share a common mixed membership across layers while degree effects and block connectivity patterns vary by layer. This model encompasses several existing models for multilayer data while preserving identifiability and interpretation. We develop a spectral estimation procedure that integrates layer-wise embeddings with corner estimation for membership recovery. A further refinement that connects the resulting representation to a ParaTuck-2 tensor decomposition is also introduced. We establish theoretical guarantees for the proposed spectral method, showing that the probability of near-corner recovery increases with the number of layers, and we provide the convergence rate for the resulting membership estimator. We establish a minimax lower bound for membership estimation, which does not improve with increasing number of layers. Simulation studies show that ML-DCMM can outperform other approaches under strong layer heterogeneity. In an analysis of London bike-sharing networks, the methodology recovers interpretable memberships and achieves lower held-out edge prediction error under edge-sampling cross-validation than competing models.
- [9] arXiv:2611.17547 [pdf, html, other]
-
Title: Uncertainty-Aware Continual Learning for 2D XRD Interpretation Using Simulated and Experimental DataSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Autonomous experimentation systems require characterization models that can interpret heterogeneous measurements and maintain predictive performance as data accumulate. Developing such models involves addressing domain mismatch between data sources, overconfident predictions on unfamiliar inputs, and catastrophic forgetting during sequential updates. In this work, we present a proof-of-concept framework for uncertainty-aware continual interpretation of two-dimensional X-ray diffraction (XRD) data. The framework integrates established components: maximum mean discrepancy-based domain adaptation to reduce simulation-to-experiment discrepancies, a convolutional neural network with a spectral-normalized neural Gaussian process (CNN--SNGP) for predictive uncertainty estimation, and replay with knowledge distillation for sequential model updating. We evaluate the framework using simulated and previously-collected experimental data for crystal structure classification of single-phase body-centered cubic, face-centered cubic, and hexagonal close-packed patterns. Data are organized into predefined sequential blocks to assess adaptation and retention of previously learned knowledge. Within this setting, the results show accurate classification across simulated and experimental data, improved alignment between the two domains, and reduced forgetting compared with naïve fine-tuning. Predictive uncertainty is also evaluated on unfamiliar and noisy inputs. The contribution is the integration and evaluation of these capabilities for sequential XRD interpretation. The framework provides a basis for future integration into autonomous experimentation systems, with experimental planning, instrument control, and closed-loop deployment remaining areas for future development.
- [10] arXiv:2611.17609 [pdf, html, other]
-
Title: Homogeneous pursuit in $β$ modelSubjects: Methodology (stat.ME); Statistics Theory (math.ST); Machine Learning (stat.ML)
Data in forms of networks has been encountered in various prevalent fields at the arrival of big data era. To unveil and recover the true model with core units with numerous links, the $\beta$-model under sparsity constraint is proposed to meet the expectation of statisticians and engineers. Not only does this network model inherit spectacular property from the Erdős-Rényi model, but also exhibits degree heterogeneity through tuning sparsity parameter $s$. In this paper, we propose Homogeneous Pursuit (HP), an $\ell_0$-constrained maximum likelihood estimation method implemented through a primal-dual algorithm. To ease the harshness of computation, we find the surrogate function of the corresponding objective function to derive Majorization Minimization algorithm through the primal-dual process. Moreover, after meticulous evaluation, the algorithm brought about captures the property of fine convergence rate in a contraction mapping way. Under some mild conditions, the proposed generalized information criterion (GIC$^*$) for selecting the sparsity level is consistent to find the truth. Combining HP with GIC$^*$ gives the adaptive procedure HP-GIC$^*$; HP-BIC denotes the same estimation method with sparsity selected by BIC. Finally, simulation studies show that GIC$^*$ improves support recovery relative to BIC in the reported comparisons, while HP achieves lower computational cost while maintaining estimation accuracy comparable to the sparse benchmark methods. A microfinance application illustrates the use of HP-GIC$^*$ for studying the association between network prominence and participation, with improved fit in the reported village-level regressions.
- [11] arXiv:2611.17715 [pdf, html, other]
-
Title: GIC model selection for high-dimensional CCAComments: 1 figureSubjects: Methodology (stat.ME)
Canonical correlation analysis (CCA) is a widely used statistical tool for exploring the dependence structure between two sets of variables. An important issue in CCA is determining the rank. Among information-based criteria for rank selection, the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) are two commonly used approaches. Both rely on the number of free parameters to quantify model complexity, which may not adequately account for the underlying dependence structure when the assumed model is misspecified. As a result, AIC and BIC may exhibit unsatisfactory performance, particularly when the remaining canonical correlations do not follow the assumed low-dimensional structure. To alleviate this difficulty, we adopt the idea of the generalized information criterion (GIC) to develop a data-adaptive measure of model complexity for CCA rank selection. The proposed complexity measure incorporates the magnitudes of the sample canonical correlations, thereby accounting for the spectral structure of the data and improving robustness to model misspecification. We investigate the asymptotic properties of the proposed GIC under a high-dimensional setting in which the sample size and both variable dimensions grow proportionally. Our theoretical analysis aims to characterize the conditions under which GIC can distinguish signal canonical correlations from noise and achieve more reliable rank selection than conventional information criteria.
- [12] arXiv:2611.17744 [pdf, html, other]
-
Title: Minimaxity in Robust Nonparametric Regression: from Heavy-Tailed to Sub-Gaussian AssumptionsComments: 50 pagesSubjects: Statistics Theory (math.ST)
We study nonparametric regression over star-shaped function classes under strong adversarial contamination. The clean (original) observations $(\tilde{X} _{i},\tilde{Y} _{i})_{i=1} ^{n}$ are i.i.d. and satisfy $\tilde{Y} _{i}=f ^{*}(\tilde{X} _{i})+\xi _{i}$, where $f ^{*}\in \mathcal{F}$, $\tilde{X}$ is distributed as $\mathbb{P} _{X}$, and the noise is conditionally centered. An adversary may inspect the clean sample and replace at most $\lfloor \epsilon n\rfloor$ observation pairs. We assume that $\mathcal{F}\subset L _{2}(\mathbb{P} _{X})$ is star-shaped with finite diameter $d$.
Under a Bernstein-type moment condition on function increments and a bounded conditional second moment for the noise, we establish the matching lower and upper bounds of order $\left((\eta ^{*}) ^{2} \vee \epsilon \right)\wedge d ^{2}$, uniformly over admissible star-shaped function classes, therefore characterizing the minimax rate of estimation error, where \begin{equation*}
\eta ^{*}:=\sup\left\{\eta >0: n\eta^2\leq\log \mathcal M _{\mathcal{F}}^{\text{loc}}(\eta,c)\right\}. \end{equation*} Here $\log \mathcal{M} _{\mathcal{F}} ^{\text{loc}}(\eta,c)$ denotes the local metric entropy of $\mathcal{F}$ in $L _2(\mathbb{P} _{X})$, and $c$ is a sufficiently large constant. Under a uniform relative sub-Gaussian condition on function increments and noise, the lower bound is adjusted to $\left((\eta ^{*})^{2}\vee \epsilon ^{2}\right)\wedge d ^{2}$ while the upper bound improves to $\left((\eta ^{*})^{2}\vee \epsilon ^{2}\log ^{2}\left(\frac{e}{\epsilon }\right)\right)\wedge d ^{2}$, yielding minimax optimality up to logarithmic factors. - [13] arXiv:2611.17771 [pdf, html, other]
-
Title: Interpretable confidence regions for causal discoverySubjects: Methodology (stat.ME)
Most work in causal discovery focuses on point estimation of the underlying causal graph, while uncertainty in the learned structure has received substantially less attention. We develop confidence regions for the completed partially directed acyclic graph (CPDAG) identified from observational data. Our approach builds on perturbed PC (Chang et al., 2026), which generates plausible CPDAGs by perturbing the conditional independence tests underlying the widely used PC algorithm. The threshold used in these tests controls how perturbations translate into variation across the resulting CPDAGs. We propose a new thresholding strategy that suppresses spurious variation across runs while preserving asymptotic coverage, producing a substantially tighter collection of plausible CPDAGs. This collection is summarized through confidence balls and model confidence intervals constructed from the partially ordered set of CPDAGs under model inclusion. We also develop new theory and algorithms for the exact computation of these regions. Simulations show that our confidence regions attain good coverage while being substantially smaller and more interpretable than the original perturbed PC collections.
- [14] arXiv:2611.17774 [pdf, html, other]
-
Title: Estimand-specific bootstrap confidence intervals after optimism correction for prediction modelsComments: 38 pages, 7 figures, 3 tables. Supporting Information included after the main articleSubjects: Methodology (stat.ME)
Confidence intervals after bootstrap optimism correction are often reported for prediction models, but what they cover depends on the estimand. We distinguish two estimands: the target-population performance of the final model developed from the observed sample, $\theta_{\mathrm{FM},n}$, and the average performance over repeated model development, $\theta_{\mathrm{proc},n}$. Using first-order expansions of the estimation errors, we identify the bootstrap root that matches the error for $\theta_{\mathrm{FM},n}$. We use this root in the nested error bootstrap (NEB) as a new reference method, and we show that the optimism error bootstrap (OEB), which needs no inner bootstrap, approximates NEB. We also show that the conventional two-stage bootstrap (TSB) and location-shifted bootstrap (LSB) reproduce the first-order variation for $\theta_{\mathrm{proc},n}$. In simulations, NEB generally achieved coverage closer to the nominal level than TSB and LSB for the area under the receiver operating characteristic curve, Brier score, and log loss. OEB was close to NEB at moderate to high events per variable (EPV) but could give narrower intervals at low EPV. The distinction is particularly clear for calibration measures: first-order effects of perturbing the training and evaluation distributions cancel in the TSB root, yielding a degenerate first-order limit. In contrast, the NEB root has a nondegenerate first-order limit and yields asymptotically valid confidence intervals for $\theta_{\mathrm{FM},n}$ under regularity conditions. Consistent with theory, TSB showed substantial undercoverage for calibration even at larger sample sizes. Our findings emphasize the need to construct confidence intervals for a specified estimand and performance measure.
- [15] arXiv:2611.17778 [pdf, html, other]
-
Title: A Consistent Bayesian Adaptive Quotient Graph Framework for Graph-Informed Random Partition ModelsSubjects: Methodology (stat.ME)
Bayesian graph-informed random partition models are widely used to model heterogeneity in structured data. A prominent approach assigns a prior directly on the space of contiguous partitions of a pre-determined graph G; however, such models are not data-adaptive and can be computationally infeasible for large graphs. To address these limitations, we propose a Bayesian adaptive quotient graph framework that induces partitions through a two-level hierarchical prior: a prior is first imposed on quotient graphs of G, followed by a conditional prior on a reduced contiguous partition space induced by each quotient graph. We develop high-level assumptions that guide the construction of the two-level prior such that the reduced space retains desirable partitions. We further establish high-level assumptions for posterior concentration on desirable quotient graphs and partitions. The framework can be readily embedded into many existing Bayesian hierarchical models with latent cluster structures as a contiguous random partition prior. As a specific example, we apply the proposed framework to the spatial random partition model proposed in Huang and Sang (2026). The resulting model adapts to the unknown number of clusters, achieves a faster clustering-consistency contraction rate, and yields substantially lower partition error and computation time in simulations. Finally, we apply the resulting model to a large spatial transcriptomics dataset to demonstrate its practical utility.
- [16] arXiv:2611.17875 [pdf, html, other]
-
Title: Stratified Randomization Incorporating Historical Control DataComments: 29 pages, 1 figure, 4 tables, 1 algorithm, Presented at ISCB-GMDS 2026, Freiburg, GermanySubjects: Methodology (stat.ME)
Randomized controlled trials are often infeasible in rare diseases because of limited sample size, high costs, or lengthy durations. Hybrid control designs address this by combining a concurrent control arm with historical data. However, existing methods such as propensity score matching (PSM) borrow historical data only at the analysis stage, which limits power gains when covariate distributions differ between historical and concurrent data. We propose a stratified block randomization method for hybrid control trials. Within each stratum, more concurrent subjects are allocated to the treatment arm than to the concurrent control arm, and the difference is filled with historical controls so that the treatment and hybrid control arms are balanced 1:1; once the available historical controls in the stratum are used or a pre-specified overall cap is reached, allocation returns to 1:1. Historical controls are incorporated either by sampling or by weighting. We compared both methods with PSM in terms of type I error rate, power, and sample size via Monte Carlo simulations. The sampling method maintained the type I error rates when there was neither heterogeneity in the outcome model nor unmeasured covariates. It also had power comparable to or greater than that of PSM, particularly when historical controls were unevenly distributed across strata. The power of the weighting method increased with the pooled historical sample size, but its type I error rate was inflated when covariates beyond the stratification factors were present or when stratification factors were dichotomized from continuous covariates. The proposed method incorporates historical data at the allocation stage rather than only at the analysis stage, enabling more efficient use of historical data.
- [17] arXiv:2611.17923 [pdf, html, other]
-
Title: High-Dimensional Gaussian and Bootstrap Approximation: Near $1/\sqrt{n}$ Rates with General CovarianceComments: 42 pagesSubjects: Statistics Theory (math.ST); Probability (math.PR)
We consider Berry-Esseen bounds over rectangles for sums of $n$ random vectors in $\mathbb{R}^p$ that are i.i.d. A longstanding open question in high-dimensional probability and statistics is whether bounds of order $1/\sqrt n$ are attainable up to factors of $\log(pn)$ when the population covariance matrix is unrestricted. We answer this question affirmatively, provided that the random vectors have sub-Gaussian entries with non-zero variance. In addition, we provide analogous results for bootstrap approximation and Gaussian comparison, which make use of a special discrepancy measure between covariance matrices that enables improvement with respect to prior work. Taking an approach based on interpolation, the novel aspects of the proofs deal with bounding the interpolation derivatives in a way that is not sensitive to the covariance structure.
- [18] arXiv:2611.17934 [pdf, html, other]
-
Title: Statistical guarantees for signature kernel ridge regressionComments: Keywords: Path space regression, Signature transform, Rough paths, Functional data analysis, Kernel methodsSubjects: Statistics Theory (math.ST); Probability (math.PR)
Path signatures are a standard tool for learning functionals on path space. The main underpinnings of their application are universal approximation results. For a wide range of function classes on path space, these results state that any function in such a class can be approximated arbitrarily well by a linear functional of the path signature. Hence, linear functionals of the signature are in principle expressive enough for learning functions on path space. Nonetheless, these results do not provide insights into the statistical question of how much data is needed to identify a suitable signature functional. In this work, we study this question in the framework of kernel ridge regression with the signature kernel. The target is the conditional expectation $\E[Y|X]$, where $X$ is the observed path and $Y$ is a real-valued response. Under different assumptions on the moments of the signature kernel and the response, we provide finite-sample and consistency guarantees for the method. We supplement our theoretical analysis with numerical experiments that illustrate our results.
- [19] arXiv:2611.17971 [pdf, html, other]
-
Title: Efficient and Practical Clustering of Variables in Causal GraphsSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)
Clustering variables in a graph may improve visual clarity and simplify causal inference. Harnessing the full potential of clustering requires an automated method to detect useful clusters in a graph. We introduce a novel class of clusters called conduits, and present a sound, complete and efficient algorithm for detecting all conduits in the graph. In addition to algorithmic detectability, conduits appear naturally in causal graphs stemming from real-world applications, enabling substantial size reduction. Furthermore, the structure of conduits enables the application of known identifiability results regarding clustering. We demonstrate these properties in a simulation study with real-world graphs.
- [20] arXiv:2611.18052 [pdf, html, other]
-
Title: evgmrf: An R Package For Gaussian Markov Random Field Extreme Value ModelsSubjects: Methodology (stat.ME); Computation (stat.CO)
We introduce \textsf{evgmrf}, an \textsf{R} package for fitting spatially varying extreme value models using Gaussian Markov random fields (GMRFs). The package supports the generalized extreme value distribution for block maxima, the generalized Pareto distribution for threshold exceedances, the Poisson--GPD point-process model, and the $r$-largest order-statistics model. Distributional parameters may vary over regular spatial grids or arbitrary regions that can be represented by a valid adjacency matrix, with optional fixed effects and covariate dependence. Spatial structure is represented through intrinsic conditional autoregressive, proper conditional autoregressive, and Besag--York--Mollié 2 models, which are extended with non-Gaussian random-effect distributions. Estimation uses penalized likelihood with smoothing or dependence hyper-parameters estimated using a Laplace approximation, exploiting sparse precision matrices for computational efficiency. The package provides methods for model summaries, spatial visualization, prediction on link and response scales, estimation of extreme quantiles, and uncertainty quantification in the form of approximate standard error estimates, or via simulations from parameter sampling distributions. Examples based on gridded and county-level precipitation data illustrate the use of the package for spatial modelling of environmental extremes. \textsf{evgmrf} provides a unified, accessible framework for extreme value analyses in which distributional parameters exhibit structured spatial variation.
- [21] arXiv:2611.18086 [pdf, html, other]
-
Title: Does Increasing Government Spending on Tertiary Education Provide a Return on Investment? A Bayesian Causal Inference AnalysisComments: 37 pages, 10 figures, 10 tables. Open-source R/C++ code, data, and interactive policy web application available at this https URLSubjects: Applications (stat.AP); Computation (stat.CO); Methodology (stat.ME)
When a government increases public spending on tertiary (university) education, does it provide a positive fiscal return on investment once general economic growth is separated out? Cross-country data make this hard to answer because growing economies naturally collect more tax, international ratios divide per-student spending by GDP per person and student headcount, and cohorts take years to graduate. We assemble a five-year panel of 121 countries (1985-2024) and fit continuous-treatment Multivariate Bayesian Causal Forests that model four tax changes jointly whilst adjusting for starting conditions, predicted funding, and follow-up economic growth. A +10 percentage-point (pp) increase in per-student university spending (% of GDP per person) shifts the tax mix toward direct income and profit taxation by +0.36 pp after 5 years (93% probability of a positive effect) and +0.71 pp after 10 years (96% probability). Overall tax-to-GDP takes a decade to respond as cohorts graduate: 10-year total non-resource tax rises by +0.24 pp of GDP (89% probability) and direct income tax by +0.12 pp (90% probability) per +10 pp of per-student spending, by +0.37 pp (100% probability) after five-year within-country funding increases, and by +0.22 pp in total tax (95% probability) and +0.15 pp in direct income tax (98% probability) per +25% increase in real PPP dollars per student. Comparing one entering cohort's degree cost against its 40-year career tax payments shows a typical three- to five-fold total tax return in high-income economies (EUR 2.37 billion in Ireland returning EUR 9.66 billion in total tax with a 77% probability of net profit and EUR 2.87 billion in direct income tax, or EUR 11.55 billion in the UK returning EUR 58.33 billion), whereas expanding the university budget without protecting spending per student yields no return.
- [22] arXiv:2611.18095 [pdf, html, other]
-
Title: Training-point consistency of kernel ridge regression with varying correlation scalesSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
The scale parameter in kernel ridge regression plays a pivotal role in modeling the correlation structure of data. In modern applications, sampling density and the underlying correlation range often change simultaneously, making a fixed-scale assumption highly restrictive. However, establishing asymptotic theories under varying scale parameters is fundamentally challenging. To address this gap, this paper develops a unified theoretical framework for kernel ridge regression under varying scale parameters. We establish that, on uniform tensor grids with product exponential kernels, the consistency of the training-point risk holds if and only if the effective local sample size diverges to infinity. This sharp criterion universally covers all regimes along arbitrary deterministic scale sequences, seamlessly unifying the long-range, fixed-scale, and short-range. Furthermore, we provide an explicit, signal-independent construction of a deterministic regularization penalty that guarantees this consistency. Numerical experiments illustrate the criterion and the corresponding penalty across all the three regimes.
- [23] arXiv:2611.18128 [pdf, html, other]
-
Title: Cross-Fitted Doubly Robust Estimation with Student-t Outcome Engines: ECM Algorithms, Monte Carlo Evidence and Schistosomiasis Programme Data from the Democratic Republic of the CongoComments: 37 pages, 27 figures, 11 tables. Python code (single file) reproducing all analyses and figures provided as an ancillary fileSubjects: Methodology (stat.ME)
Routine data on preventive chemotherapy for schistosomiasis are confounded, clustered, heavy-tailed and heaped. We study how cross-fitted augmented inverse probability weighted (AIPW) estimators behave, and how they should be computed, when least squares is replaced by Student-t outcome engines: Huber and t iteratively reweighted least squares, and a two-component t scale-mixture regression fitted by an ECM algorithm, alongside a residual-clipped augmentation. We give four short propositions (exact AIPW bias, the efficiency bound, the estimand of clipping, monotone ascent of a scale-constrained ECM), a single-file Python implementation with cluster-level cross-fitting, cluster-robust standard errors and a cluster bootstrap, and a Monte Carlo study with up to 1000 replications per cell. Robust outcome engines leave AIPW unchanged (RMSE differences at most 0.002), whereas clipping cuts RMSE by 26-45% under symmetric contamination, under-covers, and, as a plasmode experiment built on real covariates and residuals shows, shifts the estimand when residual laws differ between arms. One-sided artefacts that differ between arms bias every estimator. Heaped data also break naive code: the MAD scale collapses and the mixture likelihood is unbounded, and we give simple guards. Applied to 1485 rounds in 362 health zones of the Democratic Republic of the Congo (ESPEN, 2014-2025), integrated delivery appears to raise programme coverage by 17.4 percentage points, yet epidemiological coverage differs by only 0.9: the gap comes from targeting denominators, a real differential artefact that no outcome engine can remove.
- [24] arXiv:2611.18141 [pdf, html, other]
-
Title: Characterizing M-estimators using canonical divergences from information geometryComments: 30 pages, 1 figure. Supplementary material includedSubjects: Statistics Theory (math.ST); Methodology (stat.ME)
On Euclidean space, the losses that produce consistent M-estimators of a functional are its strictly consistent scoring functions (Dimitriadis, Fissler and Ziegel, 2024), and Osband's principle characterizes these through identification functions, which define method-of-moments estimators. We extend this characterization to the probability simplex. We define the residual, which on Euclidean space is a subtraction, as the inverse exponential map of an affine connection. It identifies the exponential barycenter of the connection: the arithmetic, normalized geometric and Fréchet means for the mixture, exponential and Levi-Civita connections. Its integral along geodesics is the canonical divergence of Ay and Amari (2015). A geometric form of Osband's principle decides which losses each connection admits. For the flat connections every strictly convex generator yields a consistent Bregman loss. For the curved $\alpha$-connections, half the squared Fisher-Rao distance is (up to scale) the only consistent loss for the Fréchet mean, and for $\alpha \notin \{-1, 0, 1\}$ no such loss exists and only the method of moments remains. The residual is integrable only at $\alpha = 0$, sharpening the integrability claim of Ay and Amari. The characterization yields the estimating equations of regression on the simplex and the calibration decomposition of compositional forecasts.
- [25] arXiv:2611.18164 [pdf, html, other]
-
Title: Detection and Classification in Latent Spaces: High-Dimensional Analysis and ValidationSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Foundation models produce high-dimensional latent representations in which suitably designed statistical decision rules can be effective yet analytically tractable. We study such detectors in the high-dimensional regime, where latent dimension and training sample size grow proportionally. For linear and quadratic statistics, we derive explicit large-deviation characterizations of false-alarm and miss-detection performance under the Neyman-Pearson paradigm, and show how the binary analysis extends to multi-class classification. The resulting error probabilities and corresponding decay rates exhibit non-monotonic dependence on the shape parameter, including double-descent and, in some regimes, multiple-descent. Experiments on vision, text, and audio datasets using modern foundation models show close agreement between theoretical predictions and empirical performance. In particular, the proposed framework allows us to set a prescribed false-alarm level and accurately predict the corresponding miss-detection probability. Overall, our results provide a theoretically grounded and interpretable framework with competitive empirical performance across different modalities.
- [26] arXiv:2611.18180 [pdf, html, other]
-
Title: LaMe: a Joint Latent Mediation Framework for High-Dimensional SystemsSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Quantitative Methods (q-bio.QM); Methodology (stat.ME)
Mediation analysis seeks to understand how an exposure influences an outcome through intermediate variables. However, as increasingly rich datasets become available, hundreds to thousands of potential mediators make these systems difficult to model and hard to interpret. Latent-variable approaches offer a natural way to accommodate these high-dimensional settings. However, existing latent mediation methods face limitations, including overparameterization, biased inference, or limited scalability. We develop LaMe, a joint linear-Gaussian framework that summarizes a high-dimensional exposure-mediator-outcome system through a small number of latent variables. We identify natural direct and indirect effects under an explicit assumption that the manifest measurements fully capture the latent mediators, and characterize conditions under which the latent representation effectively restricts the observed cross-covariance structure. Simulations assess recovery of observed association parameters, natural effects, and manifest features associated with latent mediation pathways across a range of challenging settings. Four real-world applications demonstrate competitive predictive performance with compact latent representations, including single-dimensional representations of biomarker and proteomic measurements. Together, these results establish LaMe as a practical framework for estimating and summarizing multivariate mediation systems under explicit structural and measurement assumptions.
- [27] arXiv:2611.18188 [pdf, other]
-
Title: A Bayesian approach to technological emergence, climate variability, and demographic stability in northern South AmericaSebastian Fajardo, Philip Riris, María Vélez, Natalia Álvarez Correa, A. Jacobo Hidalgo, Luis Felipe Navarro, Rodolfo Cepeda, Nicolás Loaiza, Carlos Reina, Javier Ruiz-Pérez, Marco MadellaComments: 30 pages, 6 Figures, submitted to Journal of Archaeological ScienceSubjects: Applications (stat.AP)
Human communities produce technology in almost every environment, but the conditions that enable technologies in the tropics are not well understood. We test two competing hypotheses---that technological emergence is triggered by environmental stress and that technology tracks growth in effective populations---by integrating Holocene archaeological and paleoclimate evidence. We use archaeological derived radiocarbon dates from Colombia as a proxy for human activity, including dating plant management, pottery, metallurgy, and raised fields. For palaeoclimate, we use two El Niño--Southern Oscillation (ENSO) reconstructions: Laguna Pallcacocha and Laguna El Junco in Ecuador. A Bayesian changepoint at 3575 cal BP (95% highest posterior density interval (HPDI) = 3811--3329) marks a ~13-fold increase in the growth rate of pre-Hispanic human activity ($r_2$/$r_1$ median = 13.3, 95% HPDI = 8.1--22.2). Plant management and pottery predate this shift by more than a millennium, whereas metallurgy and raised fields cluster within the early acceleration phase, during relative climatic and demographic stability. A Bayesian log-linear regression of human activity on ENSO frequency and intensity after the demographic changepoint reveals an asymmetric covariation: reduced ENSO frequency and intensity suppress activity by 58% and 21%, whereas ENSO frequency raises activity modestly (~11--13%). Therefore, ENSO regulates demographic activity rather than triggering technological emergence. These patterns suggest that pre-Columbian communities may have innovated when they could compute---when an effective population sat above a critical threshold and stability windows permitted combinatorial search---rather than when they were forced to by environmental stress, expressed as demographic or climatic variability.
- [28] arXiv:2611.18225 [pdf, html, other]
-
Title: Causal and conditional tail coefficients: asymptotics, inference and graph recoverySubjects: Statistics Theory (math.ST); Methodology (stat.ME)
The causal tail coefficient (CTC) measures the influence of an extreme value of one variable on another variable and has been proposed as a tool for causal discovery in heavy-tailed models. Despite its popularity, the asymptotic behavior of its empirical counterpart has not been well understood. We develop an asymptotic theory for the rank-based CTC estimator with an intermediate threshold under nonparametric conditions that are satisfied by all standard copula models. A key observation is that the CTC is determined by the mean of a conditional tail limit distribution, which clarifies its interpretation and determines the asymptotic variance of the estimator. We establish joint asymptotic normality for the entire CTC matrix and prove consistency of a multiplier bootstrap for simultaneous inference. For heavy-tailed linear and recursive max-linear structural equation models, we show that the CTC equals one if and only if the first variable is an ancestor of the second, which leads to confidence graphs for the ancestral graph with asymptotic coverage guarantees. All results extend to general conditional tail coefficients, defined as further functionals of the conditional tail limit distribution. A simulation study and an application to dietary intake data illustrate the methods.
- [29] arXiv:2611.18291 [pdf, html, other]
-
Title: Identification and Inference of Bi-directional Causal Effects in Mendelian Randomization without Instrument SelectionComments: 61 pages, including supplementary materialSubjects: Methodology (stat.ME)
Mendelian randomization (MR) uses genetic variants as instruments to estimate causal effects. Bidirectional MR usually uses two reversed unidirectional analyses, but feedback can invalidate instruments in at least one direction and induce correlated pleiotropy. We model both traits as a stable cyclic system and estimate both directional effects from GWAS summary data without instrument selection or a prespecified valid-instrument subset. After LD score normalisation, marginal associations form a heteroscedastic noisy independent component analysis model, assuming the variants' direct effects on the two traits are independent. If those effects are non-Gaussian, this identifies the total-effect matrix up to scaling and permutation; no-self-feedback and stability of the system further resolve these ambiguities. We develop an estimator based on noise-adjusted second- and fourth-order moments, avoiding inversion of the high-dimensional linkage disequilibrium matrix and having computational cost linear in the number of variants. Under weak local dependence, we establish consistency, asymptotic normality, and validity of a chromosome-respecting moving block bootstrap. Simulations show negligible bias, generally reliable interval coverage, and higher rejection rates for nonzero directional effects than focusing-based tests. Three real-data applications illustrate how joint analysis of feedback systems can yield causal decompositions differing from two separate unidirectional analyses.
- [30] arXiv:2611.18348 [pdf, html, other]
-
Title: Quantifying Dependence Effects on $k$-out-of-$n$ Reliability Relative to the Independent BenchmarkSubjects: Statistics Theory (math.ST); Probability (math.PR); Methodology (stat.ME)
Understanding how dependence among component lifetimes affects the reliability of redundant systems is a central problem in reliability engineering. In this paper, we develop a framework for quantifying the impact of multivariate dependence on $k$-out-of-$n$ system reliability relative to the independent benchmark. Using an orthogonal Legendre expansion of the copula density, we derive an exact decomposition that separates the dependence structure from the system architecture. This representation identifies how individual dependence contributions are transmitted to system reliability through explicit structural weights. An application to multivariate Farlie-Gumbel-Morgenstern copulas demonstrates that higher-order dependence can produce distinct reliability effects even when all bivariate copulas remain unchanged. Our findings provide a systematic approach to quantifying and interpreting the effects of multivariate dependence on the reliability of redundant systems.
- [31] arXiv:2611.18362 [pdf, html, other]
-
Title: Asymptotic properties for weak Markov-switching AR(p) modelsYacouba Boubacar Mainassara (UPHF, INSA Hauts-De-France, CERAMATHS), Jean-Armel Bra (UMLP, LMB), Landy Rabehasaina (UMLP, LMB)Subjects: Applications (stat.AP); Methodology (stat.ME)
In this paper we study the asymptotic properties of an autoregressive model of order p (AR for short), possibly non-centered, whose parameters switch according to an unobserved Markov chain. Departing from the conventional assumption of independent innovations, we merely require the errors to be uncorrelated but not necessarily independent, thereby widening substantially the range of potential applications of Markov-switching AR models. We establish sufficient conditions for the consistency and asymptotic normality of the proposed moment estimators. We also propose a consistent estimator of the asymptotic variance matrix of the moment estimators. The finite-sample performance of the proposed procedures is illustrated through Monte Carlo experiments and an empirical study based on hourly meteorological data, which together provide strong support for the theoretical findings.
- [32] arXiv:2611.18394 [pdf, html, other]
-
Title: Lipschitz-Constrained Discriminators in Generative Modeling: Asymptotic Behavior and Support MismatchSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
We study the asymptotic behavior of optimal discriminators associated to $L-$Lipschitz regularized KL-divergences of two probability distributions $Q$ and $P$ that interpolate between the rescaled Wasserstein$-1$ metric by some parameter $L>0$ and the KL divergence. The parameter $L$ plays the role of a Lipschitz constraint and is viewed as a regularization parameter: small values of $L$ enforce a more geometry-aware discriminator, while as $L$ increases the geometric regularization is progressively relaxed. It is a known result [3] that such divergences converge to the KL divergence as $L\to\infty$. In this paper, we characterize the asymptotic behavior of the optimal discriminators as $L\to\infty$ in different regimes: Firstly, under $Q\ll P$ and the Radon-Nikodym derivative $\frac{dQ}{dP}$ is bounded from above and bounded away from zero. Secondly, when $Q$ and $P$ are discrete distributions satisfying $Q\ll P$, and finally when $Q$ and $P$ are non absolutely continuous discrete distributions. This result motivates training strategies which involve discriminators, such as adversarial generative models. In practice, the data distribution is often represented by a discrete empirical measure, while, especially at the early stages of the training, the model distribution may have little or no overlap with the empirical data distribution.
- [33] arXiv:2611.18404 [pdf, html, other]
-
Title: Comments on: `CliPS: How to identify cluster distributions in Bayesian mixture models'Comments: This is an invited discussion for the paper arXiv:2603.00277v2. The citation of the published version is Malsiner-Walli, G., Frühwirth-Schnatter, S. & Grün, B. CliPS: How to identify cluster distributions in Bayesian mixture models. TEST (2026). this https URLSubjects: Methodology (stat.ME)
I congratulate the authors on their contribution, which addresses important challenges in understanding posterior distributions in Bayesian mixture models. As underlined by the authors, the Bayesian framework has several advantages, yet label-switching, identifiability issues, and the trans-dimensional nature of parameter spaces make exploiting the Bayesian approach and understanding the resulting posterior difficult in practice. The paper offers clear explanations and a simple, intuitive procedure to address this by clustering the Markov chain Monte Carlo (MCMC) draws of component-specific parameters. It is clearly illustrated on several examples.
- [34] arXiv:2611.18411 [pdf, other]
-
Title: Making Experiment Results Comparable through Sitewide Impact EstimationComments: Extended abstract accepted for presentation at the 2026 Conference on Digital Experimentation at MIT (CODE@MIT)Subjects: Applications (stat.AP); Econometrics (econ.EM)
Roblox spans many product surfaces-discovery, the virtual economy, creator tools-that differ sharply in reach and in how much the affected population contributes to any sitewide metric. A randomized experiment shows whether a treatment works for the users in its sample, but this experiment lift is not comparable across surfaces, because surfaces differ in reach and in their contribution to sitewide metrics. The quantity a strategic decision needs is a counterfactual-how much an aggregated sitewide metric (e.g., total platform play time) would move if the treatment reached 100% of its target population, which may be only part of the sitewide population, during the measurement period. We call this sitewide impact. Its scaling factor, which we call the coverage factor, combines the reach and the target population's contribution to the sitewide metric under no treatment. Hence a low-reach surface whose users carry much of the metric could outrank a high-reach surface whose users carry little. We develop two estimators under this framing: one for difference-in-means metrics and one for variant-level ratio metrics (a metric defined as a ratio of two population aggregates). Empirically, experiment lift is a poor proxy for sitewide impact-across 139 experiments that are statistically significant on time spent, the Spearman rank correlation between the two is 0.20, and 5 of the 10 largest experiment lifts are among the 10 largest sitewide impacts. We validate the estimator with A/A tests and against a long-term holdout, and apply it to cross-domain prioritization and attributing year-over-year growth to shipped innovations.
- [35] arXiv:2611.18416 [pdf, html, other]
-
Title: A fast and sound algorithm to simplify summations in interventional queries expressionsComments: 19 pagesSubjects: Methodology (stat.ME); Computation (stat.CO)
Identification formulas returned by the ID algorithm often contain summations that can be removed. The procedure proposed in arXiv:1806.07082 (TK) removes them by combining factors of the formula into joint distributions, through the chain rule and through the insertion of factors for missing variables. We make two changes to this procedure: only the factors that involve the summed variable are combined, and every factor is written with a minimal conditioning set. These two changes determine the rest of the procedure, in which each search over a power set is replaced by a single test and the variables to insert are fixed in advance, and they remove summations that the TK procedure leaves in place. The resulting procedure is sound and, for minimal input terms, complete for each individual elimination with respect to joins and insertions.
- [36] arXiv:2611.18444 [pdf, html, other]
-
Title: Constrained Stochastic Policy Targeting under Heterogeneous Partial InterferenceSubjects: Methodology (stat.ME); Econometrics (econ.EM)
Under heterogeneous interference, treating individuals with particular characteristics affects the population average outcome through their heterogeneous direct response to treatment and indirect effect on others, creating opportunities for efficient policy design. For instance, policymakers may minimize resource use by vaccinating individuals identified as superspreaders to achieve a target reduction in disease incidence. In such setting, we first study the problem of learning individualized stochastic treatment policies that minimize population average treatment probability subject to a prespecified outcome target. We propose an estimation approach based on a smooth surrogate for constraint violations and interference-aware inverse-probability-weighting or self-normalized estimators of the population average outcome, and establish finite-sample regret guarantees for both estimators. We also consider the complementary problem of maximizing the population average outcome subject to a constraint on average treatment probability. We apply the proposed methodology to a Colombian conditional cash transfer program aimed at improving school attendance. The application demonstrates that accounting for interference and individualizing treatment probabilities can reduce resource use relative to policies that ignore interference or impose homogeneous treatment probabilities.
- [37] arXiv:2611.18445 [pdf, html, other]
-
Title: Hidden Markov Models for Dynamic Co-ClusteringSubjects: Methodology (stat.ME)
To jointly identify temporal regimes and row-column structure in matrix-valued time series, we propose a hidden Markov model (HMM) with Gaussian co-clustering distributions. A single latent regime governs each matrix. Conditional on the regime, row and column memberships define Gaussian blocks, each with its own mean and variance. Membership probabilities and block parameters are shared across occurrences of a regime, while individual memberships may vary between time points.
- [38] arXiv:2611.18454 [pdf, html, other]
-
Title: Dynamic bigraphical Matérn--Whittle processes: learning dependence graphs among spatial-temporal fieldsSubjects: Methodology (stat.ME); Machine Learning (stat.ML)
Many studies record several variables at the same sites over a sequence of periods, such as pollutant concentrations at monitoring stations or case counts of infectious diseases by district. A central question concerns the dependence between the variables and its change over time. We develop dynamic bigraphical Matérn--Whittle (DBMW) processes for such data. Each variable has a spatial field that evolves by autoregression. A graph over the variables couples the temporal innovations of the fields. Its links appear and vanish over time, and a link may change sign. A graph Laplacian on the sites gives the spatial structure, and a canonical exponential family links the fields to Gaussian, count or binomial responses. The joint distribution of the latent fields over all sites, variables and periods factorizes over the eigenmodes of the graph Laplacian, and its normalizing constant is free of the graph and of the dynamics. Given the fields, the posterior of the time-varying signed graph is therefore exact, by forward--backward recursions. The estimator has three steps: a fit without a graph, which also gives the switching rates with an interval, the exact graph posterior, and a joint refit. In simulations, DBMW recovers the graph with higher average precision than graphical lasso, time-varying graphical lasso and Poisson log-normal networks, and it predicts withheld data better than a standard spatio-temporal model that fits each variable separately, with a Matérn field and autoregressive evolution, by integrated nested Laplace approximation. In weekly air-quality data from 288 U.S. sites, it recovers the seasonal change of sign between ozone and nitrogen dioxide. In German surveillance data on seven diseases, the district-level links stay positive and stable through the COVID-19 pandemic.
- [39] arXiv:2611.18455 [pdf, html, other]
-
Title: Mesh-Free Weak-Form Regression for Learning Interaction Kernels from Particle TrajectoriesComments: 47 pages, 8 figuresSubjects: Machine Learning (stat.ML); Optimization and Control (math.OC)
We propose mesh-free weak-form regression for learning interaction kernels from noisy particle trajectories. Our method uses smooth test functions to avoid differentiating noisy data and empirical particle averages to avoid density reconstruction and spatial grids. We then fit sparse combinations of candidate functions, evaluating interaction terms on sampled particle pairs to avoid all-pair quadratic cost. By transferring identifiability from the mean-field limit, we prove high-probability coefficient error bounds that separate finite-particle, time-discretization, observation-noise, and pair-sampling errors, without grid-resolution or density-estimation terms. We evaluate one-dimensional (1D) Newtonian-repulsion, 2D predator-prey, and 10D distance-localized experiments. The numerical results show trends consistent with the theoretical coefficient-error bounds. Comparisons of weak and forward-difference formulations with LASSO, Ridge, and OLS show that weak LASSO gives the smallest coefficient and drift errors in the three nominal experiments.
- [40] arXiv:2611.18466 [pdf, html, other]
-
Title: Node Localization by Multidimensional Scaling with Iterative MajorizationComments: 35 pages and 4 + 3 figuresSubjects: Machine Learning (stat.ML); Computation (stat.CO)
Node localization aims to estimate the positions of nodes based on collected data from other nodes. Depending on the data characteristics, different procedures are utilized to estimate the locations. In this paper, we introduce a comprehensive multidimensional scaling algorithm based on iterative majorization that comprises many of the different data characteristics to estimate node locations. The implicitly normalized loss function provides absolute distances and, given anchors, absolute node locations. The algorithm uses a new convergent coordinate descent procedure for the minimization of the majorization function, allowing for a natural treatment of anchors. Optimal transformations of the data seamlessly allows other data than Euclidean distances. Data from multiple sources, resulting in either asymmetric or three-way data, fit the same extensive framework.
- [41] arXiv:2611.18469 [pdf, html, other]
-
Title: Contrastive latent representation learning with denoising score matchingComments: 20 pages, 2 figures, 12 tables. Code: this https URLSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Contrastive representation learning methods operate by pulling different views of the same data sample together while pushing them away from views of different samples. A fundamental challenge is the estimation of the repulsive force, which typically requires large batch sizes, can exhibit high variance, and is susceptible to the curse of dimensionality. We introduce ScoreCLR, a contrastive representation learning method that offers a novel way to estimate this repulsion term. We consider the framework of representation learning by maximizing the mutual information between two latent views of a sample. By deriving the gradient of the Shannon entropy with respect to the encoder parameters, we obtain an equivalent objective that involves the score of the latent representation. We then use denoising score matching techniques to estimate this score with a lightweight network that is trained jointly with the encoder. We further propose regularization techniques using different noise levels and a Gaussian prior on the latent representation. ScoreCLR replaces the batch of explicit negative samples in contrastive methods with a learned repulsion whose cost is linear in the batch size rather than quadratic. On CIFAR-10 and ImageNet-100 it does so with minimal loss of linear-probe accuracy compared to SimCLR under a shared recipe, while producing projector outputs of higher effective rank, closer to the Gaussian prior.
- [42] arXiv:2611.18480 [pdf, html, other]
-
Title: Bregman geometry in regular exponential families: dual representations and relative smoothnessSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC)
Regular exponential-family distributions are ubiquitous in statistics due to their appealing and convenient properties. Notably, they can be expressed in terms of Bregman divergences, revealing fundamental connections between statistics, convex analysis, and information geometry. However, this geometric structure has not been systematically exploited to characterize these distributions with respect to their expectation parametrization. Building on a projective characterization, we establish a general construction of equivalent dual Bregman representations, together with normalization and uniqueness results. These representations express log-densities as ordinary Bregman divergences in dual coordinate systems, providing geometric characterizations of the parameter space. So-called $\beta$- and $\alpha$-scaled dual representations emerge as particular instances of this framework. We further derive dual Bregman representations of conjugate priors and establish that they preserve conjugacy. Applications to classical distributions illustrate the geometric structures underlying seemingly different statistical models. Moreover, we show that these representations provide natural reference geometries for establishing relative smoothness of the primal Bregman representation of the distribution, with explicit bounds that are sharp in the scalar and separable settings. An illustrative application to the Poisson distribution sheds new light on several works in the literature, providing complementary geometric justifications for reference geometries while deriving existing algorithms.
- [43] arXiv:2611.18542 [pdf, html, other]
-
Title: Elliptic Sampling: Optimal Interpolant Schedules for Gaussians and BeyondSubjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Generative modeling based on stochastic interpolants requires specifying two key choices: an interpolant schedule bridging the initial and target distributions, and an efficient discretization of the resulting probability-flow ODE. In this work, we introduce a new framework for joint schedule/discretization design based on the analyticity properties of the (complexified) ODE trajectories, which in turn ensure good numerical behavior. Our formulation can be solved exactly via conformal mapping, yielding a natural family of variance-preserving elliptic schedules, parametrized by Jacobi elliptic functions. Pairing an elliptic schedule with a tailored discretization, we obtain Zoloflow, a new sampling algorithm closely connected to Zolotarev's optimal rational function approximants. Crucially, both the elliptic schedule and Zoloflow explicitly incorporate the condition number $\kappa$ of the target distribution and reduce to the classical trig schedule in the limit $\kappa \rightarrow 1$. For Gaussian targets, Zoloflow achieves minimax optimality among all oblivious score-span algorithms, attaining total-variation error $\varepsilon$ using $\Theta(\log\kappa\,\log(\sqrt d/\varepsilon))$ (exact) score queries. For the log-concave case, the elliptic schedule allows sampling with at most $\varepsilon$ 2-Wasserstein-error by requiring merely $\widetilde{O}(\sqrt{d} \, \mathrm{polylog}(\kappa, 1/\varepsilon))$ score evaluations. Complementing these theoretical results, we empirically evaluate the elliptic schedule on general (non-log-concave) target distributions, including detailed comparisons with alternative schedule proposals on both synthetic benchmarks and existing image generation datasets.
- [44] arXiv:2611.18548 [pdf, html, other]
-
Title: Limitations of nonreversible lifts for subgeometric diffusionsComments: 20 pages, 1 figureSubjects: Computation (stat.CO); Probability (math.PR)
We study the effect of lifting on continuous-time Markov processes with potentially subgeometric convergence rates. Using a series of weak Poincaré-type inequalities, we derive a bound on the maximum acceleration that can be obtained through lifting. We present several examples to explore the sharpness of our derived bound, where the resulting bound has the same square-root order as in the spectral gap setting. Our results provide a framework for understanding the potential and limitations of non-reversible lifting beyond geometric convergence.
New submissions (showing 44 of 44 entries)
- [45] arXiv:2611.17447 (cross-list from math.OC) [pdf, html, other]
-
Title: Distributionally Robust Risk Evaluation with Low-Rank PerturbationsSubjects: Optimization and Control (math.OC); Machine Learning (stat.ML)
We study distributionally robust risk under perturbations restricted to low-rank matrices whose row and column spaces are shared across observations, focusing on how much worst-case risk this restriction sacrifices and how to compute the restricted worst case. Under moment or Wasserstein budgets, we derive quantitative bounds on the gap between the unrestricted and low-rank worst-case risks, characterize when the gap vanishes, and identify the smallest lossless row and column dimensions. To generate worst-case perturbations, we apply projected gradient ascent to a curvature-corrected concave problem and derive computable risk certificates. We establish global sublinear convergence for the corrected objective and linear convergence to a worst-case distribution when the budget constraint is sufficiently active.
- [46] arXiv:2611.17484 (cross-list from cs.LG) [pdf, html, other]
-
Title: SURFACE: Preference Distribution Matching on Pareto Fronts via Scalarization SteeringSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
Constructing a finite Pareto representation requires allocating a limited number of solutions across the Pareto front (PF). A desired preference distribution specifies this allocation by assigning more solutions to some PF regions and fewer to others. Scalarization generates Pareto solutions by varying objective weights and solving single-objective problems. However, a distribution of scalarization weights generally does not translate into the same distribution of Pareto solutions along the PF, because the weight-to-solution map is nonlinear and unknown in this http URL account for this distortion, we introduce SURFACE (Scalarization Updates for Redistributing Finite Approximations with Chart Estimation}), which adaptively updates scalarization weights to match a desired PF distribution. SURFACE estimates how each region of the weight grid corresponds to an area of the PF from the current scalarized solutions, then redistributes the weights accordingly. The method supports parallel, warm-started, inexact optimization with standard single-objective solvers. Theoretically, we establish that, under the stated regularity conditions, SURFACE achieves linear convergence of the preference distribution mismatch to a finite-resolution error floor of $\mathcal{O}(N^{-2/(M-1)})$, where $N$ is the number of adaptive weights and $M$ is the number of objectives. Empirically, experiments across multiple benchmarks demonstrate improved target-mass balance and PF coverage compared with established baselines.
- [47] arXiv:2611.17745 (cross-list from math.OC) [pdf, html, other]
-
Title: The Polyak-Łojasiewicz Paper: A 10-Year RetrospectiveComments: 20 pages, 6 figuresSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)
In 2016, we wrote a paper introducing the phrase "Polyak-Lojasiewicz (PL) condition" for an old idea, unsure of how readers would react. The paper ultimately had a larger impact than we expected, and we are surprised and honoured to receive the 2026 PKDD ECML Test of Time Award. With 10 years of hindsight, this note offers an informal and candid retrospective on the PL condition. We take a look back at the origins of the paper, give an AI-assisted refined version of the main theorem, review criticisms of the work, and discuss a variety of related ideas and extensions.
- [48] arXiv:2611.17746 (cross-list from cs.LG) [pdf, html, other]
-
Title: Complex Schrödinger BridgesComments: NeurIPS 2026Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Diffusion bridges have emerged as versatile frameworks for facilitating transformations between arbitrary data distributions. However, these approaches are hindered by an immense optimization space and its underlying high-order structures. This paper explores a variant of diffusion bridges for solving real-world problems in complex vector spaces governed by scaling and rotation laws, such as Hamiltonian structures. To address this, we introduce conformal optimal transport (COT), a complex-valued transportation framework designed for physical systems with geometric and dissipative dynamics under a geometric lifting. Building on this, we propose a new class of dynamical diffusion bridges, called complex Schrödinger bridges (CSBs), whose derived algorithms can handle diffusion-based tasks using a conformal geometric prior. For scalability, we propose image-to-image CSB (I$^2$CSB), a tractable diffusion-bridge technique with an analytic spiral posterior. Finally, we implement a physics-informed CSB algorithm within simulations of quantum electrodynamics, mimicking the logical behavior of quantum logic gates. Our extensive results validate the underlying theory while highlighting advantages of complex-valued diffusion.
- [49] arXiv:2611.17892 (cross-list from cs.CE) [pdf, html, other]
-
Title: Hierarchical Bayesian physics-augmented neural networks for populations of hyperelastic laws: specimen variability and uncertainty from scarce, noisy dataComments: 41 pages, 17 figures, 7 tablesSubjects: Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG); Machine Learning (stat.ML)
The hyperelastic law of a material such as soft tissue differs between specimens, and often only a few specimens can be tested. Physics-augmented neural networks (PANNs) learn such laws from data without a prescribed functional form while keeping them physically admissible, and Bayesian versions of related models quantify the uncertainty of the law of one specimen or of a pooled group of specimens. A pooled fit, however, attributes the differences between specimens to measurement noise or to uncertainty about a mean law, and neither describes a new specimen. We first develop a Bayesian PANN for one specimen, whose priors keep the learned law plausible for few, noisy data. We then extend it to a hierarchical Bayesian PANN, a nonlinear mixed-effects model in which all specimens share learned polyconvex features and each specimen is a draw from a population of laws. Features, specimens, and population are inferred jointly by variational inference, with a separate model of the measurement noise, so that the variability between specimens is not mistaken for noise, the population carries its own uncertainty, and every specimen drawn from it is polyconvex. The population predicts the response of new specimens with credible intervals, without data or from a partial test, and assigns them to cohorts with quantified uncertainty, all without a prescribed form of the law. On synthetic populations, it recovers the population from few specimens and predicts new specimens more reliably than three published methods. On Treloar's rubber and on healthy and aneurysmal aortas, it reaches the accuracy of a deterministic PANN and assigns no aorta specimen to the wrong cohort with confidence, although it underestimates the uncertainty of untested curves.
- [50] arXiv:2611.18227 (cross-list from econ.EM) [pdf, html, other]
-
Title: Broken Adaptive Ridge in High-Dimensional Vector AutoregressionsSubjects: Econometrics (econ.EM); Methodology (stat.ME)
We study regularized estimation in high-dimensional vector autoregressions under physical dependence and row-wise approximate sparsity using the Broken Adaptive Ridge (BAR) estimator. BAR is an iteratively reweighted Ridge procedure whose fixed-point first-order conditions imply a coordinatewise thresholding rule that produces exact zeros. We show that the BAR estimation error can be analyzed on the union of the population support above the BAR threshold and the support selected in the sample. The size of this union can be controlled at the rate implied by approximate sparsity, allowing us to establish the required restricted-curvature condition and derive L1, L2, and prediction error rates when the number of regressors grows polynomially with the sample size and may exceed it. Under exact sparsity, the resulting rates recover the familiar rates for sparse high-dimensional estimation. Monte Carlo experiments across a range of high-dimensional VAR designs show that BAR delivers strong estimation and forecasting performance, and retains a high share of relevant coefficients relative to Lasso, adaptive Lasso, and SCAD. An application to U.S. inflation forecasting using FRED-MD shows particularly strong performance at the one-month horizon.
- [51] arXiv:2611.18238 (cross-list from cs.LG) [pdf, html, other]
-
Title: Small Is Not Dead: Distance-Aware Cluster Revival for e-meansSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
e-means (Bessa et al., 2026) is a recent mini-batch k-means algorithm, now part of the FAISS library, that reaches the error of Lloyd's algorithm about an order of magnitude faster on large embedding datasets. Every five mini-batches, it removes every cluster whose running point count falls below a tenth of the average and reuses its centre to split a large cluster. In the authors' ablation, removing this mechanism, called dead-cluster revival, raises the error more than removing any other component added to the moving-average update. The rule, however, considers only cluster size: it cannot distinguish a cluster that is small because it is redundant from one that covers a rare, isolated group of points, and on data with such groups it repeatedly removes centres whose removal is costly. We add a second condition, based on the distance between centres: a small cluster is removed only if its centre is close to another centre, relative to a running estimate of the mean squared error. When a centre equals the mean of its points, this squared distance is an upper bound on the average increase in squared error, per point of the cluster, caused by removing the centre. The condition costs one nearest-centre search for the small clusters every five mini-batches and adds no measurable running time. On the e-means paper's embedding benchmarks, it never prevents a removal, so the results are identical to those of FAISS. On datasets with small isolated groups not used during development, it reduces the test error after two epochs by roughly 10% to 80% relative to count-based revival, and it preserves the advantage of k-means++ initialization; on the other evaluation datasets, it stays within about 1% of the better of revival enabled and disabled. We also report two failure modes that the analysis predicts. This is a CPU proof of concept; GPU implementations are left to future work.
- [52] arXiv:2611.18241 (cross-list from astro-ph.IM) [pdf, html, other]
-
Title: Extended Bayesian Estimation for In-flight Calibration of Space-based InstrumentsSubjects: Instrumentation and Methods for Astrophysics (astro-ph.IM); Applications (stat.AP)
The ongoing calibration of space-based instruments is essential to ensure accurate measurements of physical quantities such as photon flux. The goal is to obtain a unified photon-flux measurement of a single astronomical source using data obtained with multiple instruments. Calibration involves in-flight adjustments to address discrepancies arising from instrument-specific variations. Taking a statistical approach, we extend a previously proposed additive log-Normal hierarchical model with half-variance correction based on a multiplicative physical model for photon counts. The newly proposed Bayesian hierarchical model can leverage domain-specific priors to account for correlated passbands and integrate information across different instruments, sources, and passbands effectively. The model is fitted using Markov chain Monte Carlo and can provide uncertainty quantification for instrument- and source-specific parameter estimation. Moreover, we obtain valid inferences for other structural parameters of the physics model. We use data derived from Chandra and XMM-Newton X-ray observatories to illustrate the proposed inference procedure.
- [53] arXiv:2611.18367 (cross-list from cs.LG) [pdf, html, other]
-
Title: DECODE: Decomposable Environments for COntrolled Sequential-Decision EvaluationDiego Silva Prado, Bruno Caldas de Souza, Tiago Amaral Penteado de Jesus, Ricardo Guimarães Herrmann, Cristiano Breuel, Cristiano Bevitori Maffia De OliveiraSubjects: Machine Learning (cs.LG); Risk Management (q-fin.RM); Machine Learning (stat.ML)
When a sequential decision system falls short of an optimum, its score does not reveal which modifications would improve its decisions. DECODE is a benchmark for testing such modifications through controlled experiments on planning and available information. Its consumer-credit collections generator provides five partially observed environments that share known dynamics matrices and vary a memory parameter and process noise. The experiments change selected inputs or planning rules while keeping the rest of the solver fixed. Each comparison specifies the solver's state inputs, response inputs and planning horizon. On the deterministic base environment, which was selected for a large gap between six-month and one-month planning with the hidden state, extending planning from one to six months increases the scores of two gradient-boosted solvers by 15 to 17 percentage points of the exact optimum, computed with the hidden state and borrower traits. Even at this horizon, the best graded solver, whose repayment response is fitted on logs, remains 15.6 points below that optimum. In a specified intervention sequence, three changes to its state inputs and their evaluation during search lower its score by 6.3 points, and replacing its fitted repayment model with the true response, evaluated with hidden borrower traits, then raises it by 15.6, a net recovery of nine of those points. Because the intermediate effects are conditional on the repayment model and the order of the changes, they do not uniquely assign the gap to separate causes. Within these limits, DECODE makes hypotheses about solver performance testable through specified interventions under known dynamics and explicit information conditions.
- [54] arXiv:2611.18479 (cross-list from econ.EM) [pdf, html, other]
-
Title: Estimating a Rational Inattention Discrete Choice Model: A Semiparametric ApproachComments: 102 pages. Job market paper. Latest version at this https URLSubjects: Econometrics (econ.EM); Methodology (stat.ME)
People's choices depend on both the payoffs of the available options and what they learn about those payoffs before deciding. This paper shows how choice data identify both unobserved payoff heterogeneity and the cost of information acquisition in a rational inattention discrete-choice model. I establish conditions for semiparametric point identification using exogenous payoff and information-cost shifters, without imposing parametric forms on the taste-shock distribution or on how the unit cost of information varies with observed characteristics. In the model with binary choice, I construct an unobserved choice threshold whose cumulative distribution function is the choice probability as a function of the payoff shifter. Moments of this threshold identify the payoff coefficients and the unit cost of information at each value of the information-cost shifter. A deconvolution argument then identifies the taste-shock distribution. I then extend identification to the model with multinomial choice. To estimate the model, I propose a sieve minimum-distance estimator based on observed market shares and establish its consistency. For inference, I construct confidence regions by test inversion because binding parameter constraints can make the estimator's limiting distribution nonnormal. In an application to Amazon Mechanical Turk, model counterfactuals show that lower unit costs of information raise the average task-taking probability and reduce wage elasticity.
- [55] arXiv:2611.18511 (cross-list from cs.LG) [pdf, html, other]
-
Title: On numerical issues of divergences between nearly equal distributionsSubjects: Machine Learning (cs.LG); Numerical Analysis (math.NA); Machine Learning (stat.ML)
Barron et al. (2026) recently identified numerical issues in the softmax and soft cross-entropy loss. We show that similar issues are present in standard divergence operations between categorical distributions. This includes the forward and reverse Kullback-Leibler (KL) divergence and generalized Jensen-Shannon divergence (JSD), as well as Monte Carlo KL estimates and entropy. When the distributions are nearly equal, rounding and cancellation issues affect the forward and gradient computations. These issues can affect settings using divergences in optimization objectives, such as knowledge distillation, reinforcement learning, and proximal methods. We propose numerically stable reformulations and provide example implementations for each setting, validated by property-based tests.
- [56] arXiv:2611.18538 (cross-list from cs.IT) [pdf, html, other]
-
Title: Why Finite Intelligence Works in an Infinite World: A Mathematical TheoryComments: 56 pages, 5 figuresSubjects: Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)
An information system often has to learn before it knows what it will be asked. A finite representation can succeed without recording an infinite environment in full when those asks share collective structure, rather than demand unrelated facts scattered arbitrarily across it. We formalize this phenomenon through future-task information, the minimum retained information needed to preserve decision value relative to the available evidence. A joint decision-regret state records posterior comparisons among actions for every permitted task. Under compactness assumptions, every retained representation admits a calibrated replacement based on this state, preserving all task regrets without increasing information; the resulting information-regret curve equals an operational coding rate. Two exact boundaries govern finite preservation: weak-star closure for a fixed action count, and the singular mass of the source-evidence law when all finite action menus are admitted. Collective structure determines information requirements beyond individual-question difficulty. Families with identical source laws, pairwise task metrics, and individual-task rates can have different high-resolution exponents. Covariance-identical coordinate and parity families have different exact rate-distortion curves, with optimal parity retention characterized by a sharp Fourier-entropy inequality. For balanced nested task families, refinement sizes and weighted reach determine the rate within an additive constant, attained by prefix coding under explicit decay conditions. Nonlinear learning and computational separations distinguish preservation from efficient discovery and use. The proposed mathematical theory identifies which shared decision requirements allow finite information to remain useful for infinitely many possible purposes.
Cross submissions (showing 12 of 12 entries)
- [57] arXiv:2303.02510 (replaced) [pdf, html, other]
-
Title: Testing the equality of the copulas of two multivariate populations with Bernstein polynomialsSubjects: Statistics Theory (math.ST)
The empirical Bernstein copula process is utilized to conduct tests for the equal- ity of the dependence structures between two samples. To this end, three tests statistics are proposed, and their asymptotic properties are established under mild regularity conditions. Furthermore, in order to obtain a valid p-value, sub- sampling and multiplier bootstrap methods are introduced and investigated for the Bernstein based-test statistics. Through a simulation study, it is shown that Bernstein-based tests significantly outperform tests based on the empirical copula.
- [58] arXiv:2403.03228 (replaced) [pdf, html, other]
-
Title: The Spoiler Effect in Multiwinner Ranked-Choice Elections: Implications for Proportional RepresentationSubjects: Methodology (stat.ME)
In the popular debate over the use of ranked-choice voting, it is often claimed that the method of single transferable vote (STV) is immune or mostly immune to the so-called ``spoiler effect,'' where the removal of a losing candidate changes the set of winners. This claim has previously been studied only in the single-winner case. We investigate how susceptible STV is to the spoiler effect in multiwinner elections, where the output of the voting method is a committee of size at least two. To evaluate STV we compare it to numerous other voting methods including single non-transferable vote, $k$-Borda, and the Chamberlin-Courant rule. We provide simulation results under three different random models and empirical results using a large database of real-world multiwinner political elections from Scotland. Our results show that STV is not spoiler-proof in any meaningful sense in the multiwinner context, but it tends to perform well relative to other methods, especially when using real-world ballot data.
- [59] arXiv:2409.06275 (replaced) [pdf, other]
-
Title: On Sparsity and Sub-Gaussianity in the Johnson-Lindenstrauss LemmaAur{é}lien Garivier (UMPA-ENSL, MC2), Emmanuel Pilliat (UMPA-ENSL)Journal-ref: Transactions on Machine Learning Research 2025Subjects: Statistics Theory (math.ST)
We provide a simple proof of the Johnson-Lindenstrauss lemma for sub-Gaussian variables. We extend the analysis to identify how sparse projections can be, and what the cost of sparsity is on the target dimension. The Johnson-Lindenstrauss lemma is the theoretical core of the dimensionality reduction methods based on random projections. While its original formulation involves matrices with Gaussian entries, the computational cost of random projections can be drastically reduced by the use of simpler variables, especially if they vanish with a high probability. In this paper, we propose a simple and elementary analysis of random projections under classical assumptions that emphasizes the key role of sub-Gaussianity. Furthermore, we show how to extend it to sparse projections, emphasizing the limits induced by the sparsity of the data itself.
- [60] arXiv:2501.06844 (replaced) [pdf, html, other]
-
Title: REML implementations of kernel-based genomic prediction models for genotype x environment x management interactionsSubjects: Methodology (stat.ME)
High-throughput pheno-, geno-, and envirotyping allows characterization of plant genotypes and the trials they are evaluated in, producing different types of data. These different data modalities can be integrated into statistical or machine learning models for genomic prediction in several ways. One common approach within the analysis of multi-environment trial data in plant breeding is to create linear or nonlinear kernels which are subsequently used in linear mixed models for genotype by environment interactions. Current implementations of these kernel-based mixed models present several opportunities in terms of methodological extensions. Here we show how these models can be easily implemented in standard software, allowing direct restricted maximum likelihood estimation of all parameters. We also extend the models by combining the kernels with unstructured covariance matrices for three-way interactions in genotype by environment by management datasets, while simultaneously allowing for environment-specific genetic variances and multiple bandwidth parameters. We show how the models incorporating Gaussian kernels, heterogeneous variances, and management-specific bandwidths maximize the amount of genetic variance captured by environmental covariables, following a biologically intuitive pattern. Furthermore, we show that these models perform best in sparse testing prediction settings using two real-word datasets on wheat and maize. We discuss opportunities regarding models with multiple kernels or kernels obtained after feature selection, as well as the similarities to models regressing phenotypes on latent and observed environmental covariables. Finally, we discuss the flexibility provided by our implementation in terms of modeling complex plant breeding datasets, allowing for straightforward integration of phenomics, enviromics, and genomics.
- [61] arXiv:2507.18520 (replaced) [pdf, html, other]
-
Title: Recovering the Euclidean Geometry of High-Dimensional Data Under Heterogeneous NoiseSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
Pairwise Euclidean distances provide a simple geometric representation of high-dimensional data and underlie many data analysis methods, including clustering, visualization, and manifold learning. However, in real-world applications, these distances can be severely corrupted by heterogeneous noise, where different data points are subject to different, and potentially large, noise levels. Such noise distorts the distances inhomogeneously, leading to a misrepresentation of the underlying data geometry. In this work, we ask whether the underlying clean Euclidean geometry can nevertheless be recovered from the corrupted pairwise distances. We answer this question affirmatively, showing that, perhaps surprisingly, such recovery is possible across a broad class of high-dimensional settings with underlying low-dimensional geometry, even when the noise magnitudes vary substantially across observations. We develop a principled approach for recovering the clean pairwise distances, without prior knowledge of either the clean data structure or the noise distribution, and establish probabilistic recovery guarantees. In particular, the recovery error converges to zero at polynomial rates as the feature dimension and dataset size increase simultaneously. Experiments on synthetic datasets demonstrate accurate recovery of the Euclidean geometry in challenging regimes and improved robustness of downstream methods that rely on pairwise distances. When applied to real single-cell RNA sequencing data, our method reliably recovers local neighborhood structure, enabling accurate nearest-neighbor identification used in many downstream analyses.
- [62] arXiv:2507.22472 (replaced) [pdf, html, other]
-
Title: Inference in a generalized Bradley-Terry model for paired comparisons with covariates and a growing number of subjectsComments: Major revisions to theoretical results and numerical studiesSubjects: Methodology (stat.ME)
Motivated by the home-field advantage in sports, we propose a generalized Bradley--Terry model that incorporates covariate information for paired comparisons. It has an $n$-dimensional merit parameter $\bs{\beta}$ and a $p$-dimensional regression coefficient $\bs{\gamma}$ for covariates. When the number of subjects $n$ approaches infinity and the number of comparisons between any two subjects is fixed, we show the consistency of the maximum likelihood estimator (MLE) $(\widehat{\bs{\beta}}, \widehat{\bs{\gamma}})$ of $(\bs{\beta}, \bs{\gamma})$ in cases that $p$ is fixed or diverges with $n$. Furthermore, when $p$ is fixed, we derive the asymptotic normal distribution of the MLE by characterizing its asymptotic representation. The asymptotic distribution of $\widehat{\bs{\gamma}}$ is biased, while that of $\widehat{\bs{\beta}}$ is not. This phenomenon can be attributed to the different convergence rates of $\widehat{\bs{\gamma}}$ and $\widehat{\bs{\beta}}$. The consistency result is further extended to an Erdős--Rényi comparison graph. Numerical studies and a real data analysis demonstrate our theoretical findings.
- [63] arXiv:2510.07559 (replaced) [pdf, html, other]
-
Title: General coupling-based f-divergence diagnostics for Markov chain Monte CarloComments: Significantly revised manuscript including new nonasymptotic boundsSubjects: Computation (stat.CO); Probability (math.PR); Statistics Theory (math.ST); Methodology (stat.ME)
A long-standing gap exists between the theoretical analysis of Markov chain Monte Carlo convergence, which is often based on statistical divergences, and the diagnostics used in practice. We introduce the first general convergence diagnostics for Markov chain Monte Carlo based on a broad class of f-divergence, allowing users to directly monitor, among others, the Kullback-Leibler and the chi-squared divergences as well as the total variation distance. Our approach rests on a coupling-based 'weight harmonization' scheme that produces direct, computable, and consistent importance weights for interacting Markov chains with respect to their target distribution. Beyond their use as convergence diagnostics, these weights give unbiased estimates of the Radon-Nikodym derivative between the un-normalized target and the distribution of the current iterate, a richer object than the convergence bounds alone, with natural applications to importance-weighted inference. We show how such weightings can provide asymptotic upper bounds to any f-divergence, prove that these bounds converge to zero as the chains approach stationarity, and demonstrate that, while more conservative than existing coupling-based total variation estimators, our method remains a practical and broadly applicable diagnostic tool.
- [64] arXiv:2512.01817 (replaced) [pdf, html, other]
-
Title: Self-Normalized Bounds for Time Series with Sample Variance Only and Their Application to ForecastingSubjects: Statistics Theory (math.ST)
Evaluating a forecasting rule on a time series requires a risk bound that can be computed from the observed losses. Such generalization bounds rest on empirical Bernstein (EB) and self-normalized inequalities, which replace the unknown variance by a sample counterpart. These tools, however, have been developed almost exclusively for independent observations and martingale differences, and, for dependent data, the natural setting of forecasting, direct results are scarce. They require $\beta$-mixing and knowledge of the mixing coefficients, and their leading terms are not sharp. This paper fills this gap. We first establish, for martingale differences, an EB inequality in closed form that uses only the realized quadratic variation, holds with probability at least $1-\alpha$ for every sample size, and is sharp under heteroscedasticity, whereas the sharpness of existing EB inequalities is established for a constant conditional variance. Building on this, we derive EB and self-normalized inequalities for the marginal mean of $\tilde{\phi}$- and $\tau$-mixing processes, a class that contains processes that are not even strongly mixing, such as autoregressions with Bernoulli innovations. The radii use nothing but a block sample variance: no variance proxy has to be designed or tuned, and, for $\tau$-mixing processes, no dependence coefficient enters the radius. Under summability-type conditions on the dependence coefficients, their leading terms carry the constant of the oracle Bernstein inequality with the long-run variance, so, to our knowledge, these are the first sharp self-normalized inequalities under non-strong mixing conditions. As an application, we obtain a sharp upper bound on the risk of a time-series forecasting rule that is computed from its observed losses with a single block length and contains no dependence coefficient.
- [65] arXiv:2512.05024 (replaced) [pdf, html, other]
-
Title: Model-Free Assessment of Simulator Fidelity via Quantile CurvesComments: Accepted to NeurIPS 2026Subjects: Methodology (stat.ME); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
As generative AI models are increasingly used to simulate real-world systems, quantifying the ``sim-to-real'' gap is critical. For each input setting of interest -- which we call a \emph{scenario}, such as a survey question or operating condition -- the real and simulated systems are associated with unobserved latent population parameters, and their discrepancy varies across scenarios. A fundamental challenge is that, for any given scenario, this discrepancy cannot be observed directly, since both systems are accessible only through finite samples, often of heterogeneous sizes across scenarios. Standard predictive inference methods are therefore ill-suited, as they quantify uncertainty in observable outputs rather than latent population parameters. To address this, we construct confidence sets for these latent parameters and use them to derive a robust proxy for the sim-to-real discrepancy. We then estimate the quantile function of this proxy to obtain a distribution-level risk profile of the simulator, which supports a broad range of statistical summaries, including statistical inference for the real output distribution in a new scenario, the calculation of risk measures like Conditional Value-at-Risk (CVaR), and principled comparisons across simulators. Our method is model-agnostic and handles general output spaces, such as categorical survey responses and continuous multi-dimensional data. We demonstrate the practical utility of this method by evaluating the alignment of four major LLMs with human populations on the WorldValueBench dataset.
- [66] arXiv:2602.16265 (replaced) [pdf, other]
-
Title: On sparsity, extremal structure, and monotonicity properties of Wasserstein and Gromov-Wasserstein optimal transport plansTitouan Vayer (COMPACT)Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
This note gives a self-contained overview of some key properties of the Gromov-Wasserstein (GW) distance, compared with the standard linear optimal transport (OT) framework. More specifically, I explore the following questions: are GW optimal transport plans sparse? under what conditions does there exist an optimal plan supported on a permutation? do optimal plans satisfy a form of cyclical monotonicity? In particular, I present the conditionally negative/positive definite (CND/CPD) properties and show that, when they hold, there exist GW optimal plans that are sparse and supported on a permutation. In the last section, I also give examples where no permutation is optimal when the CND or CPD assumption is dropped.
- [67] arXiv:2603.13662 (replaced) [pdf, html, other]
-
Title: Fast Uncertainty Quantification for Kernel-Based Estimators in Large-Scale Causal InferenceComments: 36 pagesSubjects: Methodology (stat.ME); Applications (stat.AP); Machine Learning (stat.ML)
Kernel methods are widely used in causal inference for tasks such as treatment effect estimation, policy evaluation, and policy learning. The bootstrap is a standard tool for uncertainty quantification because of its broad applicability. As increasingly large datasets become available, such as the 2023 U.S. Natality data from the National Vital Statistics System (NVSS), which includes 3,605,081 registered births (of which 2,980,739 form our analytic sample after cohort restrictions), the computational demands of these methods increase substantially. Kernel methods are known to scale poorly with sample size, and this limitation is further exacerbated by the repeated re-fitting required by the bootstrap. As a result, bootstrap-based inference for kernel-based estimators can become computationally infeasible in large-scale settings. In this paper, we address these challenges by extending the causal Bag of Little Bootstraps (cBLB) algorithm of Kosko et al. (2023) to kernel methods. Our approach achieves computational scalability by combining subsampling and resampling while preserving first-order uncertainty quantification and asymptotically correct coverage. We evaluate the method across three representative implementations: kernelized augmented outcome-weighted learning, kernel-based minimax weighting, and double machine learning with kernel support vector machines. We show in simulations that our method yields confidence intervals with nominal coverage at a fraction of the computational cost. We further demonstrate its utility in a real-world application by estimating the effect of any amount of smoking on birth weight, as well as the optimal treatment regime, using the NVSS dataset, where the standard bootstrap is prohibitively expensive computationally and effectively infeasible at this scale.
- [68] arXiv:2603.20891 (replaced) [pdf, html, other]
-
Title: Auto-differentiable data assimilation: Co-learning of states, dynamics, and filtering algorithmsSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Signal Processing (eess.SP); Dynamical Systems (math.DS)
Data assimilation algorithms estimate the state of a dynamical system from partial observations, where the successful performance of these algorithms hinges on costly parameter tuning and on employing an accurate model for the dynamics. This paper introduces a framework for jointly learning the state, dynamics, and parameters of filtering algorithms in data assimilation through a process we refer to as auto-differentiable filtering. The framework leverages a theoretically motivated loss function that enables learning from partial, noisy observations via gradient-based optimization using auto-differentiation. We further demonstrate how several well-known data assimilation methods can be learned or tuned within this framework. To underscore the versatility of auto-differentiable filtering, we perform experiments on dynamical systems spanning multiple scientific domains, such as the Clohessy-Wiltshire equations from aerospace engineering, the Lorenz-96 system from atmospheric science, the generalized Lotka-Volterra equations from systems biology, and the two-layer quasi-geostrophic model from geophysical fluid dynamics. Finally, we provide guidelines for practitioners to customize our framework according to their observation model, accuracy requirements, and computational budget.
- [69] arXiv:2604.18152 (replaced) [pdf, other]
-
Title: mlr3torch: A Deep Learning Framework in R based on mlr3 and torchSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Deep learning (DL) has become a cornerstone of modern machine learning (ML) praxis. We introduce the R package mlr3torch, which is an extensible DL framework for the mlr3 ecosystem. It is built upon the torch package, and simplifies the definition, training, and evaluation of neural networks for both tabular data and generic tensors (e.g., images) for classification and regression. The package implements predefined architectures, and torch models can easily be converted to mlr3 learners. It also allows users to define neural networks as graphs. This representation is based on the graph language defined in mlr3pipelines and allows users to define the entire modeling workflow, including preprocessing, data augmentation, and network architecture, in a single graph. Through its integration into the mlr3 ecosystem, the package allows for convenient resampling, benchmarking, preprocessing, and more. We explain the package's design and features and show how to customize and extend it to new problems. Furthermore, we demonstrate the package's capabilities using three use cases, namely hyperparameter tuning, fine-tuning, and defining architectures for multimodal data. Finally, we present some runtime benchmarks.
- [70] arXiv:2604.24652 (replaced) [pdf, html, other]
-
Title: Benefits and Costs of Adaptive SamplingComments: 60 pages, 9 figuresSubjects: Methodology (stat.ME); Econometrics (econ.EM)
Multi-armed bandits are widely used for sequential experimentation in clinical trials, public policy treatment, recommendation systems, and online platforms. While regret minimization and valid inference from adaptively collected data have each been studied individually, one question remains: when does adaptivity \emph{improve estimation precision} relative to uniform designs, and how should inference be balanced against the online cost of experimentation? We first study arm-level mean estimation under mean-squared-error (MSE) objectives. We characterize when an adaptive Neyman allocation, which allocates samples according to arm variance, yields strict MSE improvements over uniform sampling in finite samples. When there is variance heterogeneity across arms, these improvements arise at modest sample sizes, clarifying that adaptivity can be preferable for inference not only asymptotically, but also in finite-sample settings. We then study a joint inference--regret objective that accounts for the cost of assigning units to inferior arms during experimentation. We propose three policies: the Plug-in Oracle Rate Policy, Static-Allocation Rate Policy, and Neyman-Adaptive Rate Policy, all of which attain the oracle-optimal asymptotic rate. The latter two offer computationally efficient and easy-to-implement alternatives by combining existing regret-minimizing algorithms with inference-oriented sampling through closed-form mixing rules, avoiding repeated numerical optimization while being managerially interpretable. We compare their finite-sample statistical performance through simulations and empirically calibrated applications. The results highlight the relative strengths of our policies across instances and their tradeoffs between joint inference--regret performance and computational cost.
- [71] arXiv:2605.03300 (replaced) [pdf, html, other]
-
Title: Smoothed estimation of Wasserstein barycentersSubjects: Statistics Theory (math.ST)
This paper studies the statistical estimation of exact Wasserstein barycenters. Existing non-asymptotic results for empirical barycenters exhibit a severe curse of dimensionality. Motivated by the semi-dual formulation of the barycenter problem and its associated Sobolev optimization geometry, we develop a smoothness-aware approach that combines density estimation with Sobolev geometric structure to estimate the population barycenter. We establish nonparametric convergence rates for estimating both the barycenter functional and its minimizer, demonstrating how smoothness can substantially improve statistical performance.
- [72] arXiv:2606.13850 (replaced) [pdf, html, other]
-
Title: Exact Constrained Decoding for Sequence Models: Controller-Augmented Hidden Markov Models and BeyondSubjects: Methodology (stat.ME); Machine Learning (stat.ML)
Hidden Markov models are foundational for sequential inference, yet conditioning on pathwise constraints such as precedence requirements, visitation cardinalities, or monotonic state progression can introduce long-range dependencies that require additional memory for dynamic programming. To deal with this, we present Controller-Augmented Hidden Markov Models (CHMMs), a framework that compiles each constraint into a finite-state controller tracking a sufficient finite history, after which standard forward--backward and Viterbi recursions on the augmented chain compute exact constrained posteriors and maximum a posteriori paths in discrete time and, through uniformization, maximum a posteriori augmented-state sequences at observation times in continuous time. While we establish several theoretical guarantees, a catalog of controller encodings for 11 constraint families operationalizes our framework. Empirically, we evaluate classical and neural decoders with and without controller augmentation on 3 real-world sequence-labeling tasks: gene-structure decoding in \emph{Drosophila melanogaster}, free-living activity recognition in CASAS smart homes, and protocol-defined activity recognition from wearable sensors. Together, theory and experiment characterize when exact controller augmentation is useful and when simpler approaches suffice.
- [73] arXiv:2608.05434 (replaced) [pdf, html, other]
-
Title: Risk-Aware Quantile Learning for Personalized Dynamic Treatment RegimesComments: An earlier version of this work was accepted at STAI-X 2026Subjects: Methodology (stat.ME); Statistics Theory (math.ST); Machine Learning (stat.ML)
Sequential clinical decision-making often involves more than maximizing average efficacy. Clinicians may need to simultaneously optimize clinically relevant tails of the outcome distribution, control treatment-related risk, and choose among multiple treatment options. Existing quantile dynamic treatment regime (DTR) methods capture distributional features of treatment outcomes but remain largely restricted to efficacy-only objectives and binary treatments. To address these limitations, we propose Risk-Aware Quantile Dynamic Treatment Regimes (RQDTR), a unified framework that optimizes a prespecified quantile of the cumulative potential outcome while explicitly incorporating treatment-related risk. We also develop an angle-based formulation for jointly learning decision rules across multiple treatment categories. Our framework includes three interpretable subclasses: efficacy-only quantile learning, constraint-based learning with population-level risk control, and utility-based learning through a composite benefit-risk utility. Theoretically, we establish identification and oracle equivalence, surrogate and estimator consistency under increasing quadratic penalties, asymptotic risk feasibility, and finite-sample performance error bounds. Extensive simulation studies and applications to All of Us major depressive disorder and MIMIC-III sepsis data demonstrate that RQDTR improves tail-oriented efficacy and achieves more favorable benefit-risk trade-offs than existing quantile DTR methods.
- [74] arXiv:2609.30106 (replaced) [pdf, html, other]
-
Title: Stopping models closed under pgf composition, and the stability of randomly stopped model extensionsComments: 45 pagesSubjects: Statistics Theory (math.ST); Probability (math.PR)
Statistical model transformations based on randomly stopped sums, maxima, and minima are widely used to extend statistical models by introducing an additional parameter. We characterize the finite-dimensional positive stopping models for which these randomly stopped sum and extreme transformations act as statistically stable (idempotent) model extensions. Stability requires the underlying stopping model to be closed under pgf composition. We prove that any finite-dimensional, connected, positive stopping model closed under pgf composition necessarily forms a family of random variables whose pgfs commute. Using the associated Koenigs function, we establish that these models form a statistical manifold admitting a global, one-dimensional parametrization $\theta = \Pr(N=1) \in (0, \theta_*]$, in which the probability mass at $i$ is a polynomial in $\theta$ of degree at most $i$. Within this class, we further establish a duality between those models containing the identity variable (the ones yielding statistically stable extensions) and the set of all probability distributions supported on the positive integers. By providing numerous examples of stopping models closed under pgf composition, we clarify the assertion that, under certain conditions, the only stopping model that is so is geometric.
- [75] arXiv:2609.30190 (replaced) [pdf, html, other]
-
Title: Learning the Maximum Tolerated Dose for Continuous Toxicity via Monotone Bayesian TreesSubjects: Applications (stat.AP)
Phase I cancer trials seek the maximum tolerated dose (MTD) while protecting patients from excessive toxicity. Dose assignments must therefore balance patient safety with learning the dose--toxicity relationship as data accrue. We model continuously measured toxicity outcomes using two forms of Bayesian additive regression trees (BART): isotonic BART projects posterior response curves onto nondecreasing functions, whereas monotone BART constrains the model. Joint curve and variance draws induce an MTD posterior that guides dose selection through escalation with overdose control (EWOC). We compare these methods with three parametric procedures across seven dose--toxicity curves in simulation. We assess dose-limiting toxicity (DLT) counts, above-MTD assignments, signed last-dose error, and relative absolute error. The tree procedures attained the lowest mean RAE on four nonlinear curves and jointly minimized mean DLT counts and above-MTD assignments on four curves. A Bayesian reinforcement learning perspective formulates these sequential decisions as a finite-horizon planning problem. Dose restrictions yield a lower bound on last-dose error; under exact posterior-predictive evaluation, an EWOC-based rollout policy has no greater expected weighted loss than its baseline. Dose Trial Lab, a desktop simulator for all five procedures, accompanies the supplementary material.
- [76] arXiv:2609.38524 (replaced) [pdf, html, other]
-
Title: Generative sequence modeling for infinite memory processes via predictive statesSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)
We consider estimating the one-step-ahead conditional distribution of a multivariate stochastic process. Many existing approaches rely on assumptions such as finite-range memory, sparsity, or additivity, which can be poorly suited to processes with long-range nonlinear interactions. However, without such structural assumptions, nonparametric estimation is challenging due to the curse of dimensionality. To address this challenge, we introduce a new estimator based on learning representations of the \emph{predictive states} of a stochastic process. We show that the statistical complexity of the estimation problem is determined by the dimension of this predictive state representation. Furthermore, we show that a $p$-dimensional predictive state space can be represented in a $2p+1$-dimensional feature space. Our results imply that faster convergence rates are achievable when the predictive state space is lower-dimensional, i.e., when the past history can be compressed into a lower-dimensional statistic that is sufficient for predicting the future. We establish theoretical convergence rate guarantees for an instantiation of our method based on deep neural network estimators. We implement our method in PyTorch and evaluate its performance on synthetic data and real \emph{C. elegans} data.
- [77] arXiv:2610.04474 (replaced) [pdf, html, other]
-
Title: Personalized Federated Vector Autoregression with Personalization DiversitySubjects: Methodology (stat.ME)
Personalized federated estimation of high-dimensional time series faces two challenges: identifying shared dynamics and retaining the statistical gains from pooling. We address these challenges through personalization diversity in a vector autoregressive model with shared low-rank dynamics, shared sparse links, and personalized sparse departures. A cross-client support-frequency cap yields a sharp majority threshold for shared-personalized identification, while an additional cross-client $\ell_q$ condition captures approximate diversity and controls approximation under a smaller working personalization frequency cap. We develop PerFeCT-VAR, Personalized Federated Vector Autoregression via Frequency-Capped Thresholding, which jointly enforces clientwise sparsity and cross-client frequency constraints. Under suitable sampling and cross-component compatibility conditions, we establish initialization guarantees and joint local linear convergence for heterogeneous stationary client distributions. Our componentwise error bounds show that diversity attenuates both estimation and approximation errors transmitted from personalized to shared components, and quantify the tradeoff induced by the working cap. These bounds identify regimes in which the shared components attain pooled accuracy despite non-vanishing personalized effects, while personalized estimation achieves local-sample accuracy on average. Simulations and an application to multistore retail revenue data illustrate the estimation, forecasting, and interpretive benefits of the proposed framework.
- [78] arXiv:2610.07740 (replaced) [pdf, html, other]
-
Title: High-Dimensional Online Calibration from Harmonic WeightsComments: In-depth talk about this work: this https URLSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
We study the online calibration of multidimensional forecasts over an arbitrary convex set $Y\subseteq\mathbb{R}^d$ relative to an arbitrary error norm $\|\cdot\|_L$. For forecasting $d$ binary outcomes simultaneously ($Y=[0,1]^d$), we give the first algorithm that achieves $\varepsilon$-calibration in a number of rounds that is polynomial in $d$ for every fixed accuracy. It requires $d^{O(1/\varepsilon)}$ rounds, exponentially improving the dimension dependence of previous bounds. For multi-class forecasting ($Y=\Delta_d$), we obtain the same $d^{O(1/\varepsilon)}$ rate, improving the $d^{\widetilde{O}(1/\varepsilon^2)}$ bounds of Peng and Fishelson, Golowich, Mohri, and Schneider.
Our algorithm is simple: on each round, it outputs a harmonically weighted distribution over harmonically smoothed past outcomes. The same algorithm works for every forecast set and error norm. More generally, it achieves $\varepsilon$-calibration after $\exp(O(\gamma(Y,L)/\varepsilon))$ rounds, where $\gamma(Y,L)$ is a geometric parameter defined by a matrix discrepancy problem. The harmonic weights are motivated by the fact that the discrete Hilbert transform matrix achieves the optimal discrepancy up to a universal constant, simultaneously for every $L$. This optimality result may be of independent interest.
The same harmonic responses also yield an algorithm for high-dimensional Blackwell approachability. Its guarantees are governed by a sharpened geometric parameter that captures the joint geometry of the outcome and action sets. - [79] arXiv:2610.08157 (replaced) [pdf, other]
-
Title: TEASE: Targeted elastic spatial envelopeSubjects: Methodology (stat.ME)
Envelope regression is a crucial part of multivariate linear modeling, leveraging separation into material and immaterial parts to provide parsimonious data reduction. Rekabdarkolaee et al. (2020) construct spatial envelope, a novel multivariate Gaussian process, extended by May et al. (2022) to linear coregionalization envelope. Our proposal (TEASE) combines spatial envelope and full-scale basis graphical lasso (LeDuc et al., 2025), leveraging targeted graphical elastic net (Kovács et al., 2021) for material precision matrices. Immaterial part is a multivariate multiresolution Gaussian Markov random field (Kleiber et al., 2019; Caringi and Secchi, 2026). Enhanced envelope penalty (Kwon and Zou, 2025) regularizes regression coefficients. Demonstrations are on climate data.
- [80] arXiv:2611.07219 (replaced) [pdf, html, other]
-
Title: Neyman Orthogonalized Specification Tests for Conditional Tail Index ModelsSubjects: Statistics Theory (math.ST); Methodology (stat.ME)
We develop specification tests for parametric conditional tail index models of heavy-tailed distributions. The tests are based on a projected empirical process that removes the first-order effect of estimating the null parameter. Within the conditional Hall class, we identify conditions under which projection also eliminates the leading Pareto approximation bias in the test process, permitting lower thresholds while preserving asymptotically correct size. We establish the validity of the tests via multiplier bootstrap and characterize power against fixed and local alternatives. For log-linear models, bootstrap validity extends to settings in which covariates of observations above a common threshold concentrate and the smallest eigenvalue of the information matrix tends to zero. We further propose a novel threshold selection rule based on an estimate of the Pareto approximation bias after projection. The rule uses the estimated bias to determine whether a reference threshold can be lowered to retain more observations, with potential gains in power. The finite sample performance of the tests is examined through simulations and a real data application.
- [81] arXiv:2611.11985 (replaced) [pdf, html, other]
-
Title: Trajectory Density Ratios under Weak Support: A Survey across the SciencesSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Many problems across the sciences compare two stochastic processes through the sequences they generate. Reinforcement learning asks what a new policy would return from the trajectories of an old one; molecular simulation asks how much more a ligand binds from samples of the unbound state; climate attribution asks how a heat wave shifted under warming; and language-model post-training asks what a deployed agent will do from logged interactions. In every case, one quantity would answer all these questions at once: the ratio of the two probability laws over their generated sequences, the trajectory density ratio. This ratio exists only under strong conditions. When one process reaches states the other cannot, the ratio does not exist; when their overlap is thin, its variance grows without control with the sequence length; and when they concentrate on different lower-dimensional sets, the ratio is undefined altogether. Weak support of this kind is the norm for sequence data, not a sampling accident, and its price is paid in identifiability rather than in noise alone. This survey organizes the field around a three-level diagnosis of weak support and a map of the nine retreats that methods take when the full ratio is out of reach, from direct estimation and reweighting through staged decomposition, stationarization, and partial-identification bounds to smoothing, transport couplings, operator spectra, and direct estimation of a single functional. Each retreat keeps part of what the ratio offers and pays for the rest, and each application domain has standardized the retreat its data force. The survey closes with open problems: testable diagnostics at every level, partial identification at the trajectory level, a rate theory of time-amplified error, and benchmarks with ground truth.
- [82] arXiv:2611.14785 (replaced) [pdf, html, other]
-
Title: Semiparametric Inference for Marginal Win Odds with Censored Time-to-Event OutcomesSubjects: Methodology (stat.ME)
Win statistics provide clinically interpretable measures for comparing treatment groups with prioritized composite outcomes, but efficient covariate-adjustment methods for win statistics with right censored time-to-event outcomes remain limited. We develop a semiparametric framework for marginal win probability and win odds with prioritized time-to-event outcomes, including semi-competing risks, while also accommodating scalar survival outcomes as a special case. The estimand compares independent draws from the marginal distributions of the potential outcomes under treatment and control, which preserves a marginal causal interpretation. We derive the efficient influence function under conditional independent censoring, allowing censoring to depend on baseline covariates and predictable event history. The resulting cross-fitted one-step estimator incorporates covariate adjustment and censoring martingale augmentation, allows flexible estimation of nuisance parameters, and attains the semiparametric efficiency bound under suitable conditions on consistency and convergence rate of nuisance parameters. Simulation studies show that the estimator has small bias, accurate standard error estimation and confidence interval coverage, and substantial efficiency gain relative to the inverse-probability-of-censoring weighted (IPCW) estimator. An application to a randomized breast cancer trial comparing tamoxifen combined with radiotherapy against tamoxifen alone demonstrates improved precision, with an approximately 25\% reduction in estimated variance at five years.
- [83] arXiv:2309.06305 (replaced) [pdf, html, other]
-
Title: Sensitivity Analysis for Linear EstimatorsComments: Previously circulated as Sensitivity Analysis for Linear EstimandsSubjects: Econometrics (econ.EM); Methodology (stat.ME)
We propose a framework for building sensitivity analyses for linear estimators. We measure the degree of identification failure by the change of measure between the observed distribution and a hypothetical target distribution under which a standard estimator would identify the parameter of interest. Under L_infinity bounds on that derivative, the identified set has closed-form sharp bounds that require no numerical optimization and that are computable and continuous when the derivative is unbounded, nesting Manski-type worst-case bounds and existing characterizations that require bounded changes of measure. The framework nests several selection models for average potential outcomes, yielding the established worst-case bounds of each design. The framework yields a new model family for regression discontinuity (RD) that delivers the first sensitivity analysis on the conditional average treatment effect under manipulation. We show that a bootstrap procedure yields valid confidence intervals. An empirical application shows that a modest restriction on manipulation delivers bounds far more informative than worst-case bounds.
- [84] arXiv:2410.06191 (replaced) [pdf, other]
-
Title: Benign Overfitting for Regression with Trained Two-Layer ReLU NetworksComments: There were fatal errors in the proofs. For the updated version, see https://arxiv.org/abs/2505.11621Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
We study the least-square regression problem with a two-layer fully-connected neural network, with ReLU activation function, trained by gradient flow. Our first result is a generalization result, that requires no assumptions on the underlying regression function or the noise other than that they are bounded. We operate in the neural tangent kernel regime, and our generalization result is developed via a decomposition of the excess risk into estimation and approximation errors, viewing gradient flow as an implicit regularizer. This decomposition in the context of neural networks is a novel perspective of gradient descent, and helps us avoid uniform convergence traps. In this work, we also establish that under the same setting, the trained network overfits to the data. Together, these results, establishes the first result on benign overfitting for finite-width ReLU networks for arbitrary regression functions.
- [85] arXiv:2505.17965 (replaced) [pdf, html, other]
-
Title: Bias-Optimal Bounds for Stochastic Gradient DescentComments: 30 pages. Under reviewSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)
The non-asymptotic analysis of Stochastic Gradient Descent (SGD) typically yields bounds that decompose into a bias term and a variance term. In this work, we focus on the bias component and study the extent to which SGD can match the optimal convergence behavior of deterministic gradient descent. Assuming only (strong) convexity and smoothness of the objective, we derive new bounds for the full range of constant step-sizes $\gamma L \in (0,2)$. This includes critical and large step-size regimes that were previously unexplored without additional variance assumptions, in both the convex and strongly convex cases. Those bounds are bias-optimal, in the sense that the bias term coincides with the worst-case rate of gradient descent, for all non-critical step-sizes. For critical step-sizes, we show that no bounds can be obtained with an optimal bias term, but that this optimal bias can be approximated arbitrarily closely. As an immediate application, we provide new complexity bounds for the Stochastic Proximal algorithm, without the need of regularity assumptions.
- [86] arXiv:2506.04948 (replaced) [pdf, other]
-
Title: Entropic smoothing and gradient approximation for Wasserstein distributionally robust optimizationTam Le (LPSM (UMR\_8001))Subjects: Optimization and Control (math.OC); Machine Learning (stat.ML)
Wasserstein distributionally robust optimization offers a framework for model fitting in machine learning under potential shifts in the data distribution. We study a regularized variant of this problem in which entropic smoothing produces a sampled approximation of the original objective. We establish convergence of the approximate gradients to subgradients of the unregularized objective as the regularization parameter vanishes, enabling convergence guarantees for stochastic gradient methods. We obtain qualitative convergence results under general assumptions and provide convergence rates under additional regularity. In particular, we prove rates for the convergence of the unregularized objective values, up to sampling errors, when the regularization level is decreased across iterations. Our analysis also yields results of independent interest on approximating the subdifferentials of maximum functions by gradients of their smooth approximations, together with growth-based lower bounds on optimal dual variables in Wasserstein distributionally robust optimization.
- [87] arXiv:2509.05894 (replaced) [pdf, html, other]
-
Title: Toric geometry of ReLU neural networksSubjects: Algebraic Geometry (math.AG); Machine Learning (cs.LG); Machine Learning (stat.ML)
Given a continuous piecewise linear function $f:\mathbb{R}^{n_0} \to \mathbb{R}$ and a fixed architecture of feedforward ReLU neural networks, the exact function realization problem asks when a network of that architecture realizes $f$. In contrast, expressivity concerns characterizing all functions exactly realizable by a given architecture. To address these questions systematically, we establish a connection between toric geometry and ReLU neural networks, allowing tools from algebraic geometry to analyze the structures. Starting with an unbiased ReLU network with rational weights, we define the ReLU fan, the ReLU toric variety, and the ReLU Cartier divisor associated with the network. This work also relates the tropical and the toric geometry: the former captures combinatorial data of ReLU networks, while the latter recovers the underlying geometric structure. As an application, we obtain a complete characterization of the functions realizable by unbiased shallow ReLU networks with rational weights. The characterization is expressed in terms of intersection numbers of ReLU Cartier divisors with torus-invariant curves and leads to a decomposition of piecewise-linear functions into shallow and non-shallow components.
- [88] arXiv:2510.10140 (replaced) [pdf, html, other]
-
Title: Adversarial Attacks on Downstream Weather Forecasting Models: Application to Tropical Cyclone Trajectory PredictionComments: Compared with the previous version, we (a) extend the downstream attack to an end-to-end attack; (b) further analyze physical feasibility and attack difficulty; and (c) develop a defense mechanismSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML)
Deep learning-based weather forecasting (DLWF) models leverage past weather observations to generate future forecasts, supporting a wide range of downstream applications, including tropical cyclone (TC) prediction. In this paper, we investigate the vulnerability of these models to adversarial attacks, where subtle perturbations to upstream forecasts can alter downstream TC trajectory predictions. Crafting perturbed inputs to steer downstream outputs toward attacker-specified trajectories is a challenging problem for several reasons. First, conventional TC detection systems are opaque, non-differentiable black boxes, making standard gradient-based attacks infeasible. Second, the extreme rarity of TC events leads to a severe class imbalance problem, making it difficult to develop attack methods that produce realistic-looking cyclone paths aligned with target trajectories. To overcome these limitations, we propose Cyc-Attack, a novel method for perturbing the forecasts of DLWF models to generate adversarial trajectories. The proposed method uses a differentiable surrogate model to approximate the TC detector's output, enabling the application of gradient-based attacks. Cyc-Attack also employs a skewness-aware loss function with a kernel dilation strategy to address the severe class imbalance problem. Distance-based gradient weighting and regularization strategies are also introduced to constrain perturbations to eliminate unrealistic-looking TC paths. Our experiments show that Cyc-Attack achieves a higher true positive rate in matching the target trajectories, along with lower false alarm rates and stealthier perturbations than conventional attack methods. We further demonstrate that the adversarial upstream forecasts generated by Cyc-Attack can be leveraged as initial guidance to train an end-to-end model for learning adversarial input samples that would induce target trajectories.
- [89] arXiv:2510.22953 (replaced) [pdf, html, other]
-
Title: Manifold Approximation leads to Robust Kernel AlignmentComments: 9 pages, 5 figures + supplementaryJournal-ref: Transactions on Machine Learning Research, 2026Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
Centered kernel alignment (CKA) is a popular metric for comparing representations, determining equivalence of networks, and conducting neuroscience research. However, CKA does not account for the underlying manifold and relies on numerous heuristics that cause it to behave differently at different scales of data. In this work, we propose Manifold-approximated Kernel Alignment (MKA), which incorporates manifold geometry (i.e., local neighborhood) into the alignment task. We derive a theoretical framework for MKA. Our construction removes representation-dependent degree weighting, yields a closed-form alignment score, and gives each row a probabilistic interpretation as a local Markov transition distribution. We further establish kernel- and score-level stability with respect to the neighborhood size. We perform empirical evaluations on synthetic datasets and real-world examples to characterize and compare MKA to its contemporaries. In controlled synthetic experiments, MKA is consistently less sensitive to hyperparameters and captures the selected topology-preserving relationships. On the large-scale benchmark, MKA achieves its strongest aggregate performance in vision, is competitive on graphs, and is weaker than CKA in several NLP evaluations. Our findings suggest that manifold-aware kernel alignment provides a robust foundation for measuring representations and is potentially useful when local neighborhood geometry is important, with possible applications in representation learning.
- [90] arXiv:2511.12069 (replaced) [pdf, other]
-
Title: A GNN-Based Approach for Code Smell Detection and Refactoring Opportunity IdentificationComments: We would like to withdraw our manuscript to make further revisions and improvements to its content and experimental analysis. We apologize for any inconvenience and appreciate your understandingSubjects: Software Engineering (cs.SE); Methodology (stat.ME)
Code smells indicate latent design or implementation problems that may negatively affect software maintainability. Over the past decades, numerous approaches have been proposed for code smell refactoring, including those based on software metrics, predefined rules, machine learning/deep learning (ML/DL), and, more recently, large language models (LLMs). However, traditional metric- and rule-based approaches rely heavily on manually defined criteria, while ML/DL approaches are often constrained by the availability of training data and the design of model architectures. Although LLMs provide advanced code understanding capabilities, they typically incur substantial computational and time costs, particularly when applied to large-scale software systems. In this study, we propose a graph-based deep learning approach for code smell refactoring. Specifically, we construct code graphs at both the class and method levels and employ graph- and node-level classification to detect code smells and identify corresponding refactoring opportunities for three representative code smells: Long Method, Large Class, and Feature Envy. The proposed approach is trained and evaluated using SACS, a large-scale open-source dataset constructed through a semi-automated process. In the experiments, we implement the proposed approach using three representative GNN architectures: GCN, GraphSAGE, and GAT. We further compare its performance with traditional refactoring approaches, an existing deep learning-based approach, and four representative lightweight LLMs. Experimental results show that the proposed approach outperforms the compared approaches across all evaluated code smell refactoring tasks, demonstrating that it provides both superior performance and greater efficiency.
- [91] arXiv:2602.01953 (replaced) [pdf, html, other]
-
Title: Deep Multivariate Models with Parametric ConditionalsSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
We consider deep multivariate models for heterogeneous collections of random variables. In the context of computer vision, such collections may e.g. consist of images, segmentations, image attributes, and latent variables. When developing such models, most existing works start from an application task and design the model components and their dependencies to meet the needs of the chosen task. This has the disadvantage of limiting the applicability of the resulting model for other downstream tasks. Here, instead, we propose to represent the joint probability distribution by means of conditional probability distributions for each group of variables conditioned on the rest. Such models can then be used for practically any possible downstream task. Their learning can be approached as training a parametrised Markov chain kernel by maximising the data likelihood of its limiting distribution. This has the additional advantage of allowing a wide range of semi-supervised learning scenarios.
- [92] arXiv:2602.05483 (replaced) [pdf, html, other]
-
Title: Toward Operationalizing Rasmussen in Evolving Software SystemsSubjects: Systems and Control (eess.SY); Computers and Society (cs.CY); Applications (stat.AP)
Rasmussen's qualitative drift model explains how successive, locally successful adaptations under efficiency pressures can move a system toward unacceptable operation. Brooker and Hochstein applied this perspective conceptually to software operations without specifying a quantitative procedure for observing drift. We take a step in this direction and propose a runtime observer that quantitatively operationalizes the model's observable part for evolving software. The proposed observer is intended to give engineers evidence for pausing or revising adaptation before declared operating limits are crossed. It tracks how activity or resource use is distributed across components and types of work. Measurement history quantifies accumulated change and recent movement toward or away from current limits. Assuming that direction and limits remain unchanged, the proposed observer calculates the first constraint ahead and the amount of further change required to reach it. Measurements remain comparable across supported component changes, while policy revisions are distinguished from changes in execution. Preliminary runtime studies demonstrated these distinctions under annotated software changes and revised retry limits. A separate multivariate exponentially weighted moving average (MEWMA) experiment detected CPU redistribution earlier with log-ratio coordinates than with raw shares, motivating investigation of representation-dependent observation. Further work will ground constraints in service outcomes, investigate runtime-model acquisition, and test whether decisions informed by observed drift reduce violations at acceptable useful-work loss and intervention cost.
- [93] arXiv:2602.12683 (replaced) [pdf, html, other]
-
Title: Flow Matching from Viewpoint of Proximal OperatorsComments: 58 pages, 13 figuresSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
We discuss the dynamics of Optimal Transport Conditional Flow Matching (OT-CFM) under the manifold hypothesis, where the support of the target distribution is a low dimensional manifold. We first derive an exact proximal expression of the vector field, using an extended Brenier potential for the OT; the vector field can be expressed by a denoiser, which is presented as the proximal operator associated with the potential function. Based on this explicit proximal form of the vector field, we analyze the manifold stability for perturbations to initial points, using the Lyapunov exponents of the dynamics. By analyzing the local behavior with the tools in convex analysis, we show that the dynamics contract linearly in the perturbation of the normal directions to the manifold, while perturbations in the tangential directions are preserved; this implies the stability of the manifold structure to the perturbations. To justify the theoretical arguments on the population OT, we also establish the convergence of minibatch OT-CFM with neural network learning to population OT-CFM under regularity and stability assumptions, with rate exponents governed by the intrinsic dimension of the data manifold. Experiments on synthetic data, MNIST, and CIFAR-10 support the predicted geometric behavior.
- [94] arXiv:2604.11578 (replaced) [pdf, html, other]
-
Title: Minimizing classical resources in variational measurement-based quantum computation for generative modelingComments: 15 pagesSubjects: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML)
Measurement-based quantum computation (MBQC) is a framework for quantum information processing in which a computational task is carried out through one-qubit measurements on a highly entangled resource state. Due to the indeterminacy of the outcomes of a quantum measurement, the random outcomes of these operations, if not corrected, yield a variational quantum channel family. Traditionally, this randomness is corrected through classical processing in order to ensure deterministic unitary computations. Recently, variational measurement-based quantum computation (VMBQC) has been introduced to exploit this measurement-induced randomness to gain an advantage in generative modeling. A limitation of this approach is that the corresponding channel model has twice as many parameters compared to the unitary model, scaling as $N \times D$, where $N$ is the number of logical qubits (width) and $D$ is the depth of the VMBQC model. This can often make optimization more difficult and may lead to poorly trainable models. In this paper, we present a restricted VMBQC model that extends the unitary setting to a channel-based one using only a single additional trainable parameter. We show, both numerically and algebraically, that this minimal extension is sufficient to generate probability distributions that cannot be learned by the corresponding unitary model.
- [95] arXiv:2605.11490 (replaced) [pdf, html, other]
-
Title: Adaptive Calibration in Non-Stationary EnvironmentsComments: 50 pages, NeurIPS 2026Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Making calibrated online predictions is a central challenge in modern AI systems. Much of the existing literature focuses on fully adversarial environments where outcomes may be arbitrary, leading to conservative algorithms that can perform suboptimally in more benign settings, such as when outcomes are nearly stationary. This gap raises a natural question: can we design online prediction algorithms whose calibration error automatically adapts to the degree of non-stationarity in the environment, smoothly interpolating between i.i.d. and adversarial regimes? We answer this question in the affirmative and develop a suite of algorithms that achieve adaptive calibration guarantees under multiple calibration measures. Specifically, with $T$ being the number of rounds, $W$ being the unknown number of i.i.d. segments of the environment, and $C\in[0,T]$ being another unknown non-stationary measure defined as the minimal $\ell_1$ deviation of the mean outcomes, our algorithms attain $\widetilde{\mathcal{O}}(\min\{\sqrt{T}+(TC)^{\frac{1}{3}}, \sqrt{WT}\})$ for $\ell_1$ calibration error and $\widetilde{\mathcal{O}}(\min\{(1+C)^{\frac{1}{3}}, W\})$ for both $\ell_2$ and pseudo KL calibration error. These bounds match the optimal rates in the stationary case ($C=0$ and $W=1$) and recover known guarantees in the fully adversarial regime ($C, W=\Omega(T)$). Our approach builds on and extends prior work (Hu et al., 2026; Luo et al., 2025), introducing an epoch-based scheduling together with a novel non-uniform partition of the prediction space that allocates finer resolution near the underlying ground truth.
- [96] arXiv:2605.25610 (replaced) [pdf, html, other]
-
Title: Match classification in the last round of four-team round-robin tournamentsComments: 24 pages, 4 figures, 6 tablesSubjects: Physics and Society (physics.soc-ph); General Economics (econ.GN); Optimization and Control (math.OC); Applications (stat.AP)
Classification of matches played in the last rounds of sports competitions is a well-established tool for evaluating tournament designs. Our paper offers the first comparison of deterministic and probabilistic match classification models by analysing the most prominent example of four-team round-robin competitions, the group stage of the FIFA World Cup. We show that both models are highly relevant in practice: all (four) deterministic and (six) probabilistic match types occurred in the 2014 and 2018 FIFA World Cups, respectively. The probabilistic model, which accounts for the relative benefits of attacking and defending, provides a deeper insight; for instance, the competitive matches from the deterministic approach can be of any of the six probabilistic types. Therefore, the probabilistic model is used to decompose the impact of three reforms introduced in the 2026 FIFA World Cup: the expansion to 48 teams, the modified qualification rules, and head-to-head tie-breakers.
- [97] arXiv:2605.29371 (replaced) [pdf, html, other]
-
Title: Kernel-based potential mean-field games with unbiased random Fourier $U$-statisticsSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Numerical Analysis (math.NA); Machine Learning (stat.ML)
We study the subclass of potential mean-field games in which the running interaction cost and the terminal target cost are both expressed through reproducing-kernel maximum mean discrepancy (MMD) penalties, and develop a computational framework that exploits this kernel structure. Both costs are estimated from finite-sample empirical distributions using a random Fourier U-statistic representation that is unbiased and has linear cost in the batch size. The drift of the controlled diffusion is parametrized by a neural network and trained via stochastic gradient descent. For population near-minimizers we prove convergence to the terminal-constrained problem as the penalty diverges, and show that the same limit is recovered almost surely when such controls are evaluated on independent finite samples under explicit coupling conditions on the penalty, random-feature count and sample size. The framework includes the kernel-MMD-penalty Schr{ö}dinger bridge problem as the special case of a vanishing interaction cost. Numerical experiments illustrate the method on the Schr{ö}dinger bridge problem in dimensions up to one hundred, and on an electric vehicle charging coordination problem with per-vehicle physical heterogeneity, where a control-dependent aggregate-demand cost produces a trade-off between peak normalized charging power and the terminal service level.
- [98] arXiv:2606.10944 (replaced) [pdf, html, other]
-
Title: Express Language ModelingSubjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Statistics Theory (math.ST); Methodology (stat.ME); Machine Learning (stat.ML)
We introduce a new tool, Express, for converting a non-causal attention approximation into a causal approximation with matching approximation guarantees. When combined with the state-of-the-art Thinformer approximation, Express improves upon the best known causal attention guarantees, delivering $\log^{3/2}(n)/s$ approximation error with only $O(s)$ memory and $O(s^2 \log^2(n))$ compression overhead for a sequence of length $n$. We pair these developments with an efficient I/O-aware Triton implementation, demonstrate substantial speedups over FlashAttention 2, and use Express to overcome four resource bottlenecks in the language modeling pipeline: long-context prefill, KV cache compression, long-form memory-constrained decoding, and long-form compute-constrained decoding.
- [99] arXiv:2606.28228 (replaced) [pdf, html, other]
-
Title: Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion ShiftsComments: Accepted at NeurIPS 2026 (camera-ready version). 53 pages, 15 figuresSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Causal representation learning for time series has developed strong identifiability results in discrete-time latent causal models, but identifiability in continuous-time latent stochastic differential equation (SDE) models remains largely open. We address this gap using environment-induced shifts in diffusion covariance. We study additive-noise latent SDEs observed through an unknown nonlinear diffeomorphism, with shared drift but environment-specific diffusion covariance. We show that two diagonal diffusion regimes with pairwise distinct coordinate-wise variance ratios identify the latent coordinates up to permutation, coordinate-wise scaling, and a possible constant shift, without any sparsity assumption on the drift. We first prove this result for linear Ornstein-Uhlenbeck systems and then extend it to general additive-noise latent SDEs. Under mild smoothness, the instantaneous drift-Jacobian causal graph is identifiable up to the same permutation. We propose a two-stage estimator for latent disentanglement and optional graph recovery; experiments on synthetic systems confirm the predicted identifiability boundary, and an application to Hardanger Bridge monitoring data illustrates the approach on real sensor trajectories.
- [100] arXiv:2607.29617 (replaced) [pdf, other]
-
Title: When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation LearningSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
Imitation learning (IL)---training an agent to replicate expert behavior from demonstrations---underpins applications from robotics to language model training. Standard approaches such as Behavior Cloning (BC) are known to suffer from compounding errors and performance plateaus, particularly when the learner cannot perfectly represent the expert's policy (as is typical, e.g., in distillation). Two interventions are widely understood empirically to improve performance: querying the expert interactively along the learner's own trajectories, and using value function estimation en route to generating a policy.
We investigate the nature of these improvements and their potentially surprising interplay. Our main finding is that expert interaction relaxes the representational demands on the learner: one only needs a model capable of realizing the expert's value function, bypassing the (often stricter) requirement of realizing the expert's policy itself. Concretely, we introduce OVI, an interactive on-policy IL algorithm that is statistically and computationally efficient whenever the learner can represent the expert's value function. We complement this with a negative result showing that interaction is necessary. Namely, without stronger assumptions beyond expert-value realizability alone, any offline IL algorithm must scale with the complexity of the expert policy class. Our findings bear out empirically. OVI outperforms offline policy-based (BC), interactive policy-based (DAgger), and offline value-based IL methods, with the largest gains when the learner network is substantially less expressive than the expert's. - [101] arXiv:2610.02771 (replaced) [pdf, html, other]
-
Title: Nearly Optimal Fixed-Confidence Best-Arm Identification with 1-Bit FeedbackComments: To appear in Advances in Neural Information Processing Systems 39 (NeurIPS 2026, Spotlight)Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
We study fixed-confidence best-arm identification under strict 1-bit feedback constraints. At each round, the learner selects an arm and a query set, and receives only a single bit indicating whether the sampled reward belongs to that set. We consider a distribution-free finite-variance setting with arm-wise localization, where direct empirical mean estimation is no longer available and clipping becomes unavoidable. We first formulate a time-uniform 1-bit mean-estimation primitive based on randomized threshold queries and a clipped tail-integral identity. We then embed this primitive into candidate-challenger best-arm identification algorithms. A fixed-clipping algorithm gives a simple anytime $(\epsilon,\delta)$-PAC guarantee, while a phased adaptive-clipping algorithm matches the clipping level to the current resolution and yields a gap-adaptive sample complexity. We also prove a $K$-arm worst-case information-theoretic lower bound showing that the logarithmic penalty caused by finite-variance 1-bit feedback is intrinsic. This bound matches the leading dependence of the phased algorithm up to logarithmic factors.
- [102] arXiv:2610.28301 (replaced) [pdf, html, other]
-
Title: Gaussian Fluctuations of the Gradient of the Fréchet Function and Nonstandard Asymptotics of Sample Fréchet MeansComments: 25 pagesSubjects: Probability (math.PR); Statistics Theory (math.ST)
We establish a central limit theorem for the gradient of the Fréchet function in local Euclidean coordinates on a metric space. Assuming a unique population mean, a consistent measurable sequence of sample means, semiconcavity or semiconvexity and moment bounds for the squared distance functions, and local $C^{1,\alpha}$ regularity of the population Fréchet function, we prove an asymptotic linear representation for its gradient at the sample mean. The leading term is the negative empirical average of the gradients of the squared distance functions at the population mean; these gradients exist almost surely under our assumptions. On complete connected Riemannian manifolds, uniqueness of the population mean, existence of a $(2+\eta)$-moment, and local $C^{1,\alpha}$ regularity suffice, and the leading term is expressed through the logarithm map. We also give a sufficient condition for the regularity of the population Fréchet function through probability bounds for neighborhoods of cut loci. We then show how the local behavior of the population Fréchet function transforms these Gaussian fluctuations into limit distributions for suitably rescaled sample Fréchet means, allowing for anisotropic scaling and non-Gaussian limits. A flat-torus example exhibits anisotropic scaling, and two examples on the circle illustrate logarithmic smeariness and the failure of any deterministic linear scaling to yield a nondegenerate limit, respectively.
- [103] arXiv:2610.29766 (replaced) [pdf, other]
-
Title: Bayesian Comparison of Covid Origins HypothesesComments: 35 pages,3 Figs,1 Table. about 100 references v2 fixes an equation, updates in response to detailed commentsSubjects: Populations and Evolution (q-bio.PE); Applications (stat.AP)
Applying standard Bayesian methods to the most reliably known unclassified facts about the Covid epidemic allows crude estimation of the odds for research-related vs. purely zoonotic origins. Here a broadly inclusive zoonotic hypothesis is compared with a rather specific subset of the research-related hypothesis defined by known prior work and grant proposals. The important observations are the type of virus, the city of the first outbreak, the lack of known prior hosts, the anomalous early pattern of evolution, and two specific features of the viral sequence. The resulting odds strongly favor research. The odds remain fairly strong under inclusion of uncertainties in prior odds.
- [104] arXiv:2610.30204 (replaced) [pdf, html, other]
-
Title: Triple Products of Correlated Multivariate Hermite PolynomialsComments: 25 pages, 4 figures, 1 supplementary PDF with data tablesSubjects: Numerical Analysis (math.NA); Methodology (stat.ME)
Stochastic Galerkin schemes (SGS) are used to approximate solutions to differential equations with stochastic parameters. In particular, they express the solution with an infinite orthogonal functional basis of the parameter space and derive a deterministic surrogate system of PDEs whose solution approximates that of the original stochastic problem. When the parameters are correlated, the infinite series representation of the solution does not provably converge for all joint distributions; further, classic decorrelating transformations (e.g., Rosenblatt, Nataf) often introduce known numerical instabilities. Rahman (2017) established convergence in probability and distribution for correlated Gaussian parameters when the basis functionals are multivariate Hermite polynomials (MHPs), where these polynomials are only weakly orthogonal with respect to the underlying joint density. Due to weak orthogonality, the $N$th moments of MHPs that emerge during Galerkin projection do not trivially decompose into products of one-dimensional expectations via tensorization. Namely, an $N$th-order moment has no closed-form expression for $N > 2$ and consequently is estimated numerically. Because the number of such moments grows combinatorially with parameter dimension and polynomial degree, approximating each one individually via sampling or quadrature can quickly become cost-prohibitive. In this paper, we introduce and prove several foundational properties of MHPs. These results culminate in an explicit formula for evaluating the $N = 3$ moments (triple products) directly without numerical estimation.
- [105] arXiv:2611.01929 (replaced) [pdf, html, other]
-
Title: Watch Your Step: Adam's Margin Maximization Bias Depends on the Learning Rate ScheduleSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
We study the implicit bias of Adam and its similarity to sign gradient descent on homogeneous models in binary classification tasks. Perhaps surprisingly, we uncover that these depend on the learning rate schedule. While previous works studied strongly decaying learning rate schedules $\eta(t)=\mathrm o \left(t^{1/L-1}\right)$, where $L$ is the homogeneity degree, and showed $\ell_\infty$-margin-maximization biases, we focus on the more common complementary regime, including constant and weakly decaying schedules. For a polynomial schedule $t^{\alpha}$ and homogeneity degree $L$, we show that when $\alpha > \frac 1L - \frac12$, Adam has no trajectory-independent margin-maximization bias for deep linear networks (in the KKT sense): first, there is no non-trivial, locally Lipschitz, convex implicit regularizer in this setting, including any squared norm. Second, on width-$1$ networks, the result holds without requiring convexity, implying that any implicit regularizer for wide networks must be trivial when restricted to a width-$1$ network. Nevertheless, we positively characterize the implicit bias in this regime as maximizing a trajectory-dependent weighted-$\ell_2$ margin for any $L$-homogeneous model. The fundamental mechanism behind our results is convergence of Adam's preconditioner to a trajectory-dependent vector, which also allows showing that the limiting parameter direction depends on initialization. In the intermediate regime $\frac 1L - 1 \leq \alpha \leq \frac1L - \frac12$ we prove an $\ell_\infty$ bias, conditional on gradient directional convergence to a full-support vector. Of independent interest, we establish a loss decay rate for full batch Adam on general homogeneous models, whose leading order depends only on the second-moment parameter.
- [106] arXiv:2611.07925 (replaced) [pdf, html, other]
-
Title: Efficiently Estimating Cycle Statistics on Graphs via Spanning TreesComments: 15 pages, 8 figures, plus appendixSubjects: Social and Information Networks (cs.SI); Computation (stat.CO)
Simple cycles encode feedback and alternative routes in networks and are important for routing problems, the convergence of dynamical processes on networks, message passing and belief propagation, or even static network characterizations such as structural balance in signed networks. Being able to assess and sample from the cycle structure of a graph is thus of interest in many analysis contexts. However, due to the combinatorial growth of the cycle space with graph size, the computational cost for such sampling is high. Therefore, we here introduce a sampling framework that enables us to sample from the cycle structure without enumerating all cycles. Specifically, the method draws uniform spanning trees and collects the fundamental cycles created by their non-tree edges, such that each tree provides an informative subset of simple cycles. Because cycles are included with unequal probabilities and are dependent within each subset, we formulate an inclusion-probability-based Horvitz-Thompson estimator, which we average over random subsets, a procedure we call Monte Carlo Horvitz-Thompson. For this, we sample multiple Horvitz-Thompson estimators and introduce a method to further subsample the individual estimators for increased computational performance. For sampling cycles, the estimator is unbiased if exact cycle inclusion probabilities are known. To extend the method to larger networks, we derive an efficient mean-field approximation for the inclusion probabilities and characterize the graph regimes in which it remains accurate. Experiments on synthetic and real-world networks demonstrate the computational scaling and statistical performance of the framework. We then illustrate the utility of Monte Carlo Horvitz-Thompson estimation by estimating cycle counts and cycle-level frustration statistics in signed spin systems. An implementation is available in the py-raccoon package.
- [107] arXiv:2611.13372 (replaced) [pdf, html, other]
-
Title: Extending indoor airborne infection risk modelling to 3D, with local surface depositionSubjects: Populations and Evolution (q-bio.PE); Dynamical Systems (math.DS); Other Statistics (stat.OT)
Mathematical models of indoor airborne disease transmission often rely on simplifying assumptions to reduce computational cost. In particular, aerosol concentrations may be assumed uniform across room height, allowing a two-dimensional (2D) formulation, while particle deposition on surfaces is represented by a first-order removal term acting throughout the computational domain. Here, we investigate how these assumptions affect predictions of airborne infection risk. We extend the Viral Infection Risk Indoor Simulator (VIRIS) [Xue et al. 2024] by developing a three-dimensional (3D) reaction-diffusion formulation in which aerosol deposition is represented by local removal at surfaces through a Robin boundary condition. We compare the original VIRIS framework with the extended model (VIRIS-3D-L) across three indoor spaces of increasing architectural and people-movement complexity: a courtroom, a care home and a supermarket. Across all three settings, the original 2D formulation with global removal predicts lower infection risks than the 3D formulation with local deposition, with differences of up to 50%. The predictions of VIRIS-3D-L also show encouraging agreement with observations from a documented courtroom outbreak previously used to validate VIRIS. Although computation time increases by approximately a factor of four on average, simulations remain feasible on a standard laptop, taking between 7 minutes and just over two hours. These findings suggest that VIRIS-3D-L offers a more physically realistic framework for assessing indoor airborne infection risk without prohibitive computational costs, while the original VIRIS remains useful when rapid simulations are required.