Data Analysis, Statistics and Probability
See recent articles
Showing new listings for Tuesday, 17 November 2026
- [1] arXiv:2611.18688 [pdf, html, other]
-
Title: An improved statistical treatment for radioassay-based background projection incorporating part-to-part variationSubjects: Data Analysis, Statistics and Probability (physics.data-an); Instrumentation and Methods for Astrophysics (astro-ph.IM); Nuclear Experiment (nucl-ex)
For low background experiments, prediction of the background from radioactivity in detector components is of critical importance. Radioassays are often conducted on small samples or individual parts, which may be subject to differences in production or handling resulting in a part-to-part variation. This part-to-part variation can have an impact on the total activity and is not presently captured by the averaging methods used to evaluate radioassays. We propose a new method for the averaging of radioassays to incorporate this part-to-part variation into predictions of the total activity of parts. We also consider how to add component activities based on assays for component groups that are simulated as a single entity.
- [2] arXiv:2611.18724 [pdf, html, other]
-
Title: Some Considerations on Measuring and Comparing ThingsComments: 21 pages and 7 figuresSubjects: Data Analysis, Statistics and Probability (physics.data-an)
Measurements and relationships between measurements are basic aspects underlying science and technology. In the present text, we present and discuss some of the approaches for measuring, normalizing, and comparing entities represented in terms of a set of measurements. Three main categories of measured values (constant, direct, or inverse) are proposed, underlaid by the relationship between data variation or intervals of interest and magnitude of the quantifications. For example, in certain cases the variation of measurements can be expected to be proportional to the relative magnitude (e.g.~when the magnitudes extend over long ranges). The considered approaches to comparing measurements include the Euclidean distance, three types of proximity based on the latter, as well as the Jaccard and coincidence indices. The presentation is complemented by graphical illustrations.
- [3] arXiv:2611.18773 [pdf, html, other]
-
Title: Amortized conditional inference under changing parameter information in small-angle X-ray scatteringMaria Bånkestad, Sandra Barman, Erik Kaunisto, Eskil Andreasson, Smita Chakraborty, Shun Yu, Jerk Rönnols, Sepideh PashamiComments: 24 pages, 9 figures. Code and trained models: this https URLSubjects: Data Analysis, Statistics and Probability (physics.data-an)
Many scientific measurements do not uniquely determine the physical system that produced them. Different parameter configurations may explain the same observations, and additional measurements or physical assumptions can change which explanations remain plausible. We develop an amortized conditional inference method to examine these alternatives as the available information changes. Using masked flow matching, the method approximates joint conditional posteriors when different parameters are supplied exactly or with uncertainty. This allows repeated analysis of the same measurement without retraining, while reusing its encoded representation. We can also examine the consequences of assumed parameter distributions and separately assess whether supplied information is compatible with the observation-only posterior. We validate the learned conditionals against analytic posteriors and compare posterior accuracy and inference cost in a simulated scattering benchmark, where the compressed architecture performs better than joint attention on both measures. We apply the method to small-angle X-ray scattering (SAXS) of cellulose fibrils in paper, examining structural ambiguities in simulated measurements and the effects of physical assumptions on experimental measurements of a paper during hydration. At high humidity, the inferred effective fibril size distribution broadens. Assuming that this distribution remains unchanged instead shifts the posterior towards increased interfibrillar separation, although both interpretations reproduce the measured curves similarly. The compatibility analysis identifies conserved parameter values that are difficult to reconcile with the wettest measurements.
- [4] arXiv:2611.19355 [pdf, html, other]
-
Title: Numerical Integration Instabilities in Likelihood Fits: Mechanisms and RemediesSubjects: Data Analysis, Statistics and Probability (physics.data-an); High Energy Physics - Experiment (hep-ex)
Numerical instabilities in the normalization of probability density functions can produce artificially lower negative log-likelihood (NLL) values than the true minimum of a likelihood fit, even when the resulting fit provides a poorer description of the data. Motivated by anomalously low NLL values found in non-converged fits associated with a recent CMS search for fully charmed tetraquark candidates, we use a pseudo-dataset generated from a likelihood model based on that analysis to investigate how numerical integration can yield unreliable normalization results, how these errors affect likelihood minimization, and how they can be addressed. We find that, even at fixed integration tolerance, the stopping decision of a numerical integrator can change with the fit parameters, introducing discontinuities in the normalization integrals that propagate to the likelihood surface. Such discontinuities can trap the minimizer at a numerical step and yield an artificially lower NLL. Tightening the integration tolerance mitigates these numerical discontinuities, while adaptive integration methods, such as Gauss--Kronrod quadrature, can also improve the numerical stability. A more severe failure occurs when the integration grid cannot adequately resolve a sharply localized contribution. In this case, the integrator may satisfy its estimated-error criterion despite substantially underestimating the normalization, allowing the minimizer to enter a region with an artificially low NLL. Tightening integration tolerances or using alternative adaptive integration methods does not necessarily resolve sharply localized contributions. We demonstrate that a suitable variable transformation can improve their numerical resolution, yielding more accurate normalization integrals and restoring fit convergence.
- [5] arXiv:2611.20508 [pdf, html, other]
-
Title: A Paired Monte Carlo Intensity-Ratio Method for Incorporating Detector Resolution in Amplitude AnalysisSubjects: Data Analysis, Statistics and Probability (physics.data-an); High Energy Physics - Experiment (hep-ex); Instrumentation and Detectors (physics.ins-det)
Finite detector resolution can bias resonance parameters and fit fractions extracted from amplitude analyses. We present the paired Monte Carlo intensity-ratio method for incorporating resolution effects into amplitude analysis. The method uses ratios of reference intensities evaluated at paired truth-level and reconstructed kinematics to reweight the Monte Carlo normalization and approximately account for detector resolution. A resampling study of simulated $D^+\to K^+\pi^+\pi^-\pi^0$ decays shows that the method substantially reduces resolution-induced shifts in fitted resonance widths. Likelihood evaluation uses intensity calculations and normalization sums, avoiding a response convolution at each data event.
New submissions (showing 5 of 5 entries)
- [6] arXiv:2611.18675 (cross-list from q-bio.QM) [pdf, html, other]
-
Title: Exact mean first-passage time and a Poisson-smoothing crossover in renewal shot noiseSubjects: Quantitative Methods (q-bio.QM); Statistical Mechanics (cond-mat.stat-mech); Probability (math.PR); Biological Physics (physics.bio-ph); Data Analysis, Statistics and Probability (physics.data-an)
Threshold crossings in exponentially relaxing shot noise depend on the interplay between incoming impulses and the residual signal left by earlier arrivals. For renewal input, existing high-threshold product approximations of the mean first-passage time (MFPT) leave open when evaluation at a single index preserves the mean with vanishing relative error. Here we derive an exact finite-threshold MFPT series for exponentially marked renewal shot noise and recast it as an average over an auxiliary Poisson index. When the waiting-time density exhibits short-time power-law behavior, $w(t)\sim ct^{\kappa-1}$, the dominant series contributions shift from the Poisson center by an amount of order $x^{1-\kappa}$, whereas the averaging kernel has a fluctuation width of order $\sqrt{x}$, with $x=\lambda b$. The competition between these scales yields a crossover at $\kappa=1/2$, where the deterministic-index product is asymptotically exact in ratio above this boundary, requires a finite multiplicative correction at criticality, and exhibits a growing logarithmic discrepancy below it. A moving-saddle approximation also yields an asymptotically exact MFPT ratio under general strictly positive, finite-mean waiting times. Numerical comparisons across Gamma, Weibull, and generalized-Gamma renewal laws illustrate the approximation regimes and their finite-threshold accuracy. Together, these results provide an exact finite-threshold benchmark for the mean first-passage time and reveal how short-gap renewal statistics determine the asymptotic accuracy of high-threshold approximations.
- [7] arXiv:2611.19494 (cross-list from physics.flu-dyn) [pdf, html, other]
-
Title: A quantitative measure of predictability for surface gravity wavesSubjects: Fluid Dynamics (physics.flu-dyn); Computational Physics (physics.comp-ph); Data Analysis, Statistics and Probability (physics.data-an)
We introduce a Bayesian inference framework to characterize the predictability of a one dimensional ocean surface wave field given surface elevation measurements from a sensor. Unlike the classical theories that only sketch the boundary of a predictable zone based on group velocity, we use the Bayesian posterior uncertainty of surface elevation to quantify the local predictability at every spatiotemporal location. Our approach first constructs a prior distribution to encode existing knowledge of the wave field such as the spectrum and propagation direction. By conditioning on the (possibly noisy) measurement data, we infer the posterior distribution of the wave field, from which a spatiotemporal uncertainty map is extracted. This map agrees well with the group-velocity-based classical predictable zone theory when the frequency spectrum has bounded support. Meanwhile, it also demonstrates predictability even for a spectrum with unbounded frequency support where the classical predictable zone degenerates. A refined analysis of posterior uncertainty using observability Gramian matrix reveals a `data-informed' part and an `un-informed' part of the wave field, with the latter being fundamentally un-predictable. Furthermore, nonlinear effects can be naturally introduced into the Bayesian framework by incorporating the nonlinear ocean surface wave equations. This is the first time that nonlinearity is rigorously treated in the predictable zone problem. We perform numerical tests using both bounded, piecewise constant spectrum and unbounded JONSWAP spectrum. The numerical results confirm the proposed advantages of our method for linear waves and reveal how wave steepness affects the predictability of nonlinear waves. Nonlinear modal interactions redistribute uncertainty between the inside and the outside of the predictable zone with an amount proportional to the wave steepness.
- [8] arXiv:2611.19516 (cross-list from math.DS) [pdf, html, other]
-
Title: Denoising-Induced Recovery of Effective DynamicsSubjects: Dynamical Systems (math.DS); Data Analysis, Statistics and Probability (physics.data-an)
Recovering dynamical equations from data is a central problem in nonlinear science. In practice, measurements are corrupted by noise and often contain fast oscillations that are not part of the slow dynamics of interest. For this reason, data are typically pre-processed before model recovery or classsification. Here, we reveal the effects of filtering, a specific denoising technique, on recovery. Our analysis show that equation discovery performed on filtered data recovers, up to a fixed precision, the normal-form (effective) equations rather than the original system.
- [9] arXiv:2611.21175 (cross-list from astro-ph.GA) [pdf, html, other]
-
Title: A COMPASS to the Galaxy: Joint nucleosynthetic yield set comparison and parameter inference in Galactic chemical evolutionSubjects: Astrophysics of Galaxies (astro-ph.GA); Instrumentation and Methods for Astrophysics (astro-ph.IM); Machine Learning (cs.LG); Data Analysis, Statistics and Probability (physics.data-an)
Galactic Chemical Evolution (GCE) models connect stellar nucleosynthesis to observed abundance patterns, but their interpretation requires simultaneous parameter inference and comparison between competing stellar yield prescriptions. This remains challenging due to intractable likelihoods and growing dataset complexity. Using compass, a novel simulation-based inference framework, we perform, for the first time, amortized joint Bayesian model comparison and parameter inference for GCE models with intractable likelihoods within a unified architecture. Compass employs a conditional score-based diffusion model with a transformer architecture to learn both the posterior and likelihood directly from forward simulations, enabling efficient inference and likelihood-based ranking of competing nucleosynthetic yield sets. In controlled tests, compass identifies the generating yield set and recovers ground-truth parameters with accuracy comparable to existing simulation-based inference (SBI) methods, while reducing computational cost by orders of magnitude. Applied to Milky Way data, the framework favors a hybrid NuGrid + IllustrisTNG yield prescription, confirmed by an independent classifier, and infers an initial mass function slope of $\alpha_{\rm IMF} = -2.65 \pm 0.02$ and a SN$_{\rm Ia}$ normalization of $\log(N_{\rm Ia}) = -2.81 \pm 0.02$, with a systematic spread of $\approx 0.3$ in $\alpha_{\rm IMF}$ between the best-ranked yield sets, consistent with independent constraints. The attention mechanism confirms that the inferred parameters are driven by physically meaningful abundance signatures. By unifying amortized parameter inference, Bayesian model comparison, and interpretability in a single probabilistic framework, compass provides a scalable approach for precision chemical evolution studies with current and forthcoming spectroscopic surveys.
Cross submissions (showing 4 of 4 entries)
- [10] arXiv:2611.12563 (replaced) [pdf, html, other]
-
Title: Distinguishing thyroid nodule malignancy using digital palpationBrendan M. Unikewicz, Joseph E. Bonavia, Ekaterina L. Koelliker, Abigail Klein, Katie C. Spaeth, Hannah M. Varner, S. Kiana Naghibzadeh, Yoav Horev, Sareh Parangi, Tal CohenSubjects: Data Analysis, Statistics and Probability (physics.data-an); Soft Condensed Matter (cond-mat.soft)
Accurate pre-operative diagnosis of thyroid nodules remains a major clinical challenge. Current diagnostic techniques rely on fine-needle aspiration and demonstrate high rates of inconclusive results, often leading to avoidable thyroidectomies with lifelong implications. Here, we introduce a mechanics-based diagnostic method that leverages Digital Palpation to determine malignancy and identify tissue subtypes in human thyroid nodules. From a cohort of patients undergoing thyroidectomies, we evaluated ex vivo thyroid specimens and processed the resulting mechanical data from 36 tests in 10 thyroid nodules. Principal Component Analysis reveals that these experimental results are distributed within a three-dimensional space of interpretable scalar features reflecting tissue grittiness, strength, and toughness, which map closely to descriptions of malignant and benign nodules given by clinicians. Our results further demonstrate that malignant nodules exhibit distinct mechanical signatures compared to their benign counterparts. This revealed a statistically significant difference (p = 9 x 10^-4) and separability (95% lower-bound Hedges' g = 1.08) between benign and malignant nodules along a direction in principal component space representing effective stiffness. Preliminary clustering patterns were also observed among malignant subtypes, including follicular, oncocytic, and papillary thyroid carcinomas. Together, these results highlight the potential of Digital Palpation as a novel mechanics-based method for tissue characterization and motivate its further development toward diagnostic applications.
- [11] arXiv:2601.12306 (replaced) [pdf, html, other]
-
Title: Logarithmic dispersion at marginal stability in collective attentionComments: Main Text: 5 pages (1 figure); Supplemental Material: 12 pages (5 figures, 5 tables)Journal-ref: Phys. Rev. E 114 (2026) L052302Subjects: Physics and Society (physics.soc-ph); Statistical Mechanics (cond-mat.stat-mech); Digital Libraries (cs.DL); Social and Information Networks (cs.SI); Data Analysis, Statistics and Probability (physics.data-an)
How cross-sectional heterogeneity evolves in nonequilibrium populations of competing, ageing units is a collective question that cannot be resolved from a representative trajectory or the ensemble mean alone. Analysis of more than two million age-aligned English Wikipedia page-view trajectories from six yearly cohorts spanning 2015-2020 shows that the variance of logarithmic attention grows linearly with $\ln t$, while the ensemble mean relaxes on the same logarithmic clock. This dispersion law recurs across all six cohorts and is systematically favoured over the tested power-law alternative; temporal increments also exhibit consistent antipersistent scaling. A complementary trajectory-level analysis reveals broad, non-Gaussian variation in effective relaxation rates, showing that this collective regularity coexists with heterogeneous observed trajectories. A minimal scale-free stochastic model combines mean relaxation with self-affine temporal fluctuations of age-dependent amplitude. Within this model class, logarithmic spreading emerges when fluctuation accumulation balances ageing attenuation, defining the marginal boundary between algebraic broadening and saturation. Together, age-resolved cross-sectional dispersion and temporal-increment scaling provide a transferable ensemble-level criterion for distinguishing amplification, marginal spreading, and saturation in other populations of competing, ageing units.
- [12] arXiv:2609.00883 (replaced) [pdf, html, other]
-
Title: iPINN for Broadband CARS Phase Retrieval: A Framework for Function Approximation and Inverse Modeling Problems in Nonlinear SpectroscopyRavi Teja Vulchi, Carl Messerschmidt, Mohammadsadegh Vafaeinezhad, Rajendhar Junjuri, Tobias Meyer-Zedler, Juergen Popp, Thomas BocklitzSubjects: Machine Learning (cs.LG); Data Analysis, Statistics and Probability (physics.data-an)
Broadband coherent anti-Stokes Raman scattering (BCARS) records chemically specific vibrational spectra, but the measured signal mixes the resonant, Raman-like response of the sample with a non-resonant background that changes from one measurement to the next. Recovering the resonant response, known as phase retrieval, is an ill-posed inverse problem. We present iPINN, an inverse physics-informed neural network that predicts the positions, amplitudes, and widths of Lorentzian vibrational bands directly from a BCARS spectrum. A transformer network estimates these band parameters, and a differentiable analytical model of the resonant susceptibility rebuilds the Raman-like spectrum from them as a sum of physically defined bands. We train the network on simulated spectra in which the bands of each reference compound stay fixed while the shape and strength of the background and the noise vary. Training also encourages similar band predictions for the same compound across these variations. On a public benchmark of toluene and dimethyl sulfoxide spectra, the network reached a mean absolute error about one third of that of the best of five deep-learning baselines. Its error for dimethyl sulfoxide stayed nearly constant across three noise levels. We also used 28 spectra of seven solvents, recorded at four focal depths with different background strengths. For five of the seven solvents, the mean absolute error changed little with depth, while it varied more for the other two. These results indicate that this approach keeps phase retrieval stable for most of the tested solvents when the background changes.
- [13] arXiv:2610.40749 (replaced) [pdf, html, other]
-
Title: Combinatorial Residual Network: a decay-chain-aware classifier without explicit jet assignmentJohn Alison, Patrick Bryant, Alejandro Gomez Espinosa, Christian Guinto-Brody, Aniket Khanal, Chuyuan Liu, Sindhu Murthy, Guanglei ZhaoComments: 24 pages, 7 figures, 6 tables. Code available at this https URLSubjects: High Energy Physics - Experiment (hep-ex); Machine Learning (cs.LG); High Energy Physics - Phenomenology (hep-ph); Data Analysis, Statistics and Probability (physics.data-an)
Final states with several jets from the decays of heavy particles pose a combinatorial problem: the jets must be assigned, at least implicitly, to their parent particles. The Combinatorial Residual Network (CoReNet) is a classifier that encodes the assumed decay chain in its architecture, building the event representation level by level along the decay hierarchy and combining all jet assignment hypotheses, evaluated with shared weights, with learned weights. The exchange symmetries of the decay are built in, the network size does not depend on the number of hypotheses, and no generator-level jet assignments are used in training. Developed for the $b\bar{b}b\bar{b}$ final state and used in several CMS searches, CoReNet with 1508 parameters performs well above kinematic baselines on fast-simulated Higgs boson pair production against QCD multijet events and trains in minutes on a single GPU, and its hypothesis weights select the truth-matched jet pairing in most signal events in which it is defined. Since only the encoded decay chain is specific to the final state, we test the construction on $HH \to b\bar{b}W^+W^-$ against semileptonic $t\bar{t}$, where encoding the decay chain of the background as well improves the classification at a cost of about $10^{3}$ parameters. In both final states, CoReNet performs comparably to a state-of-the-art transformer-based network more than two orders of magnitude larger. The implementation is publicly available at this https URL.
- [14] arXiv:2611.09116 (replaced) [pdf, html, other]
-
Title: Domain-Adapted Deep Optical Flow for Multiphase Velocimetry in Unsteady Cloud CavitationComments: 42 pages, 17 figures, 6 tables, 3 appendices. Submitted to Experiments in Fluids. Code: this https URLSubjects: Fluid Dynamics (physics.flu-dyn); Data Analysis, Statistics and Probability (physics.data-an)
Time-resolved velocimetry inside cloud cavitation is difficult because tracers do not penetrate vapor structures and deforming interfaces distort the particle images that PIV relies on. We present a domain-adapted deep optical flow framework for two-phase velocimetry, a RAFT model fine-tuned on synthetic image pairs rendered from multiphase OpenFOAM simulations (RAFT-CloudCav). It is compared with a pre-trained RAFT model, Farnebäck and OpenPIV on 130,000 frames/s recordings of a Venturi at 30 and 48 L/min. As no experimental ground truth exists, validation uses the integrated flow rate against a turbine meter, the liquid-phase velocity divergence and the station-to-station spread of the flow rate. The four methods agree within 2% in the liquid core and integrate to 41.6-43.2 L/min against a metered 48.0 $\pm$ 0.5 L/min, a shortfall common to all four that lies in the measurement and is partly traced to the coarse imaging configuration. RAFT-CloudCav gives the most consistent flow rate between stations (1.8% against 4.2-4.5%). The learned estimators reduce the liquid-phase divergence three- to seventeenfold relative to the classical ones, with the pre-trained model lowest, and OpenPIV under-reads the cavitating near-wall shear layer by up to 38%. Every method resolves the re-entrant jet in the instantaneous field. In a non-cavitating control the boundary layer is captured better as the imaging is refined, from three grid points across a 0.66 mm layer at 49 $\mu$m/pixel to twelve at 13 $\mu$m/pixel. The mean field is recovered without tracer particles to within 13%. Uncertainties are estimated a posteriori from the signed image residual and propagated conservatively.