The starting point of this framework is an observation about how a human predictor operates in real time under high uncertainty and severe consequences: a driver executing high-speed overtaking maneuvers at narrow distances and not crashing.
The question is structural: why does the driver succeed? The answer reveals a predictive structure that conventional statistical models do not replicate explicitly.
The predictive process executed by an experienced driver consists of three stages:
Understanding the population reference. The driver carries an internal model of the average driver in a given country: typical reaction times, modal aggressiveness or caution, characteristic patterns of acceleration, braking, lane-changing. This population model functions as a prior.
Rapid identification of individual deviation. Within a one- or two-second window, the driver observes signals from the specific other driver —driving style, relative speed, type of vehicle, micro-movements of the steering wheel— and from these signals estimates how and how much the specific driver deviates from the average. More aggressive? More cautious? More impatient? Does the vehicle allow bolder maneuvers?
Conditional prediction and decision. With the individual profile estimated —always defined as a deviation relative to the average— the driver predicts: “If an average driver behaves in such-and-such a way, and this driver differs in this fashion, then in this situation the driver will or will not accelerate, will or will not slow down, will or will not move out of the current trajectory.” On that prediction, a maneuver is decided.
Conventional predictive models, in their simplest form, execute individual predictions through parameters computed from the data. Once computed, those parameters are fixed. Individual predictions arise from evaluating a function with those fixed parameters at the individual’s covariates:
\[\hat{y}_i \;=\; f(x_i;\; \hat{\theta}),\]
with \(\hat{\theta}\) identical for all individuals.
More elaborate predictive models do allow parameters to vary across individuals: hierarchical models with covariate-dependent random effects, varying-coefficient models, hypernetworks conditioned on global context, mixture-of-experts with gating networks, and state-space models with shared dynamics all admit individual variation in different forms. Some of these can be made to behave, under specific configurations, in ways functionally similar to what we describe below. The framework’s contribution is therefore not the absence of antecedents but the explicit and canonical formulation of a specific structural pattern that those antecedents do not typically make explicit.
What the driver does, framed precisely, is captured by the following compact statement.
Distinguishing structure (short form). Conventional predictive models do not typically make explicit a structure of the form \[\theta_i = \theta_{\text{ref}} + \Delta(x_i, \theta_{\text{ref}})\] in which the population reference \(\theta_{\text{ref}}\) enters the deviation function \(\Delta\) as a structural argument, not merely as a centering constant.
Distinguishing structure (extended form). Conventional predictive models, including those that admit individual heterogeneity or covariate-dependent parameters, do not typically formulate individual parameters as explicit deviations from a population reference where the deviation function depends both on the individual’s observable characteristics and on the reference itself. To the best of our knowledge, no standard formulation makes this functional dependence explicit and canonical.
The substantive content of this distinguishing structure unpacks into three points:
Crucially, the deviation is itself a function of the population reference: \(\Delta_i = \Delta(x_i, \theta_{\text{ref}})\). If the population reference changes —e.g., the framework is transferred to a new population— the deviation function may behave differently because \(\theta_{\text{ref}}\) is one of its arguments. This structural dependence of \(\Delta\) on \(\theta_{\text{ref}}\) is the precise mathematical content of the framework’s contribution and the property that the AMM canonical form (Vignette 01) makes operational.
For each observation \(i\) with observable covariates \(x_i\), the framework defines:
\[\theta_i \;=\; \theta_{\text{ref}} \;+\; \Delta(x_i,\; \theta_{\text{ref}}),\]
where:
The prediction for individual \(i\) is
\[\hat{y}_i \;=\; f\!\bigl(x_i;\; \theta_i\bigr) \;=\; f\!\bigl(x_i;\; \theta_{\text{ref}} + \Delta(x_i,\; \theta_{\text{ref}})\bigr).\]
The framework is fully specified by three design choices:
The canonical functional form for \(\Delta\) is the Additive-Multiplicative-Modulated (AMM) form developed in Vignette 01.
This path is the most faithful to the original cognitive analogy and the most rigorous from a theoretical standpoint. It captures the three levels of the process: the population reference (top level), the individual deviation (intermediate level), and the noisy observation (bottom level).
\[\theta_{\text{ref}} \;\sim\; p(\theta \mid \text{hyperparameters}).\]
This level establishes the prior over the population parameters. Hyperparameters may themselves carry hyperpriors, completing the hierarchy.
\[\theta_i \mid x_i \;\sim\; \mathcal{N}\!\bigl(\theta_{\text{ref}} + \Delta(x_i, \theta_{\text{ref}}),\; \Sigma_i\bigr),\]
where \(\Sigma_i\) is the covariance of individual parameters, possibly depending on \(x_i\) to allow individual heteroscedasticity: individuals with certain characteristics may have more or less variable deviations from the reference.
\[Y_i \mid \theta_i \;\sim\; \mathcal{D}(\theta_i),\]
with \(\mathcal{D}\) the distribution appropriate to the modelling problem (Normal, Poisson, Binomial, Gamma, etc.).
When the phenomenon admits an excessive proportion of zeros, the zeros may be of two natures:
The probability of observing a zero is
\[P(Y_i = 0 \mid x_i) \;=\; \pi_i + (1 - \pi_i) \cdot P_{\text{count}}(0 \mid \theta_i),\]
with \(\pi_i\) the probability of structural zero for individual \(i\).
Crucially, in this framework both \(\pi_i\) and \(\theta_i\) are defined as deviations from their respective population references:
\[\mathrm{logit}(\pi_i) \;=\; \mathrm{logit}(\pi_{\text{ref}}) + \Delta_\pi(x_i,\; \pi_{\text{ref}}),\]
\[\theta_i \;=\; \theta_{\text{ref}} + \Delta_\theta(x_i,\; \theta_{\text{ref}}).\]
The dual deviation structure —both components of the inflated model anchored to the population reference— is the framework’s distinctive contribution to zero-inflated count modelling.
When observations are not independent, the correlation structure is introduced at the individual level:
When multiple response variables \(Y_i = (Y_{i1}, \ldots, Y_{iK})\) are present, the parameter vector becomes a vector of vectors:
\[\boldsymbol{\theta}_i \;=\; \boldsymbol{\theta}_{\text{ref}} + \boldsymbol{\Delta}(x_i,\; \boldsymbol{\theta}_{\text{ref}}).\]
The joint distribution \(\mathcal{D}\) is modelled in two ways:
Stan is the most flexible tool for this structure, accessed through
the cmdstanr interface. PyMC and INLA are alternatives
—PyMC for Python integration, INLA for fast approximations sacrificing
some flexibility.
Implementation status (
gdpar 0.0.0.9001). Path 1 is the operational default and the only currently implemented path ingdpar. Paths 2 (varying-coefficient) and 3 (hypernetwork) are described conceptually in §4 and §5 below, with implementation queued for future versions; calls of the formgdpar(..., path = "vcm")orgdpar(..., path = "hyper")abort with agdpar_unsupported_feature_error. See vignettesv05_asymptotics_path2_vcmandv06_asymptotics_path3_hypernetworkfor the reference-grade asymptotic theory and prospective implementation notes.
This is the most direct frequentist version of the framework. The central idea is that the model coefficients are not constant but functions of the individual’s characteristics.
\[Y_i \;=\; x_i^\top\, \beta(z_i) + \varepsilon_i,\]
where:
The population reference emerges naturally:
\[\beta_{\text{ref}} \;=\; \beta(\bar{z}),\]
i.e., the coefficients evaluated at the average individual (\(\bar{z}\) being the vector of means or modes of individual descriptors). The individual deviation is then
\[\Delta_i \;=\; \beta(z_i) - \beta(\bar{z}).\]
The function \(\beta(\cdot)\) can be estimated in several ways:
Interpretability is the principal strength: one can visualize and quantify explicitly how each individual trait (\(z_i\)) modifies each model coefficient (\(\beta_k\)) relative to the average. This addresses questions such as “by how much does the effect of variable \(x_k\) on \(Y\) change when the individual is more aggressive than the average?”
Scaling to high-dimensional problems (many components in \(z_i\) or many coefficients in \(\beta\)) is harder than for the Bayesian path, since non-parametric estimation of multivariate functions suffers from the curse of dimensionality.
The structure is wrapped in a two-part model:
Generalized estimating equations (GEE) handle correlation among observations without requiring full specification of the joint distribution; alternatively, the model is extended with explicit correlation structure in the residuals.
This path has the highest expressive capacity. The central idea: a neural network \(h_\phi\) takes the individual’s characteristics and generates the parameters of the predictive model for that specific individual.
\[\theta_i \;=\; h_\phi(x_i, \theta_{\text{ref}}),\]
\[Y_i \;\sim\; \mathcal{D}(\theta_i),\]
where:
The link to the population reference is established in two complementary ways:
During training, the network learns to map from “what is observable about the individual” to “what its parameters should be.” The typical loss is
\[\mathcal{L}(\phi) \;=\; -\sum_{i=1}^n \log p(Y_i \mid \theta_i) + \lambda \sum_{i=1}^n \|\theta_i - \theta_{\text{ref}}\|^2,\]
where the first term maximizes the likelihood under the individual parameters, and the second penalizes excessive deviations from the reference.
The network produces both \(\pi_i\) (structural zero probability) and the count parameters \(\theta_i\) as separate outputs:
\[(\pi_i, \theta_i) \;=\; h_\phi(x_i, \pi_{\text{ref}}, \theta_{\text{ref}}).\]
The hypernetwork is combined with architectures designed for sequential or structured data:
The network produces the parameters of the joint distribution:
The deviation function \(\Delta\) can be arbitrarily nonlinear. There is no restriction on the functional form of the relation between individual characteristics and parameters. This permits capture of complex interactions that the other two paths cannot represent without manual specification.
Interpretability is reduced: it is not possible to state analytically “this individual is more aggressive than the average by this specific amount and that modifies this coefficient by this magnitude.” The network produces parameters but does not explain why.
torch (the R port of LibTorch) is the natural choice.
For complex conditional distributions, normalizing flows extend the
architecture.
Implementation status (
gdpar 0.0.0.9001). Only the Hierarchical Bayesian column below is operationally available in the current release. The Varying-Coefficient and Hypernetwork columns describe the framework’s planned coverage; their implementation ingdparis queued for future versions (gdpar(..., path = "vcm")andgdpar(..., path = "hyper")abort withgdpar_unsupported_feature_error).
| Criterion | Hierarchical_Bayesian | Varying_Coefficient | Hypernetwork |
|---|---|---|---|
| Fidelity to cognitive analogy | High | Moderate | Moderate |
| Theoretical rigor | Very high | High | Moderate |
| Interpretability | High | Very high | Low |
| Expressive capacity of Delta | Moderate (parametric) | Moderate to high | Arbitrarily high |
| Scalability to high dimension | Moderate | Low (curse of dim.) | High |
| Uncertainty quantification | Native (full posteriors) | Asymptotic (CIs) | Requires extensions (MC dropout, etc.) |
| Implementation effort | Moderate | High | Low |
| Primary tools | Stan, cmdstanr | mgcv, splines | torch |
| Scenario | Recommended_Path |
|---|---|
| Few variables, interpretability central | Varying-coefficient (Path 2) |
| Complex structure, full uncertainty | Hierarchical Bayesian (Path 1) |
| High dimension, nonlinear relations | Hypernetwork (Path 3) |
| Inflated zeros (any kind) | Any of the three, with mixture model |
| Dependent observations | Bayesian (more natural) or VCM with GEE |
| Multivariate response | Bayesian with copulas, or multi-output hypernetwork |
Every predictive model can benefit from a structure in which individual parameters are defined as explicit deviations from a population reference, where the deviation function depends both on the individual’s observable characteristics and on the reference itself.
\[\boxed{\theta_i \;=\; \theta_{\text{ref}} + \Delta(x_i,\; \theta_{\text{ref}})}\]
The canonical functional form for \(\Delta\) is the Additive-Multiplicative-Modulated (AMM) form,
\[\Delta(x, \theta_{\text{ref}}) \;=\; a(x) + b(x) \odot \theta_{\text{ref}} + W(\theta_{\text{ref}})\, x,\]
developed in Vignette 01.
The framework operates without restriction on:
Each condition may require a specific extension of the model, but all operate under the same canonical principle \(\theta_i = \theta_{\text{ref}} + \Delta(x_i, \theta_{\text{ref}})\).
| Symbol | Meaning |
|---|---|
| \(\theta_i\) | Vector of model parameters for individual \(i\) |
| \(\theta_{\text{ref}}\) | Population reference parameters (expectation) |
| \(\Delta(x_i, \theta_{\text{ref}})\) | Individual deviation function |
| \(x_i\) | Observable covariates of individual \(i\) |
| \(z_i\) | Individual descriptors (Path 2) |
| \(W(\theta_{\text{ref}})\) | Modulating matrix dependent on the reference |
| \(\Sigma_i\) | Individual covariance matrix (Path 1) |
| \(\pi_i\) | Structural zero probability for individual \(i\) |
| \(\pi_{\text{ref}}\) | Population structural zero probability |
| \(h_\phi\) | Hypernetwork with parameters \(\phi\) (Path 3) |
| \(\mathcal{D}(\theta_i)\) | Distribution of the response given \(\theta_i\) |
| \(\beta(z_i)\) | Varying coefficients as a function of \(z_i\) (Path 2) |
| \(\phi\) | Temporal persistence parameter of the deviation |
| \(\lambda\) | Regularization parameter for deviation penalty (Path 3) |
| \(\odot\) | Hadamard (elementwise) product |