↓ Skip to main content
  1. Posts/

AR(1) Process Autocorrelation: Theory vs Simulation in Python

Author
Yauheni Yakauleu
Short, practical articles on quant methods, alternative data, and applied AI. Code Insights, Touch IT LLC.

You simulate an AR(1) process with parameter ρ = 0.7 and compute its sample autocorrelation at lag 1. What value do you expect? The theoretical answer is simple: the lag-1 autocorrelation of an AR(1) process is exactly ρ. But how quickly does the sample autocorrelation converge to this value — and how noisy is the estimator at realistic sample sizes?

In this post, we derive the theoretical autocorrelation function of an AR(1) process, simulate it in NumPy, and verify both the point estimate and its dispersion against theory with a Monte-Carlo experiment.

The AR(1) process definition
#

An autoregressive process of order 1 (AR(1)) is defined by:

X_t = c + ρ·X_{t-1} + ε_t

Where:

  • c is a constant (drift term)
  • ρ is the autoregressive coefficient, with |ρ| < 1 for stationarity
  • ε_t is white noise: ε_t ~ N(0, σ²) i.i.d.

For a stationary AR(1) process (|ρ| < 1), the mean, variance, and autocorrelation function all have closed forms.

Theoretical properties
#

Mean and variance
#

The stationary mean is:

E[X_t] = c / (1 - ρ)

The stationary variance is:

Var(X_t) = σ² / (1 - ρ²)

Autocorrelation function (ACF)
#

The key result we’ll verify: the autocorrelation at lag k is:

ρ_k = ρ^k

In particular, the lag-1 autocorrelation is simply ρ_1 = ρ — regardless of the noise variance σ² or the constant c.

Simulation setup
#

import numpy as np

rng = np.random.default_rng(42)

# AR(1) parameters
c = 0.5          # drift term
rho_true = 0.7   # true AR coefficient
sigma = 1.0      # noise standard deviation

def simulate_ar1(n, c, rho, sigma, rng, burn_in=100):
    """Simulate an AR(1) process with burn-in to reach stationarity."""
    x = np.zeros(burn_in)
    for t in range(1, burn_in):
        x[t] = c + rho * x[t-1] + rng.normal(0, sigma)

    x_full = np.zeros(n + burn_in)
    x_full[:burn_in] = x
    for t in range(burn_in, n + burn_in):
        x_full[t] = c + rho * x_full[t-1] + rng.normal(0, sigma)

    return x_full[burn_in:]  # stationary portion only

def sample_rho1(x):
    x_centered = x - np.mean(x)
    auto_cov_1 = np.mean(x_centered[:-1] * x_centered[1:])
    auto_var = np.mean(x_centered ** 2)
    return auto_cov_1 / auto_var

for n in [100, 1000, 10000]:
    x = simulate_ar1(n, c, rho_true, sigma, rng)
    print(f"n={n:6d}: sample rho1 = {sample_rho1(x):.6f} (true = {rho_true})")

Output:

n=   100: sample rho1 = 0.672382 (true = 0.7)
n=  1000: sample rho1 = 0.740188 (true = 0.7)
n= 10000: sample rho1 = 0.696657 (true = 0.7)

Notice the n = 1000 result: a miss of 0.04 in the wrong direction, and a bigger miss than at n = 100. That is not a bug — it is exactly what the sampling variance of the estimator predicts, as we verify next.

Dispersion: Bartlett’s formula, verified by Monte Carlo
#

For a stationary AR(1), the large-sample (Bartlett) approximate variance of the lag-1 sample autocorrelation is:

Var(ρ̂_1) ≈ (1 − ρ²) / n

For ρ = 0.7:

  • n = 100: theoretical sd ≈ √(0.51/100) = 0.071; observed deviation |0.672 − 0.7| = 0.028 — a 0.4σ event
  • n = 1000: theoretical sd ≈ 0.023; observed deviation 0.040 — a 1.7σ event, unremarkable
  • n = 10000: theoretical sd ≈ 0.0071; observed deviation 0.0033 — a 0.5σ event

One path is anecdote, so let’s verify the distribution itself over many replications:

from scipy.signal import lfilter

def rho1_estimates(n, reps=2000, rho=0.7, c=0.5, sigma=1.0, rng=rng, burn_in=100):
    """Vectorized AR(1) simulation via a linear filter — returns `reps` lag-1 ACF estimates."""
    eps = rng.normal(0.0, sigma, (reps, n + burn_in))
    x = lfilter([1.0], [1.0, -rho], eps, axis=1) + c / (1.0 - rho)
    x = x[:, burn_in:]
    xc = x - x.mean(axis=1, keepdims=True)
    return (xc[:, :-1] * xc[:, 1:]).mean(axis=1) / (xc ** 2).mean(axis=1)

print(f"{'n':>7} | {'emp mean':>9} | {'emp sd':>7} | {'theory sd':>9} | {'-1/n':>7}")
for n in [100, 1000, 10000]:
    r1 = rho1_estimates(n)
    theo = np.sqrt((1 - rho_true ** 2) / n)
    print(f"{n:>7} | {r1.mean():>9.4f} | {r1.std():>7.4f} | {theo:>9.4f} | {-1.0/n:>7.4f}")

Output:

      n |  emp mean |  emp sd | theory sd |    -1/n
    100 |    0.6684 |  0.0770 |    0.0714 | -0.0100
   1000 |    0.6969 |  0.0232 |    0.0226 | -0.0010
  10000 |    0.6998 |  0.0070 |    0.0071 | -0.0001

Reading the table
#

  1. Dispersion matches theory. The empirical standard deviations (0.077, 0.023, 0.0070) sit within ~8% of Bartlett’s formula (√((1−ρ²)/n) = 0.071, 0.023, 0.0071) and converge to it as n grows.
  2. There is a small-sample bias, not just noise. The empirical mean sits below ρ at every n — about −3.1/n (i.e. −(1 + 3ρ)/n): −0.031 at n = 100, −0.003 at n = 1000. At short samples the sample autocorrelation is downward-biased, which is exactly why ARIMA identification on short series needs care.
  3. So the honest statement is bivariate: the estimator converges to ρ and needs (1 − ρ²)/n + (1 + 3ρ)²/n²-scale error bars around it.

Conclusion
#

The AR(1) process demonstrates a clean connection between theory and simulation — but only if you track both moments of the estimator:

  1. For an AR(1), ρ_k = ρ^k, so the lag-1 sample autocorrelation estimates the AR coefficient directly.
  2. The estimator’s dispersion follows Bartlett’s formula (1 − ρ²)/n — verified here over 2000 replications and three sample sizes.
  3. At short samples there is a systematic downward bias of order (1 + 3ρ)/n — a reminder that point estimates on ~100 observations carry more structure than “close to truth”.

When building time series models, use these error bars: an AR(1) coefficient estimated as 0.67 on 100 observations is fully consistent with a true value anywhere in roughly [0.53, 0.81] — which matters when the next decision, like an ARIMA order, is based on whether the coefficient is “significant”.


This post includes fully reproducible code (fixed seeds throughout). Run it yourself and vary ρ, n, and σ — the Monte-Carlo block will confirm the same two laws.

Related

Kalman Filtering from Scratch in Python: Linear State Estimation with Ground Truth Verification

You observe a system’s position every second with a noisy GPS sensor, but you need to know both position and velocity for control. You have a physics model (constant velocity) but the measurements are unreliable. How do you recover the true state? The Kalman Filter answers this question: it estimates a system’s hidden state from noisy measurements, combining a prediction from a dynamic model with a correction from new measurements, weighted by their respective uncertainties. In this post, we build a complete Kalman Filter from scratch in NumPy, verify it against a known ground-truth trajectory, and show exactly how much uncertainty it reduces compared to raw measurements.

Pricing Options with Monte Carlo in NumPy — Verified Against Black-Scholes

Black-Scholes gives you a closed-form price for a vanilla European option in one line — so why bother simulating? Because the moment an option’s payoff depends on the path (Asian, lookback, barrier) or you add non-linearity (American exercise, Vol surfaces, XVA), the formulas stop existing and Monte Carlo keeps working. In this post we build a complete Monte Carlo pricer from scratch in NumPy, verify it against Black-Scholes, then extend it to path-dependent payoffs and Greeks. Every number shown comes from a single run with a fixed seed (numpy.random.default_rng(42)) — you should be able to reproduce all of them.

Build an Apple Home-Style Dashboard on a Mac or Raspberry Pi (Powered by Home Assistant)

I really like the Apple Home app: dark tiles, a badge that lights up, no thinking about protocols. What I don’t like is being limited to what Apple decides to surface, with the only escape hatch being HomeKit bridges. Here’s the deal I actually want: the same experience, self-hosted, running on the Mac (or a Raspberry Pi) that already sits on my home network, talking to Home Assistant — which is already the brain of the house. That turned out to be one Python file, standard library only. This post is a complete, verified walkthrough: every output below is from a real run.