Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

From Euclidean space to a manifold

A normal law has several coordinate descriptions: mean and standard deviation, mean and variance, natural parameters, or expected sufficient statistics. A change of coordinates changes the numbers used to describe a perturbation of the law. It should preserve the perturbation itself and its statistical size.

This note develops the geometry needed to make that distinction precise. We construct tangent vectors, use Fisher information to measure them, and define connections that differentiate and transport them. The family N(μ,σ2)N(\mu,\sigma^2) provides explicit calculations throughout. It also exhibits a distinction that matters later: its Fisher metric has negative Riemannian curvature, while its exponential and mixture connections are flat.

Notation and conventions

We follow the shared notation, with ξ\xi used locally in this note for a general chart. For normals, ξ=(μ,σ)T\xi=(\mu,\sigma)^{\mathsf T}; θ\theta is reserved for natural parameters, and η\eta for expectation parameters. This local use of ξ\xi is unrelated to the extreme-value index in the Incerto track.

We use matrices for explicit Jacobians and the Legendre identities η=∇ψ(θ)\eta=\nabla\psi(\theta) and ∇2ψ∗(η)=(∇2ψ(θ))−1\nabla^2\psi^*(\eta)=(\nabla^2\psi(\theta))^{-1}. Here the gradients and Hessians are ordinary derivatives in the named affine coordinates. Their entries will be related to tensor components explicitly. Curve parameters are denoted by rr, leaving t(x)t(x) for sufficient statistics. U,V,WU,V,W denote vector fields, while XX denotes an observation.

A running example: normal location and scale

Let MM be the set of laws N(μ,σ2)N(\mu,\sigma^2) with μ∈R\mu\in\mathbb R and σ>0\sigma>0. Relative to Lebesgue measure on R\mathbb R,

pξ(x)=12π σexp⁡ ⁣(−(x−μ)22σ2),ξ=(μ,σ)T.p_\xi(x)=\frac{1}{\sqrt{2\pi}\,\sigma} \exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right), \qquad \xi=(\mu,\sigma)^{\mathsf T}.

The sample space is one-dimensional, but the family of laws is two-dimensional. The mean and variance determine each law uniquely. We exclude σ=0\sigma=0, whose point mass is outside this density model.

Four charts on the same family

The following are global charts on MM. The symbol τ\tau is local notation for variance.

ChartCoordinatesCoordinate domain
Location and scaleξ=(μ,σ)T\xi=(\mu,\sigma)^{\mathsf T}σ>0\sigma>0
Location and varianceϕ=(μ,τ)T\phi=(\mu,\tau)^{\mathsf T}, τ=σ2\tau=\sigma^2τ>0\tau>0
Naturalθ=(μ/σ2,−1/(2σ2))T\theta=(\mu/\sigma^2,-1/(2\sigma^2))^{\mathsf T}θ2<0\theta^2<0
Expectationη=(μ,μ2+σ2)T\eta=(\mu,\mu^2+\sigma^2)^{\mathsf T}η2>(η1)2\eta^2>(\eta^1)^2

With t(x)=(x,x2)Tt(x)=(x,x^2)^{\mathsf T}, completing the square gives

pθ(x)=exp⁡{θTt(x)−ψ(θ)},ψ(θ)=−(θ1)24θ2+12log⁡π−θ2,η=Eθ[t(X)]=∇ψ(θ)=(μ,μ2+σ2)T.\begin{aligned} p_\theta(x)&=\exp\{\theta^{\mathsf T}t(x)-\psi(\theta)\},\\ \psi(\theta)&=-\frac{(\theta^1)^2}{4\theta^2} +\frac12\log\frac{\pi}{-\theta^2},\\ \eta&=\mathbb E_\theta[t(X)]=\nabla\psi(\theta) =(\mu,\mu^2+\sigma^2)^{\mathsf T}. \end{aligned}

Thus the second expectation coordinate is the second raw moment, not the variance. The exponential-family note derives these coordinates in the broader setting of sufficient statistics; its statistic TT is written tt here, following the shared canon.

The inverse coordinate maps include

μ=−θ12θ2,σ=−12θ2;μ=η1,σ=η2−(η1)2.\mu=-\frac{\theta^1}{2\theta^2},\qquad \sigma=\sqrt{-\frac1{2\theta^2}}; \qquad \mu=\eta^1,\qquad \sigma=\sqrt{\eta^2-(\eta^1)^2}.

Write fϕf_\phi, fθf_\theta, and fηf_\eta for the maps from the ξ\xi chart to these three charts. Direct differentiation gives

Dfϕ(ξ)=(1002σ),Dfθ(ξ)=(σ−2−2μσ−30σ−3),Dfη(ξ)=(102μ2σ).\begin{aligned} Df_\phi(\xi)&=\begin{pmatrix}1&0\\0&2\sigma\end{pmatrix},\\[3pt] Df_\theta(\xi)&=\begin{pmatrix} \sigma^{-2}&-2\mu\sigma^{-3}\\0&\sigma^{-3} \end{pmatrix},\\[3pt] Df_\eta(\xi)&=\begin{pmatrix}1&0\\2\mu&2\sigma\end{pmatrix}. \end{aligned}

Their determinants are nonzero for σ>0\sigma>0. The inverse Jacobian is Df−1(f(ξ))=Df(ξ)−1Df^{-1}(f(\xi))=Df(\xi)^{-1}; specifying its evaluation point prevents confusing the two directions of a coordinate change.

What makes this a manifold

A smooth dd-manifold is a Hausdorff, second-countable topological space covered by coordinate neighborhoods, each homeomorphic to an open subset of Rd\mathbb R^d, with smooth transition maps between these charts. A chart assigns coordinates to points; its inverse interprets coordinate values as points of the manifold. The normal family obtains such a structure from its bijection with R×(0,∞)\mathbb R\times(0,\infty), and the other charts above are smoothly compatible with it.

A general parameterized statistical model needs more care. Parameters may repeat the same law, and even an injective parameterization can have a vanishing first derivative at a point. We work on regular regions where the model is locally identifiable and the Fisher matrix is smooth and positive definite. The normal calculation below verifies that condition. A statistical manifold need not be presented as a curved surface inside a Euclidean space.

Tangent vectors, functions, and differentials

Curves define tangent vectors

Fix q∈Mq\in M and a chart ξ\xi containing it. Two smooth curves c1,c2c_1,c_2 through qq at r=0r=0 are equivalent when

ddrξi(c1(r))∣r=0=ddrξi(c2(r))∣r=0for every i.\left.\frac{d}{dr}\xi^i(c_1(r))\right|_{r=0} = \left.\frac{d}{dr}\xi^i(c_2(r))\right|_{r=0} \quad\text{for every }i.

The chain rule makes this equivalence independent of the chart. A tangent vector is an equivalence class v=[c]v=[c] of such curves. Its components are vi=(ξi∘c)′(0)v^i=(\xi^i\circ c)'(0). Every column of dd real numbers is realized by a coordinate line ξ(c(r))=ξ(q)+rv\xi(c(r))=\xi(q)+rv for sufficiently small rr. These components give the set of classes a dd-dimensional vector-space structure, called TqMT_qM; coordinate changes act linearly on that space.

For normals, v=vμ∂μ+vσ∂σv=v^\mu\partial_\mu+v^\sigma\partial_\sigma is represented by (μ(r),σ(r))=(μ+rvμ,σ+rvσ)(\mu(r),\sigma(r))=(\mu+rv^\mu,\sigma+rv^\sigma) near r=0r=0. The coordinates of a law and the components of a tangent vector have different roles: one specifies a distribution; the other specifies rates of change at that distribution.

The equivalent definition by derivations

A derivation at qq is an R\mathbb R-linear map A:C∞(M)→RA:C^\infty(M)\to\mathbb R satisfying the pointwise Leibniz rule

A(FH)=F(q)A(H)+H(q)A(F).A(FH)=F(q)A(H)+H(q)A(F).

It depends only on functions near qq. Indeed, if FF vanishes near qq, choose a smooth cutoff equal to one at qq and supported where F=0F=0; applying the product rule to their zero product gives A(F)=0A(F)=0. Thus we can also work with germs of smooth functions at qq, and apply AA to local coordinate functions.

A curve class defines such a derivation by

v[F]=ddrF(c(r))∣r=0=vi∂iF(q),∂i∣q[F]=∂(F∘ξ−1)∂ξi∣ξ(q).v[F]=\left.\frac{d}{dr}F(c(r))\right|_{r=0} =v^i\partial_iF(q),\qquad \partial_i\big|_q[F] =\left.\frac{\partial(F\circ\xi^{-1})}{\partial\xi^i}\right|_{\xi(q)}.

We now show that every derivation arises this way. On a small coordinate ball, the integral form of first-order Taylor expansion writes a smooth function as

F(y)−F(q)=(ξi(y)−ξi(q))Hi(y),Hi(q)=∂iF(q).F(y)-F(q)=(\xi^i(y)-\xi^i(q))H_i(y),\qquad H_i(q)=\partial_iF(q).

A derivation annihilates constants, since A(1)=A(1⋅1)=2A(1)A(1)=A(1\cdot1)=2A(1). Applying its product rule to this expansion gives A(F)=A(ξi)∂iF(q)A(F)=A(\xi^i)\partial_iF(q). Hence A=vi∂i∣qA=v^i\partial_i|_q with vi=A(ξi)v^i=A(\xi^i), represented by the coordinate line above. The two constructions are inverse, and {∂i∣q}\{\partial_i|_q\} is a basis of TqMT_qM.

For example, the smooth function F(Pμ,σ)=Eξ[X2]=μ2+σ2F(P_{\mu,\sigma})=\mathbb E_\xi[X^2] =\mu^2+\sigma^2 satisfies

∂μF=2μ,∂σF=2σ,v[F]=2μvμ+2σvσ.\partial_\mu F=2\mu,\qquad \partial_\sigma F=2\sigma, \qquad v[F]=2\mu v^\mu+2\sigma v^\sigma.

Here ∂μ\partial_\mu holds σ\sigma fixed, and ∂σ\partial_\sigma holds μ\mu fixed. Holding a different second coordinate fixed can produce a different basis vector even if the first coordinate has the same name.

Coordinate changes and pushforwards

Let ϕ=f(ξ)\phi=f(\xi) be a transition map and set Aai=∂ϕa/∂ξiA^a{}_i=\partial\phi^a/\partial\xi^i and Bia=∂ξi/∂ϕaB^i{}_a=\partial\xi^i/\partial\phi^a, so B=A−1B=A^{-1}. The chain rule gives

∂i=Aai∂a′,v′a=Aaivi,vϕ=Df(ξ)vξ.\partial_i=A^a{}_i\partial'_a,\qquad v'^a=A^a{}_i v^i,\qquad v_\phi=Df(\xi)v_\xi.

The basis and components change together, leaving the derivation vv fixed. For the variance chart, (4) gives

∂μ∣σ=∂μ∣τ,∂σ=2σ∂τ,vτ=2σvσ.\partial_\mu\big|_\sigma=\partial_\mu\big|_\tau,\qquad \partial_\sigma=2\sigma\partial_\tau,\qquad v^\tau=2\sigma v^\sigma.

At σ=2\sigma=2, a unit scale velocity has variance velocity 4. In expectation coordinates the same vector has components (vμ,2μvμ+2σvσ)T(v^\mu,2\mu v^\mu+2\sigma v^\sigma)^{\mathsf T}.

More generally, a smooth map f:M→Nf:M\to N has a differential, or pushforward, defined intrinsically by

dfq([c])=[f∘c],(dfqv)[H]=v[H∘f],H∈C∞(N).df_q([c])=[f\circ c],\qquad (df_qv)[H]=v[H\circ f],\quad H\in C^\infty(N).

In source coordinates ξ\xi and target coordinates ϕ\phi, its matrix is the Jacobian of ϕ∘f∘ξ−1\phi\circ f\circ\xi^{-1}, denoted Df(ξ)Df(\xi). Its entries are ∂fa/∂ξi\partial f^a/\partial\xi^i, and (dfqv)a=(∂fa/∂ξi)vi(df_qv)^a=(\partial f^a/\partial\xi^i)v^i. Unlike a coordinate change, a smooth map need not be invertible or preserve dimension. The variance map f:M→(0,∞)f:M\to(0,\infty), f(Pμ,σ)=σ2f(P_{\mu,\sigma})=\sigma^2, has dfqv=2σvσ∂τdf_qv=2\sigma v^\sigma\partial_\tau: it discards the location direction.

Covectors and vector fields

A cotangent vector is a linear functional ω:TqM→R\omega:T_qM\to\mathbb R. The differentials dξid\xi^i form the dual basis because dξi(∂j)=δijd\xi^i(\partial_j)=\delta^i{}_j. Thus

ω=ωi dξi,ω(v)=ωivi,dF=(∂iF) dξi.\omega=\omega_i\,d\xi^i,\qquad \omega(v)=\omega_i v^i,\qquad dF=(\partial_iF)\,d\xi^i.

Under the same transition map, dϕa=Aaidξid\phi^a=A^a{}_i d\xi^i and ωa′=Biaωi\omega'_a=B^i{}_a\omega_i. This inverse transformation is what makes ω(v)\omega(v) independent of coordinates.

A smooth vector field VV assigns Vq∈TqMV_q\in T_qM at every point, with smooth components ViV^i in every chart. It differentiates functions by V[F]=Vi∂iFV[F]=V^i\partial_iF. On the normal family, V=σ∂σV=\sigma\partial_\sigma describes relative scale change. Its integral curves satisfy μ˙=0\dot\mu=0, σ˙=σ\dot\sigma=\sigma, so σ(r)=σ(0)er\sigma(r)=\sigma(0)e^r. In the variance chart this same field is 2τ∂τ2\tau\partial_\tau.

Metrics and Fisher information

A metric is more than a parameter norm

A Riemannian metric gg assigns a smooth positive definite symmetric bilinear form gqg_q to every tangent space. Its components are gij=g(∂i,∂j)g_{ij}=g(\partial_i,\partial_j). Substituting vi=Biav′av^i=B^i{}_a v'^a and wj=Bjbw′bw^j=B^j{}_b w'^b gives

gab′(ϕ)=BiaBjbgij(ξ),Gϕ(ϕ)=BTGξ(ξ)B=Df(ξ)−TGξ(ξ)Df(ξ)−1,ϕ=f(ξ).\begin{aligned} g'_{ab}(\phi)&=B^i{}_a B^j{}_b g_{ij}(\xi),\\ G_\phi(\phi)&=B^{\mathsf T}G_\xi(\xi)B =Df(\xi)^{-\mathsf T}G_\xi(\xi)Df(\xi)^{-1}, \qquad \phi=f(\xi). \end{aligned}

Every Riemannian metric has this covariance. It does not by itself select Fisher information. Even a Euclidean metric acquires nonconstant coefficients in nonlinear coordinates. On an angular sector of the Euclidean plane, write x=ρcos⁡φx=\rho\cos\varphi, y=ρsin⁡φy=\rho\sin\varphi, with radius ρ>0\rho>0. Then

ds2=dx2+dy2=dρ2+ρ2dφ2.ds^2=dx^2+dy^2=d\rho^2+\rho^2d\varphi^2.

Thus nonconstant metric coefficients alone do not establish curvature.

For normals, choosing dμ2+dσ2d\mu^2+d\sigma^2 in the scale chart and choosing dμ2+dτ2d\mu^2+d\tau^2 afresh in the variance chart gives incompatible metrics:

dμ2+dτ2=dμ2+4σ2dσ2.d\mu^2+d\tau^2=d\mu^2+4\sigma^2d\sigma^2.

At σ=2\sigma=2, the same unit scale velocity has squared lengths 1 and 16. A Euclidean metric chosen in one chart is legitimate, but must then be transformed by (14). A statistical principle is needed to choose a metric without privileging that chart.

For a piecewise smooth curve c:[a,b]→Mc:[a,b]\to M, the metric determines length and intrinsic distance by

Lg(c)=∫abgij(ξ(c(r)))ξ˙iξ˙j dr,dg(q0,q1)=inf⁡c:q0→q1Lg(c).L_g(c)=\int_a^b\sqrt{g_{ij}(\xi(c(r)))\dot\xi^i\dot\xi^j}\,dr, \qquad d_g(q_0,q_1)=\inf_{c:q_0\to q_1}L_g(c).

The infimum is over curves in the specified manifold. Restricting to a subfamily can increase this distance.

Scores measure changes in the law

For a general model, assume positive densities on a common support relative to a fixed measure ν\nu. Assume smooth parameter dependence, square-integrable scores, and local integrable bounds allowing differentiation of the normalizer and the expectations used below. We will specify the additional Taylor and third-moment assumptions where they are needed.

Differentiating the density and its normalization gives

∂ipξ=pξsi,Eξ[si]=0,ddrℓξ(c(r))(x)∣r=0=si(x)vi.\partial_i p_\xi=p_\xi s_i,\qquad \mathbb E_\xi[s_i]=0,\qquad \left.\frac{d}{dr}\ell_{\xi(c(r))}(x)\right|_{r=0}=s_i(x)v^i.

The Fisher metric per observation is

gξ(v,w)=Eξ[(sivi)(sjwj)],gij(ξ)=Eξ[sisj],I(ξ)=Eξ[sξsξT].\begin{aligned} g_\xi(v,w)&=\mathbb E_\xi[(s_i v^i)(s_j w^j)],\\ g_{ij}(\xi)&=\mathbb E_\xi[s_i s_j],\qquad I(\xi)=\mathbb E_\xi[s_\xi s_\xi^{\mathsf T}]. \end{aligned}

It is positive semidefinite; our regularity assumption requires positive definiteness. A zero length direction would have zero first-order density change almost everywhere. Under reparameterization the score obeys sa′=Biasis'_a=B^i{}_a s_i, so its covariance obeys exactly the metric transformation law. Invertible, parameter-independent changes of the observed variable also preserve Fisher information: their density Jacobian contributes a term to the log density whose parameter derivative is zero.

Computing the normal metric

Set Z=(X−μ)/σ∼N(0,1)Z=(X-\mu)/\sigma\sim N(0,1). Differentiating (1),

sμ=X−μσ2=Zσ,sσ=−1σ+(X−μ)2σ3=Z2−1σ.s_\mu=\frac{X-\mu}{\sigma^2}=\frac Z\sigma, \qquad s_\sigma=-\frac1\sigma+\frac{(X-\mu)^2}{\sigma^3} =\frac{Z^2-1}{\sigma}.

The subscripts here denote covector components, not differentiation of an already defined score. Odd standard-normal moments vanish, and integration by parts gives E[Z2k]=(2k−1)E[Z2k−2]\mathbb E[Z^{2k}]=(2k-1)\mathbb E[Z^{2k-2}]. In particular, the second and fourth moments are 1 and 3. Consequently,

Gξ=I(ξ)=(σ−2002σ−2),gξ(v,w)=vμwμ+2vσwσσ2,ds2=dμ2+2dσ2σ2.\begin{aligned} G_\xi=I(\xi)&=\begin{pmatrix}\sigma^{-2}&0\\0&2\sigma^{-2}\end{pmatrix},\\ g_\xi(v,w)&=\frac{v^\mu w^\mu+2v^\sigma w^\sigma}{\sigma^2},\\ ds^2&=\frac{d\mu^2+2d\sigma^2}{\sigma^2}. \end{aligned}

This is positive definite for every σ>0\sigma>0. On compact parameter neighborhoods with scale bounded away from zero, Gaussian tail bounds justify the differentiations and moments used here and later. A fixed location velocity has smaller Fisher length at a larger scale, because its change in log density is smaller in mean square.

Using B=diag⁡(1,1/(2σ))B=\operatorname{diag}(1,1/(2\sigma)) in the variance chart gives

Gϕ=(τ−100(2τ2)−1),ds2=dμ2+2dσ2σ2=dμ2τ+dτ22τ2.G_\phi=\begin{pmatrix}\tau^{-1}&0\\0&(2\tau^2)^{-1}\end{pmatrix}, \qquad ds^2=\frac{d\mu^2+2d\sigma^2}{\sigma^2} =\frac{d\mu^2}{\tau}+\frac{d\tau^2}{2\tau^2}.

Substitution of vτ=2σvσv^\tau=2\sigma v^\sigma recovers the same squared length.

Natural and expectation coordinates

In natural coordinates sθ=t−ηs_\theta=t-\eta, so differentiating the normalizer in (2) gives the matrix identities

Gθ(θ)=I(θ)=∇2ψ(θ)=Cov⁡θ(t(X))=(σ22μσ22μσ24μ2σ2+2σ4),ψ∗(η)=θTη−ψ(θ)=−12log⁡ ⁣(2πe[η2−(η1)2]),Gη(η)=∇2ψ∗(η)=Gθ(θ(η))−1=(τ−1+2μ2τ−2−μτ−2−μτ−2(2τ2)−1),μ=η1,τ=η2−(η1)2.\begin{aligned} G_\theta(\theta)=I(\theta) &=\nabla^2\psi(\theta)=\operatorname{Cov}_\theta(t(X))\\ &=\begin{pmatrix} \sigma^2&2\mu\sigma^2\\ 2\mu\sigma^2&4\mu^2\sigma^2+2\sigma^4 \end{pmatrix},\\ \psi^*(\eta)&=\theta^{\mathsf T}\eta-\psi(\theta) =-\frac12\log\!\left(2\pi e[\eta^2-(\eta^1)^2]\right),\\ G_\eta(\eta)&=\nabla^2\psi^*(\eta)=G_\theta(\theta(\eta))^{-1}\\ &=\begin{pmatrix} \tau^{-1}+2\mu^2\tau^{-2}&-\mu\tau^{-2}\\ -\mu\tau^{-2}&(2\tau^2)^{-1} \end{pmatrix},\quad \mu=\eta^1,\quad \tau=\eta^2-(\eta^1)^2. \end{aligned}

The expression for ψ∗\psi^* is the Legendre transform, attained at the natural parameter corresponding to η\eta. Indeed, dη=Gθdθd\eta=G_\theta d\theta, whence dθ=Gθ−1dηd\theta=G_\theta^{-1}d\eta; the metric transformation law gives GηG_\eta. Here (Gθ)ij=∂2ψ/(∂θi∂θj)(G_\theta)_{ij}=\partial^2\psi/(\partial\theta^i\partial\theta^j) and (Gη)ab=∂2ψ∗/(∂ηa∂ηb)(G_\eta)_{ab}=\partial^2\psi^*/(\partial\eta^a\partial\eta^b). The inverse of one coordinate matrix supplies the components in the other chart; it does not change the covariant type of the metric. The Jacobians in (4) pull both matrices back to GξG_\xi. An ordinary Hessian need not transform as a tensor under arbitrary nonlinear coordinates; these Hessian formulas use the particular affine coordinates θ\theta and η\eta.

Why Fisher information is distinguished

Coordinate covariance is required of every metric. Fisher information has additional properties concerning statistical distinguishability and the loss of observations. We prove the local KL and monotonicity statements below; the uniqueness theorem is cited with its finite-space hypotheses.

The Hessian of KL at the diagonal

Use the direction D(pξ∥pζ)=Eξ[ℓξ−ℓζ]D(p_\xi\|p_\zeta)=\mathbb E_\xi[\ell_\xi-\ell_\zeta], holding the first law fixed. Differentiating normalization twice yields

0=Eξ[∂i∂jℓξ+sisj],Eξ[∂i∂jℓξ]=−gij.0=\mathbb E_\xi[\partial_i\partial_j\ell_\xi+s_i s_j], \qquad \mathbb E_\xi[\partial_i\partial_j\ell_\xi]=-g_{ij}.

Assume in addition that ζ↦Eξ[ℓζ]\zeta\mapsto\mathbb E_\xi[\ell_\zeta] has a third-order Taylor remainder bounded by a constant times ∥ζ−ξ∥3\lVert\zeta-\xi\rVert^3 locally. For example, locally bounded integrable third derivatives of the log density under PξP_\xi suffice. Taylor expansion then gives

D(pξ∥pξ+δξ)=12gij(ξ) δξiδξj+O(∥δξ∥3).D(p_\xi\|p_{\xi+\delta\xi}) =\frac12 g_{ij}(\xi)\,\delta\xi^i\delta\xi^j +O(\lVert\delta\xi\rVert^3).

The zero expected score removes the linear term. The Hessian in the second argument at equality is therefore gijg_{ij}; because the first derivative vanishes there, this Hessian transforms tensorially. With only a second-order Taylor expansion, the remainder is o(∥δξ∥2)o(\lVert\delta\xi\rVert^2), as in the Fisher-versus-L2L^2 note. Finite KL remains asymmetric and is not half the squared Fisher–Rao distance.

For normals the exact expression is

D ⁣(N(μ,σ2)∥N(μ~,σ~2))=log⁡σ~σ−12+σ2+(μ−μ~)22σ~2.D\!\left(N(\mu,\sigma^2)\middle\|N(\widetilde\mu,\widetilde\sigma^2)\right) =\log\frac{\widetilde\sigma}{\sigma}-\frac12 +\frac{\sigma^2+(\mu-\widetilde\mu)^2}{2\widetilde\sigma^2}.

Putting (μ~,σ~)=(μ+εa,σ+εb)(\widetilde\mu,\widetilde\sigma) =(\mu+\varepsilon a,\sigma+\varepsilon b) at a fixed σ>0\sigma>0 gives ε2(a2+2b2)/(2σ2)+O(ε3)\varepsilon^2(a^2+2b^2)/(2\sigma^2)+O(\varepsilon^3), in agreement with (21).

Coarse-graining, monotonicity, and sufficiency

Let an observation XX pass through a parameter-independent Markov kernel K(dy∣x)K(dy\mid x), giving YY. A deterministic statistic Y=S(X)Y=S(X) is a special case. Assume the transformed model admits densities and differentiation under the kernel integral, and that the original scores are square integrable. Denote the two score covectors by siXs^X_i and siYs^Y_i, and their component columns by sξXs^X_\xi and sξYs^Y_\xi. Differentiating the marginal density of YY gives

siY(Y)=Eξ[siX(X)∣Y].s^Y_i(Y)=\mathbb E_\xi[s^X_i(X)\mid Y].

Thus the new directional score is the conditional expectation of the old one. The conditional-variance identity proves information monotonicity:

gξX(v,v)−gξY(v,v)=Eξ ⁣[Var⁡ξ(visiX(X)∣Y)]≥0,IX(ξ)−IY(ξ)=Eξ[Cov⁡ξ(sξX∣Y)]⪰0.\begin{aligned} g^X_\xi(v,v)-g^Y_\xi(v,v) &=\mathbb E_\xi\!\left[ \operatorname{Var}_\xi(v^i s^X_i(X)\mid Y)\right]\geq0,\\ I_X(\xi)-I_Y(\xi) &=\mathbb E_\xi[\operatorname{Cov}_\xi(s^X_\xi\mid Y)]\succeq0. \end{aligned}

The superscripts X,YX,Y identify experiments; they are not tensor indices. The output information can be singular. Equality for one vector at one parameter means exactly that its directional score is determined by YY almost surely. Equality of the matrices at that parameter means this for every score component. This is a statement about first-order information at that point.

A statistic is sufficient for the family if the conditional distribution of XX given the statistic can be chosen independently of the parameter. Equivalently in the regular dominated setting, the density factors into a parameter-dependent function of the statistic and a parameter-independent factor. Its scores are functions of the statistic, so sufficiency implies equality of Fisher information throughout the family.

For a precise converse, take a finite sample space with strictly positive smooth probabilities on a connected open parameter domain, and a fixed statistic SS. Equality of Fisher information at every parameter is then equivalent to sufficiency. To prove the converse, equality in (28) gives, for every outcome,

∂ilog⁡Pξ(X=x∣S(X)=S(x))=siX(x)−siS(S(x))=0.\partial_i\log P_\xi(X=x\mid S(X)=S(x)) =s^X_i(x)-s^S_i(S(x))=0.

Connectedness makes each conditional probability constant in the parameter, which proves sufficiency. Equality at one parameter, or in one direction, is insufficient for this conclusion. Extensions beyond this finite positive setting need corresponding regularity assumptions; see Ay, Jost, Lê, and Schwachhöfer on sufficient statistics.

The normal family illustrates both preservation and loss. For independent observations X1,…,XnX_1,\ldots,X_n, the statistic S=(∑aXa,∑aXa2)S=(\sum_a X_a,\sum_a X_a^2) is sufficient by the factorization in (2). Its information equals that of the full sample, namely nI(ξ)nI(\xi): independent zero-mean scores add and their cross-covariances vanish. By contrast, retaining only Y=1{X>0}Y=\mathbf 1_{\{X>0\}} gives a Bernoulli probability q=Φ(μ/σ)q=\Phi(\mu/\sigma), where Φ\Phi is the standard-normal cdf. At μ=0\mu=0, ∂μq=1/(2πσ)\partial_\mu q=1/(\sqrt{2\pi}\sigma) and ∂σq=0\partial_\sigma q=0, so

IY(0,σ)=(2/(πσ2)000).I_Y(0,\sigma)=\begin{pmatrix}2/(\pi\sigma^2)&0\\0&0\end{pmatrix}.

The sign loses all local scale information and retains only a fraction 2/π2/\pi of the location information there.

Čencov’s finite-space uniqueness theorem

The theorem concerns a compatible choice of metrics on all finite probability simplices, not an arbitrary metric on one fixed normal family. Let

Δn−1∘={p∈Rn:p(a)>0, ∑a=1np(a)=1},TpΔn−1∘={u∈Rn:∑a=1nu(a)=0}.\Delta_{n-1}^{\circ} =\left\{p\in\mathbb R^n:p(a)>0,\ \sum_{a=1}^n p(a)=1\right\}, \qquad T_p\Delta_{n-1}^{\circ} =\left\{u\in\mathbb R^n:\sum_{a=1}^n u(a)=0\right\}.

A Markov map is a stochastic linear map (Kp)(b)=∑aK(b∣a)p(a)(Kp)(b)=\sum_a K(b\mid a)p(a). A congruent Markov embedding has a Markov left inverse LL with LK=idLK=\mathrm{id} and maps the interior into the interior. It splits each input outcome into a disjoint block of output outcomes with fixed conditional probabilities; the block label recovers the input. Such a splitting preserves the statistical experiment.

Čencov’s theorem states that if smooth Riemannian metrics h(n)h^{(n)} on Δn−1∘\Delta_{n-1}^{\circ}, for every n≥2n\geq2, make every congruent Markov embedding an isometry, then there is a single constant c>0c>0, independent of nn and pp, such that

hp(n)(u,w)=c∑a=1nu(a)w(a)p(a).h^{(n)}_p(u,w)=c\sum_{a=1}^n\frac{u(a)w(a)}{p(a)}.

Conversely, these metrics have that invariance. We cite this classification, not prove it: see Čencov’s Statistical Decision Rules and Optimal Inference (1982) and the explicit finite-simplex formulation in Theorem 2 of Montúfar, Rauh, and Ay (2014).

Pulling (32) back along a regular finite statistical model, with u(a)=∂ipξ(a)vi=pξ(a)si(a)viu(a)=\partial_i p_\xi(a)v^i=p_\xi(a)s_i(a)v^i, gives cgξ(v,w)cg_\xi(v,w). We choose c=1c=1 for information per observation. General Markov maps are contractions by (28), whereas sufficient transformations preserve information. Requiring invariance under every Markov map would be wrong: a kernel can discard the observation completely. Equivalently, demanding monotonicity for a family of metrics under all such maps forces the isometries above, since KK and its Markov left inverse give opposite inequalities.

For general sample spaces, Ay et al. (2017) develop invariance and uniqueness results for suitable integrable models and continuous local tensor fields. A different extension by Bauer–Bruveris–Michor (2016) characterizes the Fisher metric among smooth weak Riemannian metrics invariant under observation-space diffeomorphisms on smooth positive probability densities over a closed manifold of dimension greater than one. Neither statement is an unrestricted uniqueness theorem for every continuous or singular statistical model; in particular, that latter theorem does not directly cover densities on R\mathbb R.

Density L2L^2 and square-root densities

The raw density metric, when finite, gijraw=∫(∂ip)(∂jp) dνg^{\mathrm{raw}}_{ij}=\int(\partial_i p)(\partial_j p)\,d\nu, also transforms correctly under parameter changes. It depends on the reference measure and the measurement coordinate: for Lebesgue densities and Y=aXY=aX, a>0a>0, ∫(pY−qY)2 dy=a−1∫(pX−qX)2 dx\int(p_Y-q_Y)^2\,dy=a^{-1}\int(p_X-q_X)^2\,dx. It therefore differs from Fisher geometry even though both constructions are covariant under reparameterization.

Assuming differentiability into L2(ν)L^2(\nu), the map p↦2pp\mapsto2\sqrt p has derivative ∂i(2p)=p si\partial_i(2\sqrt p)=\sqrt p\,s_i. Its pullback of the L2L^2 inner product is exactly gijg_{ij}. With H2(p,q)=12∫(p−q)2 dνH^2(p,q)=\tfrac12\int(\sqrt p-\sqrt q)^2\,d\nu, this gives H2(pξ,pξ+δξ)=18gijδξiδξj+o(∥δξ∥2)H^2(p_\xi,p_{\xi+\delta\xi})=\tfrac18g_{ij}\delta\xi^i\delta\xi^j +o(\lVert\delta\xi\rVert^2). The Fisher-versus-L2L^2 note explains the distinct roles of parameter norms, scores in L2(Pξ)L^2(P_\xi), and densities in L2(ν)L^2(\nu). A square-root embedding preserves tangent lengths; the distance between two normals still minimizes paths within the normal family, rather than over the whole sphere of square-root densities.

Connections differentiate vector fields

The ordinary derivative of vector components depends on the chosen chart. For example, a field with constant scale component has variance component 2σ2\sigma times as large, which changes along a scale curve. A connection supplies the correction that makes differentiation independent of coordinates. It is extra geometric structure; a metric singles out one connection only when we impose the conditions below.

Axioms and Christoffel symbols

An affine connection on MM is an R\mathbb R-bilinear operation (U,V)↦∇UV(U,V)\mapsto\nabla_UV on smooth vector fields satisfying, for every smooth function aa,

∇aUV=a∇UV,∇U(aV)=U[a]V+a∇UV.\nabla_{aU}V=a\nabla_UV,\qquad \nabla_U(aV)=U[a]V+a\nabla_UV.

Thus the lower argument UU is linear over functions; in the differentiated argument VV, the Leibniz term is essential. At a point, ∇UV\nabla_UV depends only on the value of UU and on the first-order behavior of VV nearby.

Define the Christoffel symbols in a coordinate basis by

∇∂i∂j=Γijk∂k.\nabla_{\partial_i}\partial_j=\Gamma_{ij}^k\partial_k.

Expanding U=Ui∂iU=U^i\partial_i and V=Vj∂jV=V^j\partial_j with the axioms gives

∇UV=Ui(∂iVk+ΓijkVj)∂k.\nabla_UV=U^i\bigl(\partial_iV^k+\Gamma_{ij}^kV^j\bigr)\partial_k.

The second term accounts for the change of the coordinate basis under the connection. Prescribing smooth Γijk\Gamma_{ij}^k defines a connection on a chart; prescriptions on overlapping charts must satisfy the next law.

Why Christoffel symbols are not a tensor

Use ϕ=f(ξ)\phi=f(\xi), Aai=∂ϕa/∂ξiA^a{}_i=\partial\phi^a/\partial\xi^i, and Bia=∂ξi/∂ϕaB^i{}_a=\partial\xi^i/\partial\phi^a as before. Apply the connection axioms to ∇Bia∂i(Bjb∂j)\nabla_{B^i{}_a\partial_i}(B^j{}_b\partial_j) to obtain

Γ′abc=AckBiaBjbΓijk+Ack∂2ξk∂ϕa∂ϕb.\Gamma'{}^c_{ab} =A^c{}_k B^i{}_a B^j{}_b\Gamma_{ij}^k +A^c{}_k\frac{\partial^2\xi^k}{\partial\phi^a\partial\phi^b}.

A (1,2)(1,2)-tensor has only the first term. The inhomogeneous second derivative is why coefficients that vanish in one chart need not vanish in another. For example, take the connection with Γijk=0\Gamma_{ij}^k=0 in the normal location-scale chart. In the variance chart, σ=τ\sigma=\sqrt\tau, so it has

Γ′τττ=2σd2τdτ2=−12τ.\Gamma'{}^{\tau}_{\tau\tau} =2\sigma\frac{d^2\sqrt\tau}{d\tau^2}=-\frac1{2\tau}.

This is a coordinate-flat connection chosen on the parameter domain; it is not yet the Fisher Levi-Civita connection. The example isolates the effect of the second term in (36).

The difference of two connections is a (1,2)(1,2)-tensor: their inhomogeneous terms cancel. Equivalently, the Leibniz terms cancel in ∇UV−∇~UV\nabla_UV-\widetilde\nabla_UV, leaving an expression linear over functions in both UU and VV. We will use this fact to compare statistical connections.

Along a curve: parallel transport and geodesics

For a curve c(r)c(r), let V(r)=Vk(r)∂k∣c(r)V(r)=V^k(r)\partial_k|_{c(r)} be a vector field along the curve. It need not extend to a single vector field on all of MM. Its covariant derivative is

DVdr:=∇c˙V=(dVkdr+Γijk(ξ(c(r)))ξ˙iVj)∂k.\frac{DV}{dr}:=\nabla_{\dot c}V =\left(\frac{dV^k}{dr} +\Gamma_{ij}^k(\xi(c(r)))\dot\xi^i V^j\right)\partial_k.

The transformation law makes this independent of coordinates. A field is parallel when DV/dr=0DV/dr=0. Given an initial vector, this linear ODE has a unique solution along a smooth curve on a compact parameter interval. It defines a linear isomorphism between the endpoint tangent spaces, called parallel transport. Reversing the curve gives its inverse. Transport generally depends on the path, not just its endpoints.

An affinely parameterized geodesic has parallel velocity:

∇c˙c˙=0,ξ¨k+Γijkξ˙iξ˙j=0.\nabla_{\dot c}\dot c=0, \qquad \ddot\xi^k+\Gamma_{ij}^k\dot\xi^i\dot\xi^j=0.

An arbitrary nonlinear reparameterization need not preserve this equation; affine changes of the parameter do. A connection defines geodesics without requiring a metric or a length-minimization interpretation.

Torsion, compatibility, and the Levi-Civita connection

The Lie bracket is the vector field [U,V][F]=U[V[F]]−V[U[F]][U,V][F]=U[V[F]]-V[U[F]]. In coordinates, [U,V]k=Ui∂iVk−Vi∂iUk[U,V]^k=U^i\partial_iV^k-V^i\partial_iU^k. The torsion of a connection is

T(U,V)=∇UV−∇VU−[U,V],T(∂i,∂j)=(Γijk−Γjik)∂k.T(U,V)=\nabla_UV-\nabla_VU-[U,V],\qquad T(\partial_i,\partial_j) =(\Gamma_{ij}^k-\Gamma_{ji}^k)\partial_k.

It is a tensor. A connection is torsion-free exactly when its lower Christoffel indices are symmetric in a coordinate basis.

A connection is metric-compatible, written ∇g=0\nabla g=0, when

U[g(V,W)]=g(∇UV,W)+g(V,∇UW).U[g(V,W)]=g(\nabla_UV,W)+g(V,\nabla_UW).

Equivalently, ∂igjk=Γijℓgℓk+Γikℓgjℓ\partial_i g_{jk}=\Gamma_{ij}^{\ell}g_{\ell k} +\Gamma_{ik}^{\ell}g_{j\ell}. Along a curve this product rule shows that parallel transport preserves inner products. In particular, a geodesic for a metric-compatible connection has constant speed.

The Levi-Civita connection ∇LC\nabla^{\mathrm{LC}} is the unique connection that is both torsion-free and compatible with gg. The identity that determines it is the Koszul formula:

2g(∇ULCV,W)=U[g(V,W)]+V[g(W,U)]−W[g(U,V)]+g([U,V],W)−g([V,W],U)+g([W,U],V).\begin{aligned} 2g(\nabla^{\mathrm{LC}}_UV,W) ={}&U[g(V,W)]+V[g(W,U)]-W[g(U,V)]\\ &+g([U,V],W)-g([V,W],U)+g([W,U],V). \end{aligned}

To see why this determines the connection, write the compatibility identity for the three cyclic choices of U,V,WU,V,W, add the first two and subtract the third, and replace ∇UV−∇VU\nabla_UV-\nabla_VU by [U,V][U,V]. This gives (42). Nondegeneracy of gg then uniquely determines ∇ULCV\nabla^{\mathrm{LC}}_UV. For existence, the right side is linear over functions in WW, so it defines a vector field by nondegeneracy of gg. The product rule gives the connection axioms; subtracting the expression with U,VU,V exchanged gives torsion-freeness, and adding the expressions with V,WV,W exchanged gives metric compatibility. In a coordinate basis the brackets vanish, yielding the computable formula

Γij(0)k=12gkℓ(∂igjℓ+∂jgiℓ−∂ℓgij).\Gamma^{(0)k}_{ij} =\frac12g^{k\ell} \left(\partial_i g_{j\ell}+\partial_j g_{i\ell} -\partial_\ell g_{ij}\right).

We write (0)(0) for Levi-Civita because it will be the α=0\alpha=0 statistical connection. We use the standard Riemannian result that its geodesics minimize length on sufficiently short segments. It does not say every geodesic segment is globally minimizing on every manifold. For background on this result and the connection framework, see §2 of Nielsen’s 2020 introduction.

Levi-Civita geometry of the normal family

Christoffel symbols and parallel transport

Substituting (21) into (43) gives exactly these nonzero coefficients:

Γμσ(0)μ=Γσμ(0)μ=−1σ,Γμμ(0)σ=12σ,Γσσ(0)σ=−1σ.\Gamma^{(0)\mu}_{\mu\sigma} =\Gamma^{(0)\mu}_{\sigma\mu}=-\frac1\sigma, \qquad \Gamma^{(0)\sigma}_{\mu\mu}=\frac1{2\sigma}, \qquad \Gamma^{(0)\sigma}_{\sigma\sigma}=-\frac1\sigma.

For example, Γμμ(0)σ=−12gσσ∂σgμμ=−12(σ2/2)(−2/σ3)=1/(2σ)\Gamma^{(0)\sigma}_{\mu\mu} =-\tfrac12 g^{\sigma\sigma}\partial_\sigma g_{\mu\mu} =-\tfrac12(\sigma^2/2)(-2/\sigma^3)=1/(2\sigma). This coefficient already shows why a horizontal line with constant scale and changing mean is not a Levi-Civita geodesic in the full family.

Along any fixed-mean curve c(r)=(μ0,σ(r))c(r)=(\mu_0,\sigma(r)), the parallel equations reduce to

V˙μ−σ˙σVμ=0,V˙σ−σ˙σVσ=0.\dot V^\mu-\frac{\dot\sigma}{\sigma}V^\mu=0, \qquad \dot V^\sigma-\frac{\dot\sigma}{\sigma}V^\sigma=0.

Thus a vector transported from scale σ0\sigma_0 to scale σ1\sigma_1 has components multiplied by σ1/σ0\sigma_1/\sigma_0. Its Fisher squared length ((Vμ)2+2(Vσ)2)/σ2( (V^\mu)^2+2(V^\sigma)^2 )/\sigma^2 stays fixed. Constant coordinate components would not give parallel transport along this curve.

Hyperbolic curvature and geodesics

Make the local coordinate substitution μ=2 u\mu=\sqrt2\,u. Then

ds2=2du2+dσ2σ2,σ>0.ds^2=2\frac{du^2+d\sigma^2}{\sigma^2},\qquad \sigma>0.

This is twice the Poincaré upper-half-plane metric. Its curvature is −1/2-1/2, not -1: multiplying a metric by 2 multiplies lengths by 2\sqrt2 and divides sectional curvature by 2.

We can verify the curvature directly from the connection. With the convention

R(U,V)W=∇U∇VW−∇V∇UW−∇[U,V]W,R(U,V)W=\nabla_U\nabla_VW-\nabla_V\nabla_UW-\nabla_{[U,V]}W,

(44) gives R(∂μ,∂σ)∂σ=−σ−2∂μR(\partial_\mu,\partial_\sigma)\partial_\sigma =-\sigma^{-2}\partial_\mu. Hence the Gaussian curvature is

K=g(R(∂μ,∂σ)∂σ,∂μ)gμμgσσ−gμσ2=−σ−42σ−4=−12.K=\frac{g(R(\partial_\mu,\partial_\sigma)\partial_\sigma,\partial_\mu)} {g_{\mu\mu}g_{\sigma\sigma}-g_{\mu\sigma}^2} =\frac{-\sigma^{-4}}{2\sigma^{-4}}=-\frac12.

The geodesic equations are

μ¨−2μ˙σ˙σ=0,σ¨+μ˙22σ−σ˙2σ=0.\ddot\mu-2\frac{\dot\mu\dot\sigma}{\sigma}=0, \qquad \ddot\sigma+\frac{\dot\mu^2}{2\sigma} -\frac{\dot\sigma^2}{\sigma}=0.

The first equation integrates to μ˙/σ2=constant\dot\mu/\sigma^2=\text{constant}. When this constant is zero, the solutions are vertical lines, μ(r)=c\mu(r)=c, σ(r)=Aebr\sigma(r)=A e^{br}, with A>0A>0. Otherwise, in the rescaled coordinate uu, the solutions trace semicircles orthogonal to the boundary σ=0\sigma=0. An affine parameterization in the original chart is

μ(r)=c+2 Rtanh⁡(br+a),σ(r)=Rsech⁡(br+a),R>0,b≠0,(μ−c)22+σ2=R2.\begin{aligned} \mu(r)&=c+\sqrt2\,R\tanh(br+a),\\ \sigma(r)&=R\operatorname{sech}(br+a),\qquad R>0,\quad b\ne0,\\ \frac{(\mu-c)^2}{2}+\sigma^2&=R^2. \end{aligned}

Substitution proves that these curves solve both equations and have constant squared speed 2b22b^2. They exhaust the nonvertical initial conditions: at any point with μ˙≠0\dot\mu\ne0, choose c=μ+2σσ˙/μ˙c=\mu+2\sigma\dot\sigma/\dot\mu, then choose R,a,bR,a,b to match the point and velocity. Local uniqueness for the geodesic ODE identifies the solution. The images are half-ellipses in (μ,σ)(\mu,\sigma) and semicircles in (u,σ)(u,\sigma); their parameterizations are not linear in the angle.

For endpoints (μ0,σ0)(\mu_0,\sigma_0) and (μ1,σ1)(\mu_1,\sigma_1) with different means, the center and radius are

c=μ12−μ02+2(σ12−σ02)2(μ1−μ0),R2=(μ0−c)22+σ02.c=\frac{\mu_1^2-\mu_0^2+2(\sigma_1^2-\sigma_0^2)}{2(\mu_1-\mu_0)}, \qquad R^2=\frac{(\mu_0-c)^2}{2}+\sigma_0^2.

Set a=artanh⁡((μ0−c)/(2R))a=\operatorname{artanh}((\mu_0-c)/(\sqrt2R)) and b=artanh⁡((μ1−c)/(2R))−ab=\operatorname{artanh}((\mu_1-c)/(\sqrt2R))-a in (50) to join them over 0≤r≤10\leq r\leq1. For equal means the corresponding segment is σ(r)=σ01−rσ1r\sigma(r)=\sigma_0^{1-r}\sigma_1^r. In this complete hyperbolic plane, these segments are the unique minimizing paths up to reparameterization.

Fisher–Rao distance between two normals

The upper-half-plane distance satisfies cosh⁡dH=1+((u1−u0)2+(σ1−σ0)2)/(2σ0σ1)\cosh d_H=1+((u_1-u_0)^2+(\sigma_1-\sigma_0)^2)/(2\sigma_0\sigma_1). Rescaling by (46) gives

dFR ⁣(N(μ0,σ02),N(μ1,σ12))=2 arcosh⁡ ⁣(1+(μ1−μ0)2+2(σ1−σ0)24σ0σ1).\begin{aligned} &d_{\mathrm{FR}}\!\left(N(\mu_0,\sigma_0^2),N(\mu_1,\sigma_1^2)\right)\\ &\quad=\sqrt2\,\operatorname{arcosh}\!\left( 1+\frac{(\mu_1-\mu_0)^2+2(\sigma_1-\sigma_0)^2} {4\sigma_0\sigma_1}\right). \end{aligned}

The hyperbolic minimizing-path and distance results used here are treated in Costa–Santos–Strapasson, Fisher information distance: a geometrical reading (2015), §2. Integrating the constant speed in (50) gives the same expression. In particular,

μ0=μ1:dFR=2∣log⁡σ1σ0∣,σ0=σ1=σ:dFR=2 arcosh⁡ ⁣(1+(μ1−μ0)24σ2).\begin{aligned} \mu_0=\mu_1:\quad d_{\mathrm{FR}}&=\sqrt2\left|\log\frac{\sigma_1}{\sigma_0}\right|,\\ \sigma_0=\sigma_1=\sigma:\quad d_{\mathrm{FR}}&=\sqrt2\,\operatorname{arcosh}\!\left( 1+\frac{(\mu_1-\mu_0)^2}{4\sigma^2}\right). \end{aligned}

In the fixed-scale one-dimensional subfamily, the distance is instead ∣μ1−μ0∣/σ|\mu_1-\mu_0|/\sigma. The full-family minimizing path can increase scale while changing location, then decrease it again. For distinct means it is strictly shorter than the horizontal fixed-scale path.

Statistical connections and duality

A regular statistical model has more structure than its Fisher metric: third moments of scores distinguish a family of connections. We give their definitions and normal-family calculations here. The duality note develops the resulting Bregman divergence and KL Pythagoras identities.

The cubic tensor and the alpha family

Assume sufficient smoothness and integrability to differentiate the Fisher metric, and finite absolute third score moments. Define the Amari–Chentsov cubic tensor by

Cijk=Eξ[sisjsk],C(U,V,W)=CijkUiVjWk.C_{ijk}=\mathbb E_\xi[s_i s_j s_k],\qquad C(U,V,W)=C_{ijk}U^iV^jW^k.

It is a symmetric covariant tensor because each score transforms as a covector. Raising the last index defines (C♯(U,V))k=gkℓCijℓUiVj(C^\sharp(U,V))^k=g^{k\ell}C_{ij\ell}U^iV^j.

For α∈R\alpha\in\mathbb R, the statistical α\alpha-connection has coefficients with the output index lowered

Γij,k(α)=Eξ ⁣[(∂i∂jℓξ+1−α2sisj)sk],Γij(α)k=gkℓΓij,ℓ(α).\begin{aligned} \Gamma^{(\alpha)}_{ij,k} &=\mathbb E_\xi\!\left[ \left(\partial_i\partial_j\ell_\xi +\frac{1-\alpha}{2}s_i s_j\right)s_k\right],\\ \Gamma^{(\alpha)k}_{ij}&=g^{k\ell}\Gamma^{(\alpha)}_{ij,\ell}. \end{aligned}

The comma separates the lowered output index; these coefficients themselves are not a covariant tensor. Under a change of chart, the second derivative of ℓ\ell contributes the inhomogeneous term in (36). The coefficients are symmetric in i,ji,j, so all these connections are torsion-free. We use the convention ∇(e)=∇(1)\nabla^{(e)}=\nabla^{(1)} and ∇(m)=∇(−1)\nabla^{(m)}=\nabla^{(-1)}.

We can identify their relation to Levi-Civita directly. Differentiation of an expectation obeys ∂iEξ[F]=Eξ[∂iF+Fsi]\partial_i\mathbb E_\xi[F]=\mathbb E_\xi[\partial_iF+Fs_i]. Applying it to gjkg_{jk} gives

∂igjk=Eξ[(∂i∂jℓ)sk]+Eξ[sj(∂i∂kℓ)]+Cijk.\partial_i g_{jk} =\mathbb E_\xi[(\partial_i\partial_j\ell)s_k] +\mathbb E_\xi[s_j(\partial_i\partial_k\ell)]+C_{ijk}.

Insert this into the lowered version of the Levi-Civita formula. The mixed terms cancel, leaving

Γij,k(0)=Eξ[(∂i∂jℓ)sk]+12Cijk,∇U(α)V=∇ULCV−α2C♯(U,V).\begin{aligned} \Gamma^{(0)}_{ij,k} &=\mathbb E_\xi[(\partial_i\partial_j\ell)s_k]+\frac12C_{ijk},\\ \nabla^{(\alpha)}_UV &=\nabla^{\mathrm{LC}}_UV-\frac\alpha2 C^\sharp(U,V). \end{aligned}

This proves that the α=0\alpha=0 connection is Levi-Civita and expresses the difference of connections as a tensor.

Dual connections

Two connections ∇\nabla and ∇∗\nabla^* are dual with respect to gg when

U[g(V,W)]=g(∇UV,W)+g(V,∇U∗W).U[g(V,W)]=g(\nabla_UV,W)+g(V,\nabla^*_UW).

Substitute (57) on the right, once with α\alpha and once with −α-\alpha. The cubic terms cancel by symmetry, and Levi-Civita compatibility supplies the left side. Thus ∇(α)\nabla^{(\alpha)} and ∇(−α)\nabla^{(-\alpha)} are dual, including the e/m pair. If VV is parallel for one connection and WW for its dual along the same curve, g(V,W)g(V,W) is constant. Neither transport separately needs to preserve g(V,V)g(V,V). Indeed, (∇U(α)g)(V,W)=αC(U,V,W)(\nabla^{(\alpha)}_U g)(V,W)=\alpha C(U,V,W), so nonzero α\alpha generally breaks metric compatibility.

These definitions and the sign convention agree with Amari & Nagaoka (2000), Methods of Information Geometry and Amari (2016), Information Geometry and Its Applications. The calculations above establish the displayed relations under the stated regularity assumptions.

The normal alpha-connections

For the scores Z/σZ/\sigma and (Z2−1)/σ(Z^2-1)/\sigma, use E[Z6]=15\mathbb E[Z^6]=15. The only nonzero cubic components are

Cμμσ=Cμσμ=Cσμμ=2σ3,Cσσσ=8σ3.C_{\mu\mu\sigma}=C_{\mu\sigma\mu}=C_{\sigma\mu\mu} =\frac2{\sigma^3},\qquad C_{\sigma\sigma\sigma}=\frac8{\sigma^3}.

For example, E[Z2(Z2−1)]=3−1=2\mathbb E[Z^2(Z^2-1)]=3-1=2 and E[(Z2−1)3]=15−9+3−1=8\mathbb E[(Z^2-1)^3]=15-9+3-1=8. Raising the output index in (57) gives the following coefficients; all others vanish:

Γμσ(α)μ=Γσμ(α)μ=−1+ασ,Γμμ(α)σ=1−α2σ,Γσσ(α)σ=−1+2ασ.\begin{aligned} \Gamma^{(\alpha)\mu}_{\mu\sigma} =\Gamma^{(\alpha)\mu}_{\sigma\mu}&=-\frac{1+\alpha}{\sigma},\\ \Gamma^{(\alpha)\sigma}_{\mu\mu}&=\frac{1-\alpha}{2\sigma},\\ \Gamma^{(\alpha)\sigma}_{\sigma\sigma}&=-\frac{1+2\alpha}{\sigma}. \end{aligned}

Setting α=0\alpha=0 recovers (44). The general geodesic equations become

μ¨−2(1+α)μ˙σ˙σ=0,σ¨+1−α2σμ˙2−1+2ασσ˙2=0.\ddot\mu-2(1+\alpha)\frac{\dot\mu\dot\sigma}{\sigma}=0, \qquad \ddot\sigma+\frac{1-\alpha}{2\sigma}\dot\mu^2 -\frac{1+2\alpha}{\sigma}\dot\sigma^2=0.

There is also an explicit transport comparison. Along a fixed-mean curve, α\alpha-parallel transport from scale σ0\sigma_0 to σ(r)\sigma(r) gives

Vμ(r)=Vμ(0)(σ(r)σ0)1+α,Vσ(r)=Vσ(0)(σ(r)σ0)1+2α.V^\mu(r)=V^\mu(0)\left(\frac{\sigma(r)}{\sigma_0}\right)^{1+\alpha}, \qquad V^\sigma(r)=V^\sigma(0)\left(\frac{\sigma(r)}{\sigma_0}\right)^{1+2\alpha}.

For the dual connection the exponents have the opposite α\alpha terms. The paired inner product stays constant, although a vector’s own Fisher norm generally changes when α≠0\alpha\ne0.

Natural and expectation coordinates make different connections flat

In the natural chart, ∂i∂jℓθ=−∂i∂jψ\partial_i\partial_j\ell_\theta=-\partial_i\partial_j\psi is independent of the observation. Since every score has expectation zero, (55) at α=1\alpha=1 gives Γij(e)k=0\Gamma^{(e)k}_{ij}=0. Also Cijk=∂i∂j∂kψC_{ijk}=\partial_i\partial_j\partial_k\psi in these coordinates, by differentiating the covariance of t(X)t(X).

The expectation chart makes the dual mixture coefficients vanish. Here is a coordinate argument. The natural coordinate vector fields Ei=∂/∂θiE_i=\partial/\partial\theta^i are e-parallel. Let Fa=∂/∂ηaF_a=\partial/\partial\eta^a. Since dθ=Gθ−1dηd\theta=G_\theta^{-1}d\eta,

g(Ei,Fa)=∑j(Gθ)ij(Gθ−1)ja={1,i=a,0,i≠a.g(E_i,F_a) =\sum_j (G_\theta)_{ij}(G_\theta^{-1})_{ja} =\begin{cases}1,&i=a,\\0,&i\ne a.\end{cases}

The explicit sum here multiplies matrices labeled by two different coordinate bases; the numeric identity compares their paired labels. Differentiate this constant pairing with any field UU and use e/m duality. Because ∇U(e)Ei=0\nabla^{(e)}_UE_i=0, we obtain g(Ei,∇U(m)Fa)=0g(E_i,\nabla^{(m)}_UF_a)=0 for every ii, hence ∇U(m)Fa=0\nabla^{(m)}_UF_a=0. In particular, all m-Christoffel symbols vanish in the η\eta chart.

Both connections therefore have zero curvature. This dual flatness coexists with the curvature −1/2-1/2 of ∇LC\nabla^{\mathrm{LC}}. Curvature is attached to a specified connection; the vanishing of the e- or m-coefficients in their affine charts does not make the Fisher metric Euclidean.

Three geodesics between two normal laws

Let P0=N(μ0,σ02)P_0=N(\mu_0,\sigma_0^2) and P1=N(μ1,σ12)P_1=N(\mu_1,\sigma_1^2) with positive scales, and let 0≤r≤10\leq r\leq1. A straight natural-coordinate segment is an affinely parameterized e-geodesic. In location-scale coordinates it is

θ(r)=(1−r)θ0+rθ1,σe2(r)=(1−rσ02+rσ12)−1,μe(r)=σe2(r)((1−r)μ0σ02+rμ1σ12).\begin{aligned} \theta(r)&=(1-r)\theta_0+r\theta_1,\\ \sigma_e^2(r)&=\left(\frac{1-r}{\sigma_0^2} +\frac r{\sigma_1^2}\right)^{-1},\\ \mu_e(r)&=\sigma_e^2(r) \left(\frac{(1-r)\mu_0}{\sigma_0^2} +\frac{r\mu_1}{\sigma_1^2}\right). \end{aligned}

It interpolates precision and precision-weighted mean. Its density is proportional to p01−rp1rp_0^{1-r}p_1^r, as follows by interpolating the natural form. The precision stays positive, so the whole segment belongs to the family.

A straight expectation-coordinate segment is an affinely parameterized m-geodesic of the connection on this normal family:

η(r)=(1−r)η0+rη1,μm(r)=(1−r)μ0+rμ1,σm2(r)=(1−r)σ02+rσ12+r(1−r)(μ1−μ0)2.\begin{aligned} \eta(r)&=(1-r)\eta_0+r\eta_1,\\ \mu_m(r)&=(1-r)\mu_0+r\mu_1,\\ \sigma_m^2(r)&=(1-r)\sigma_0^2+r\sigma_1^2 +r(1-r)(\mu_1-\mu_0)^2. \end{aligned}

The last term follows by subtracting the square of the interpolated mean from the interpolated second raw moment. It keeps variance positive. This normal has the same first two moments as (1−r)P0+rP1(1-r)P_0+rP_1, but is generally not that literal mixture: a mixture of two distinct normals need not be normal. Thus m-flatness of this family does not assert closure under density mixtures. Differentiating the two displayed paths verifies (61) at α=1\alpha=1 and α=−1\alpha=-1 respectively.

For comparison, the Levi-Civita segment is the vertical path or half-ellipse in (50), with endpoints chosen as above. It minimizes Fisher length; e- and m-geodesics are straight for their respective connections and have no general Fisher-length-minimization property.

For P0=N(0,1)P_0=N(0,1) and P1=N(2,1)P_1=N(2,1) the difference is explicit:

ConnectionPath in (μ,σ)(\mu,\sigma)Scale where μ=1\mu=1
e, α=1\alpha=1μ=2r\mu=2r, σ=1\sigma=11
Levi-Civita, α=0\alpha=0(μ−1)2/2+σ2=3/2(\mu-1)^2/2+\sigma^2=3/2 on the joining arc3/2\sqrt{3/2}
m, α=−1\alpha=-1μ=2r\mu=2r, σ2=1+4r(1−r)\sigma^2=1+4r(1-r)2\sqrt2

The Levi-Civita parameter in this comparison is the affine hyperbolic parameter, not generally μ/2\mu/2. Its length is 2 arcosh⁡(2)\sqrt2\,\operatorname{arcosh}(2); the horizontal e-path has length 2. If the endpoints have equal means, all three paths trace the same vertical segment, but their affine parameterizations generally differ: e interpolates precision, m interpolates variance, and Levi-Civita interpolates log scale.

Continue with exponential families for sufficient statistics and the log-partition function, or dual coordinates and KL projections for the Bregman and Pythagorean consequences. The Information Geometry hub lists the complete entry path.

References
  1. Ay, N., Jost, J., Lê, H. V., & Schwachhöfer, L. (2017). Information Geometry (Vol. 64). Springer. 10.1007/978-3-319-56478-4
  2. Amari, S., & Nagaoka, H. (2000). Methods of Information Geometry (Vol. 191). American Mathematical Society. 10.1090/mmono/191
  3. Amari, S. (2016). Information Geometry and Its Applications (Vol. 194). Springer Japan. 10.1007/978-4-431-55978-8