A normal law has several coordinate descriptions: mean and standard deviation, mean and variance, natural parameters, or expected sufficient statistics. A change of coordinates changes the numbers used to describe a perturbation of the law. It should preserve the perturbation itself and its statistical size.
This note develops the geometry needed to make that distinction precise. We construct tangent vectors, use Fisher information to measure them, and define connections that differentiate and transport them. The family provides explicit calculations throughout. It also exhibits a distinction that matters later: its Fisher metric has negative Riemannian curvature, while its exponential and mixture connections are flat.
Notation and conventions¶
We follow the shared notation, with used locally in this note for a general chart. For normals, ; is reserved for natural parameters, and for expectation parameters. This local use of is unrelated to the extreme-value index in the Incerto track.
Coordinates and tangent components carry upper indices; covector components and metric components carry lower indices. An index repeated once above and once below is summed from 1 to the chart dimension. Explicit sums over sample outcomes are written out. Superscripts naming components, such as , are not powers.
is the coordinate tangent basis, and is its dual covector basis. In particular, denotes the dual basis in natural coordinates. The inverse metric has components , with .
is the differential of a smooth map. denotes its Jacobian matrix in specified charts, also when is a change of coordinates. Matrices act on column components; the transpose is .
is the metric matrix in the chart. Thus . Once the metric is Fisher information, . denotes the corresponding quadratic form.
means ; lowers its output index. A superscript identifies a connection, not a tensor index.
, , and is the column of score components. For fixed , these are covector components: the coordinate gradient symbol in does not mean a Riemannian gradient. means expectation under ; in natural coordinates we write , , and .
We use matrices for explicit Jacobians and the Legendre identities and . Here the gradients and Hessians are ordinary derivatives in the named affine coordinates. Their entries will be related to tensor components explicitly. Curve parameters are denoted by , leaving for sufficient statistics. denote vector fields, while denotes an observation.
A running example: normal location and scale¶
Let be the set of laws with and . Relative to Lebesgue measure on ,
The sample space is one-dimensional, but the family of laws is two-dimensional. The mean and variance determine each law uniquely. We exclude , whose point mass is outside this density model.
Four charts on the same family¶
The following are global charts on . The symbol is local notation for variance.
| Chart | Coordinates | Coordinate domain |
|---|---|---|
| Location and scale | ||
| Location and variance | , | |
| Natural | ||
| Expectation |
With , completing the square gives
Thus the second expectation coordinate is the second raw moment, not the variance. The exponential-family note derives these coordinates in the broader setting of sufficient statistics; its statistic is written here, following the shared canon.
The inverse coordinate maps include
Write , , and for the maps from the chart to these three charts. Direct differentiation gives
Their determinants are nonzero for . The inverse Jacobian is ; specifying its evaluation point prevents confusing the two directions of a coordinate change.
What makes this a manifold¶
A smooth -manifold is a Hausdorff, second-countable topological space covered by coordinate neighborhoods, each homeomorphic to an open subset of , with smooth transition maps between these charts. A chart assigns coordinates to points; its inverse interprets coordinate values as points of the manifold. The normal family obtains such a structure from its bijection with , and the other charts above are smoothly compatible with it.
A general parameterized statistical model needs more care. Parameters may repeat the same law, and even an injective parameterization can have a vanishing first derivative at a point. We work on regular regions where the model is locally identifiable and the Fisher matrix is smooth and positive definite. The normal calculation below verifies that condition. A statistical manifold need not be presented as a curved surface inside a Euclidean space.
Tangent vectors, functions, and differentials¶
Curves define tangent vectors¶
Fix and a chart containing it. Two smooth curves through at are equivalent when
The chain rule makes this equivalence independent of the chart. A tangent vector is an equivalence class of such curves. Its components are . Every column of real numbers is realized by a coordinate line for sufficiently small . These components give the set of classes a -dimensional vector-space structure, called ; coordinate changes act linearly on that space.
For normals, is represented by near . The coordinates of a law and the components of a tangent vector have different roles: one specifies a distribution; the other specifies rates of change at that distribution.
The equivalent definition by derivations¶
A derivation at is an -linear map satisfying the pointwise Leibniz rule
It depends only on functions near . Indeed, if vanishes near , choose a smooth cutoff equal to one at and supported where ; applying the product rule to their zero product gives . Thus we can also work with germs of smooth functions at , and apply to local coordinate functions.
A curve class defines such a derivation by
We now show that every derivation arises this way. On a small coordinate ball, the integral form of first-order Taylor expansion writes a smooth function as
A derivation annihilates constants, since . Applying its product rule to this expansion gives . Hence with , represented by the coordinate line above. The two constructions are inverse, and is a basis of .
For example, the smooth function satisfies
Here holds fixed, and holds fixed. Holding a different second coordinate fixed can produce a different basis vector even if the first coordinate has the same name.
Coordinate changes and pushforwards¶
Let be a transition map and set and , so . The chain rule gives
The basis and components change together, leaving the derivation fixed. For the variance chart, (4) gives
At , a unit scale velocity has variance velocity 4. In expectation coordinates the same vector has components .
More generally, a smooth map has a differential, or pushforward, defined intrinsically by
In source coordinates and target coordinates , its matrix is the Jacobian of , denoted . Its entries are , and . Unlike a coordinate change, a smooth map need not be invertible or preserve dimension. The variance map , , has : it discards the location direction.
Covectors and vector fields¶
A cotangent vector is a linear functional . The differentials form the dual basis because . Thus
Under the same transition map, and . This inverse transformation is what makes independent of coordinates.
A smooth vector field assigns at every point, with smooth components in every chart. It differentiates functions by . On the normal family, describes relative scale change. Its integral curves satisfy , , so . In the variance chart this same field is .
Metrics and Fisher information¶
A metric is more than a parameter norm¶
A Riemannian metric assigns a smooth positive definite symmetric bilinear form to every tangent space. Its components are . Substituting and gives
Every Riemannian metric has this covariance. It does not by itself select Fisher information. Even a Euclidean metric acquires nonconstant coefficients in nonlinear coordinates. On an angular sector of the Euclidean plane, write , , with radius . Then
Thus nonconstant metric coefficients alone do not establish curvature.
For normals, choosing in the scale chart and choosing afresh in the variance chart gives incompatible metrics:
At , the same unit scale velocity has squared lengths 1 and 16. A Euclidean metric chosen in one chart is legitimate, but must then be transformed by (14). A statistical principle is needed to choose a metric without privileging that chart.
For a piecewise smooth curve , the metric determines length and intrinsic distance by
The infimum is over curves in the specified manifold. Restricting to a subfamily can increase this distance.
Scores measure changes in the law¶
For a general model, assume positive densities on a common support relative to a fixed measure . Assume smooth parameter dependence, square-integrable scores, and local integrable bounds allowing differentiation of the normalizer and the expectations used below. We will specify the additional Taylor and third-moment assumptions where they are needed.
Differentiating the density and its normalization gives
The Fisher metric per observation is
It is positive semidefinite; our regularity assumption requires positive definiteness. A zero length direction would have zero first-order density change almost everywhere. Under reparameterization the score obeys , so its covariance obeys exactly the metric transformation law. Invertible, parameter-independent changes of the observed variable also preserve Fisher information: their density Jacobian contributes a term to the log density whose parameter derivative is zero.
Computing the normal metric¶
Set . Differentiating (1),
The subscripts here denote covector components, not differentiation of an already defined score. Odd standard-normal moments vanish, and integration by parts gives . In particular, the second and fourth moments are 1 and 3. Consequently,
This is positive definite for every . On compact parameter neighborhoods with scale bounded away from zero, Gaussian tail bounds justify the differentiations and moments used here and later. A fixed location velocity has smaller Fisher length at a larger scale, because its change in log density is smaller in mean square.
Using in the variance chart gives
Substitution of recovers the same squared length.
Natural and expectation coordinates¶
In natural coordinates , so differentiating the normalizer in (2) gives the matrix identities
The expression for is the Legendre transform, attained at the natural parameter corresponding to . Indeed, , whence ; the metric transformation law gives . Here and . The inverse of one coordinate matrix supplies the components in the other chart; it does not change the covariant type of the metric. The Jacobians in (4) pull both matrices back to . An ordinary Hessian need not transform as a tensor under arbitrary nonlinear coordinates; these Hessian formulas use the particular affine coordinates and .
Why Fisher information is distinguished¶
Coordinate covariance is required of every metric. Fisher information has additional properties concerning statistical distinguishability and the loss of observations. We prove the local KL and monotonicity statements below; the uniqueness theorem is cited with its finite-space hypotheses.
The Hessian of KL at the diagonal¶
Use the direction , holding the first law fixed. Differentiating normalization twice yields
Assume in addition that has a third-order Taylor remainder bounded by a constant times locally. For example, locally bounded integrable third derivatives of the log density under suffice. Taylor expansion then gives
The zero expected score removes the linear term. The Hessian in the second argument at equality is therefore ; because the first derivative vanishes there, this Hessian transforms tensorially. With only a second-order Taylor expansion, the remainder is , as in the Fisher-versus- note. Finite KL remains asymmetric and is not half the squared Fisher–Rao distance.
For normals the exact expression is
Putting at a fixed gives , in agreement with (21).
Coarse-graining, monotonicity, and sufficiency¶
Let an observation pass through a parameter-independent Markov kernel , giving . A deterministic statistic is a special case. Assume the transformed model admits densities and differentiation under the kernel integral, and that the original scores are square integrable. Denote the two score covectors by and , and their component columns by and . Differentiating the marginal density of gives
Thus the new directional score is the conditional expectation of the old one. The conditional-variance identity proves information monotonicity:
The superscripts identify experiments; they are not tensor indices. The output information can be singular. Equality for one vector at one parameter means exactly that its directional score is determined by almost surely. Equality of the matrices at that parameter means this for every score component. This is a statement about first-order information at that point.
A statistic is sufficient for the family if the conditional distribution of given the statistic can be chosen independently of the parameter. Equivalently in the regular dominated setting, the density factors into a parameter-dependent function of the statistic and a parameter-independent factor. Its scores are functions of the statistic, so sufficiency implies equality of Fisher information throughout the family.
For a precise converse, take a finite sample space with strictly positive smooth probabilities on a connected open parameter domain, and a fixed statistic . Equality of Fisher information at every parameter is then equivalent to sufficiency. To prove the converse, equality in (28) gives, for every outcome,
Connectedness makes each conditional probability constant in the parameter, which proves sufficiency. Equality at one parameter, or in one direction, is insufficient for this conclusion. Extensions beyond this finite positive setting need corresponding regularity assumptions; see Ay, Jost, Lê, and Schwachhöfer on sufficient statistics.
The normal family illustrates both preservation and loss. For independent observations , the statistic is sufficient by the factorization in (2). Its information equals that of the full sample, namely : independent zero-mean scores add and their cross-covariances vanish. By contrast, retaining only gives a Bernoulli probability , where is the standard-normal cdf. At , and , so
The sign loses all local scale information and retains only a fraction of the location information there.
Čencov’s finite-space uniqueness theorem¶
The theorem concerns a compatible choice of metrics on all finite probability simplices, not an arbitrary metric on one fixed normal family. Let
A Markov map is a stochastic linear map . A congruent Markov embedding has a Markov left inverse with and maps the interior into the interior. It splits each input outcome into a disjoint block of output outcomes with fixed conditional probabilities; the block label recovers the input. Such a splitting preserves the statistical experiment.
Čencov’s theorem states that if smooth Riemannian metrics on , for every , make every congruent Markov embedding an isometry, then there is a single constant , independent of and , such that
Conversely, these metrics have that invariance. We cite this classification, not prove it: see Čencov’s Statistical Decision Rules and Optimal Inference (1982) and the explicit finite-simplex formulation in Theorem 2 of Montúfar, Rauh, and Ay (2014).
Pulling (32) back along a regular finite statistical model, with , gives . We choose for information per observation. General Markov maps are contractions by (28), whereas sufficient transformations preserve information. Requiring invariance under every Markov map would be wrong: a kernel can discard the observation completely. Equivalently, demanding monotonicity for a family of metrics under all such maps forces the isometries above, since and its Markov left inverse give opposite inequalities.
For general sample spaces, Ay et al. (2017) develop invariance and uniqueness results for suitable integrable models and continuous local tensor fields. A different extension by Bauer–Bruveris–Michor (2016) characterizes the Fisher metric among smooth weak Riemannian metrics invariant under observation-space diffeomorphisms on smooth positive probability densities over a closed manifold of dimension greater than one. Neither statement is an unrestricted uniqueness theorem for every continuous or singular statistical model; in particular, that latter theorem does not directly cover densities on .
Density and square-root densities¶
The raw density metric, when finite, , also transforms correctly under parameter changes. It depends on the reference measure and the measurement coordinate: for Lebesgue densities and , , . It therefore differs from Fisher geometry even though both constructions are covariant under reparameterization.
Assuming differentiability into , the map has derivative . Its pullback of the inner product is exactly . With , this gives . The Fisher-versus- note explains the distinct roles of parameter norms, scores in , and densities in . A square-root embedding preserves tangent lengths; the distance between two normals still minimizes paths within the normal family, rather than over the whole sphere of square-root densities.
Connections differentiate vector fields¶
The ordinary derivative of vector components depends on the chosen chart. For example, a field with constant scale component has variance component times as large, which changes along a scale curve. A connection supplies the correction that makes differentiation independent of coordinates. It is extra geometric structure; a metric singles out one connection only when we impose the conditions below.
Axioms and Christoffel symbols¶
An affine connection on is an -bilinear operation on smooth vector fields satisfying, for every smooth function ,
Thus the lower argument is linear over functions; in the differentiated argument , the Leibniz term is essential. At a point, depends only on the value of and on the first-order behavior of nearby.
Define the Christoffel symbols in a coordinate basis by
Expanding and with the axioms gives
The second term accounts for the change of the coordinate basis under the connection. Prescribing smooth defines a connection on a chart; prescriptions on overlapping charts must satisfy the next law.
Why Christoffel symbols are not a tensor¶
Use , , and as before. Apply the connection axioms to to obtain
A -tensor has only the first term. The inhomogeneous second derivative is why coefficients that vanish in one chart need not vanish in another. For example, take the connection with in the normal location-scale chart. In the variance chart, , so it has
This is a coordinate-flat connection chosen on the parameter domain; it is not yet the Fisher Levi-Civita connection. The example isolates the effect of the second term in (36).
The difference of two connections is a -tensor: their inhomogeneous terms cancel. Equivalently, the Leibniz terms cancel in , leaving an expression linear over functions in both and . We will use this fact to compare statistical connections.
Along a curve: parallel transport and geodesics¶
For a curve , let be a vector field along the curve. It need not extend to a single vector field on all of . Its covariant derivative is
The transformation law makes this independent of coordinates. A field is parallel when . Given an initial vector, this linear ODE has a unique solution along a smooth curve on a compact parameter interval. It defines a linear isomorphism between the endpoint tangent spaces, called parallel transport. Reversing the curve gives its inverse. Transport generally depends on the path, not just its endpoints.
An affinely parameterized geodesic has parallel velocity:
An arbitrary nonlinear reparameterization need not preserve this equation; affine changes of the parameter do. A connection defines geodesics without requiring a metric or a length-minimization interpretation.
Torsion, compatibility, and the Levi-Civita connection¶
The Lie bracket is the vector field . In coordinates, . The torsion of a connection is
It is a tensor. A connection is torsion-free exactly when its lower Christoffel indices are symmetric in a coordinate basis.
A connection is metric-compatible, written , when
Equivalently, . Along a curve this product rule shows that parallel transport preserves inner products. In particular, a geodesic for a metric-compatible connection has constant speed.
The Levi-Civita connection is the unique connection that is both torsion-free and compatible with . The identity that determines it is the Koszul formula:
To see why this determines the connection, write the compatibility identity for the three cyclic choices of , add the first two and subtract the third, and replace by . This gives (42). Nondegeneracy of then uniquely determines . For existence, the right side is linear over functions in , so it defines a vector field by nondegeneracy of . The product rule gives the connection axioms; subtracting the expression with exchanged gives torsion-freeness, and adding the expressions with exchanged gives metric compatibility. In a coordinate basis the brackets vanish, yielding the computable formula
We write for Levi-Civita because it will be the statistical connection. We use the standard Riemannian result that its geodesics minimize length on sufficiently short segments. It does not say every geodesic segment is globally minimizing on every manifold. For background on this result and the connection framework, see §2 of Nielsen’s 2020 introduction.
Levi-Civita geometry of the normal family¶
Christoffel symbols and parallel transport¶
Substituting (21) into (43) gives exactly these nonzero coefficients:
For example, . This coefficient already shows why a horizontal line with constant scale and changing mean is not a Levi-Civita geodesic in the full family.
Along any fixed-mean curve , the parallel equations reduce to
Thus a vector transported from scale to scale has components multiplied by . Its Fisher squared length stays fixed. Constant coordinate components would not give parallel transport along this curve.
Hyperbolic curvature and geodesics¶
Make the local coordinate substitution . Then
This is twice the Poincaré upper-half-plane metric. Its curvature is , not -1: multiplying a metric by 2 multiplies lengths by and divides sectional curvature by 2.
We can verify the curvature directly from the connection. With the convention
(44) gives . Hence the Gaussian curvature is
The geodesic equations are
The first equation integrates to . When this constant is zero, the solutions are vertical lines, , , with . Otherwise, in the rescaled coordinate , the solutions trace semicircles orthogonal to the boundary . An affine parameterization in the original chart is
Substitution proves that these curves solve both equations and have constant squared speed . They exhaust the nonvertical initial conditions: at any point with , choose , then choose to match the point and velocity. Local uniqueness for the geodesic ODE identifies the solution. The images are half-ellipses in and semicircles in ; their parameterizations are not linear in the angle.
For endpoints and with different means, the center and radius are
Set and in (50) to join them over . For equal means the corresponding segment is . In this complete hyperbolic plane, these segments are the unique minimizing paths up to reparameterization.
Fisher–Rao distance between two normals¶
The upper-half-plane distance satisfies . Rescaling by (46) gives
The hyperbolic minimizing-path and distance results used here are treated in Costa–Santos–Strapasson, Fisher information distance: a geometrical reading (2015), §2. Integrating the constant speed in (50) gives the same expression. In particular,
In the fixed-scale one-dimensional subfamily, the distance is instead . The full-family minimizing path can increase scale while changing location, then decrease it again. For distinct means it is strictly shorter than the horizontal fixed-scale path.
Statistical connections and duality¶
A regular statistical model has more structure than its Fisher metric: third moments of scores distinguish a family of connections. We give their definitions and normal-family calculations here. The duality note develops the resulting Bregman divergence and KL Pythagoras identities.
The cubic tensor and the alpha family¶
Assume sufficient smoothness and integrability to differentiate the Fisher metric, and finite absolute third score moments. Define the Amari–Chentsov cubic tensor by
It is a symmetric covariant tensor because each score transforms as a covector. Raising the last index defines .
For , the statistical -connection has coefficients with the output index lowered
The comma separates the lowered output index; these coefficients themselves are not a covariant tensor. Under a change of chart, the second derivative of contributes the inhomogeneous term in (36). The coefficients are symmetric in , so all these connections are torsion-free. We use the convention and .
We can identify their relation to Levi-Civita directly. Differentiation of an expectation obeys . Applying it to gives
Insert this into the lowered version of the Levi-Civita formula. The mixed terms cancel, leaving
This proves that the connection is Levi-Civita and expresses the difference of connections as a tensor.
Dual connections¶
Two connections and are dual with respect to when
Substitute (57) on the right, once with and once with . The cubic terms cancel by symmetry, and Levi-Civita compatibility supplies the left side. Thus and are dual, including the e/m pair. If is parallel for one connection and for its dual along the same curve, is constant. Neither transport separately needs to preserve . Indeed, , so nonzero generally breaks metric compatibility.
These definitions and the sign convention agree with Amari & Nagaoka (2000), Methods of Information Geometry and Amari (2016), Information Geometry and Its Applications. The calculations above establish the displayed relations under the stated regularity assumptions.
The normal alpha-connections¶
For the scores and , use . The only nonzero cubic components are
For example, and . Raising the output index in (57) gives the following coefficients; all others vanish:
Setting recovers (44). The general geodesic equations become
There is also an explicit transport comparison. Along a fixed-mean curve, -parallel transport from scale to gives
For the dual connection the exponents have the opposite terms. The paired inner product stays constant, although a vector’s own Fisher norm generally changes when .
Natural and expectation coordinates make different connections flat¶
In the natural chart, is independent of the observation. Since every score has expectation zero, (55) at gives . Also in these coordinates, by differentiating the covariance of .
The expectation chart makes the dual mixture coefficients vanish. Here is a coordinate argument. The natural coordinate vector fields are e-parallel. Let . Since ,
The explicit sum here multiplies matrices labeled by two different coordinate bases; the numeric identity compares their paired labels. Differentiate this constant pairing with any field and use e/m duality. Because , we obtain for every , hence . In particular, all m-Christoffel symbols vanish in the chart.
Both connections therefore have zero curvature. This dual flatness coexists with the curvature of . Curvature is attached to a specified connection; the vanishing of the e- or m-coefficients in their affine charts does not make the Fisher metric Euclidean.
Three geodesics between two normal laws¶
Let and with positive scales, and let . A straight natural-coordinate segment is an affinely parameterized e-geodesic. In location-scale coordinates it is
It interpolates precision and precision-weighted mean. Its density is proportional to , as follows by interpolating the natural form. The precision stays positive, so the whole segment belongs to the family.
A straight expectation-coordinate segment is an affinely parameterized m-geodesic of the connection on this normal family:
The last term follows by subtracting the square of the interpolated mean from the interpolated second raw moment. It keeps variance positive. This normal has the same first two moments as , but is generally not that literal mixture: a mixture of two distinct normals need not be normal. Thus m-flatness of this family does not assert closure under density mixtures. Differentiating the two displayed paths verifies (61) at and respectively.
For comparison, the Levi-Civita segment is the vertical path or half-ellipse in (50), with endpoints chosen as above. It minimizes Fisher length; e- and m-geodesics are straight for their respective connections and have no general Fisher-length-minimization property.
For and the difference is explicit:
| Connection | Path in | Scale where |
|---|---|---|
| e, | , | 1 |
| Levi-Civita, | on the joining arc | |
| m, | , |
The Levi-Civita parameter in this comparison is the affine hyperbolic parameter, not generally . Its length is ; the horizontal e-path has length 2. If the endpoints have equal means, all three paths trace the same vertical segment, but their affine parameterizations generally differ: e interpolates precision, m interpolates variance, and Levi-Civita interpolates log scale.
Continue with exponential families for sufficient statistics and the log-partition function, or dual coordinates and KL projections for the Bregman and Pythagorean consequences. The Information Geometry hub lists the complete entry path.
- Ay, N., Jost, J., Lê, H. V., & Schwachhöfer, L. (2017). Information Geometry (Vol. 64). Springer. 10.1007/978-3-319-56478-4
- Amari, S., & Nagaoka, H. (2000). Methods of Information Geometry (Vol. 191). American Mathematical Society. 10.1090/mmono/191
- Amari, S. (2016). Information Geometry and Its Applications (Vol. 194). Springer Japan. 10.1007/978-4-431-55978-8