A great ellipse is an ellipse passing through two points on a spheroid and having the same center as that of the spheroid. Equivalently, it is an ellipse on the surface of a spheroid and centered on the origin, or the curve formed by intersecting the spheroid by a plane through its center.^[1] For points that are separated by less than about a quarter of the circumference of the earth, about ${\displaystyle 10\,000\,\mathrm {km} }$, the length of the great ellipse connecting the points is close (within one part in 500,000) to the geodesic distance.^[2][3][4] The great ellipse therefore is sometimes proposed as a suitable route for marine navigation. The great ellipse is special case of an earth section path.

Assume that the spheroid, an ellipsoid of revolution, has an equatorial radius ${\displaystyle a}$ and polar semi-axis ${\displaystyle b}$. Define the flattening ${\displaystyle f=(a-b)/a}$, the eccentricity ${\displaystyle e={\sqrt {f(2-f)}}}$, and the second eccentricity ${\displaystyle e'=e/(1-f)}$. Consider two points: ${\displaystyle A}$ at (geographic) latitude ${\displaystyle \phi _{1}}$ and longitude ${\displaystyle \lambda _{1}}$ and ${\displaystyle B}$ at latitude ${\displaystyle \phi _{2}}$ and longitude ${\displaystyle \lambda _{2}}$. The connecting great ellipse (from ${\displaystyle A}$ to ${\displaystyle B}$) has length ${\displaystyle s_{12}}$ and has azimuths ${\displaystyle \alpha _{1}}$ and ${\displaystyle \alpha _{2}}$ at the two endpoints.

There are various ways to map an ellipsoid into a sphere of radius ${\displaystyle a}$ in such a way as to map the great ellipse into a great circle, allowing the methods of great-circle navigation to be used:

• The ellipsoid can be stretched in a direction parallel to the axis of rotation; this maps a point of latitude ${\displaystyle \phi }$ on the ellipsoid to a point on the sphere with latitude ${\displaystyle \beta }$, the parametric latitude.
• A point on the ellipsoid can mapped radially onto the sphere along the line connecting it with the center of the ellipsoid; this maps a point of latitude ${\displaystyle \phi }$ on the ellipsoid to a point on the sphere with latitude ${\displaystyle \theta }$, the geocentric latitude.
• The ellipsoid can be stretched into a prolate ellipsoid with polar semi-axis ${\displaystyle a^{2}/b}$ and then mapped radially onto the sphere; this preserves the latitude—the latitude on the sphere is ${\displaystyle \phi }$, the geographic latitude.

The last method gives an easy way to generate a succession of way-points on the great ellipse connecting two known points ${\displaystyle A}$ and ${\displaystyle B}$. Solve for the great circle between ${\displaystyle (\phi _{1},\lambda _{1})}$ and ${\displaystyle (\phi _{2},\lambda _{2})}$ and find the way-points on the great circle. These map into way-points on the corresponding great ellipse.

If distances and headings are needed, it is simplest to use the first of the mappings.^[5] In detail, the mapping is as follows (this description is taken from ^[6]):

• The geographic latitude ${\displaystyle \phi }$ on the ellipsoid maps to the parametric latitude ${\displaystyle \beta }$ on the sphere, where

  ${\displaystyle a\tan \beta =b\tan \phi .}$

• The longitude ${\displaystyle \lambda }$ is unchanged.
• The azimuth ${\displaystyle \alpha }$ on the ellipsoid maps to an azimuth ${\displaystyle \gamma }$ on the sphere where

  ${\displaystyle {\begin{aligned}\tan \alpha &={\frac {\tan \gamma }{\sqrt {1-e^{2}\cos ^{2}\beta }}},\\\tan \gamma &={\frac {\tan \alpha }{\sqrt {1+e'^{2}\cos ^{2}\phi }}},\end{aligned}}}$

  and the quadrants of ${\displaystyle \alpha }$ and ${\displaystyle \gamma }$ are the same.
• Positions on the great circle of radius ${\displaystyle a}$ are parametrized by arc length ${\displaystyle \sigma }$ measured from the northward crossing of the equator. The great ellipse has a semi-axes ${\displaystyle a}$ and ${\displaystyle a{\sqrt {1-e^{2}\cos ^{2}\gamma _{0}}}}$, where ${\displaystyle \gamma _{0}}$ is the great-circle azimuth at the northward equator crossing, and ${\displaystyle \sigma }$ is the parametric angle on the ellipse.

(A similar mapping to an auxiliary sphere is carried out in the solution of geodesics on an ellipsoid. The differences are that the azimuth ${\displaystyle \alpha }$ is conserved in the mapping, while the longitude ${\displaystyle \lambda }$ maps to a "spherical" longitude ${\displaystyle \omega }$. The equivalent ellipse used for distance calculations has semi-axes ${\displaystyle b{\sqrt {1+e'^{2}\cos ^{2}\alpha _{0}}}}$ and ${\displaystyle b}$.)

The "inverse problem" is the determination of ${\displaystyle s_{12}}$, ${\displaystyle \alpha _{1}}$, and ${\displaystyle \alpha _{2}}$, given the positions of ${\displaystyle A}$ and ${\displaystyle B}$. This is solved by computing ${\displaystyle \beta _{1}}$ and ${\displaystyle \beta _{2}}$ and solving for the great-circle between ${\displaystyle (\beta _{1},\lambda _{1})}$ and ${\displaystyle (\beta _{2},\lambda _{2})}$.

The spherical azimuths are relabeled as ${\displaystyle \gamma }$ (from ${\displaystyle \alpha }$). Thus ${\displaystyle \gamma _{0}}$, ${\displaystyle \gamma _{1}}$, and ${\displaystyle \gamma _{2}}$ and the spherical azimuths at the equator and at ${\displaystyle A}$ and ${\displaystyle B}$. The azimuths of the endpoints of great ellipse, ${\displaystyle \alpha _{1}}$ and ${\displaystyle \alpha _{2}}$, are computed from ${\displaystyle \gamma _{1}}$ and ${\displaystyle \gamma _{2}}$.

The semi-axes of the great ellipse can be found using the value of ${\displaystyle \gamma _{0}}$.

Also determined as part of the solution of the great circle problem are the arc lengths, ${\displaystyle \sigma _{01}}$ and ${\displaystyle \sigma _{02}}$, measured from the equator crossing to ${\displaystyle A}$ and ${\displaystyle B}$. The distance ${\displaystyle s_{12}}$ is found by computing the length of a portion of perimeter of the ellipse using the formula giving the meridian arc in terms the parametric latitude. In applying this formula, use the semi-axes for the great ellipse (instead of for the meridian) and substitute ${\displaystyle \sigma _{01}}$ and ${\displaystyle \sigma _{02}}$ for ${\displaystyle \beta }$.

The solution of the "direct problem", determining the position of ${\displaystyle B}$ given ${\displaystyle A}$, ${\displaystyle \alpha _{1}}$, and ${\displaystyle s_{12}}$, can be similarly be found (this requires, in addition, the inverse meridian distance formula). This also enables way-points (e.g., a series of equally spaced intermediate points) to be found in the solution of the inverse problem.

• Earth section paths
• Great-circle navigation
• Geodesics on an ellipsoid
• Meridian arc
• Rhumb line