Here's an example of finding a suitable coordinate system for the equation

3x^2 -10xy +3y^2 + 14x -2y + 3 = 0.

For this equation, M = \pmatrix{3&-5\\-5&3} d = {14 \choose -2}.

The first step is to find the eigenvalues and eigenvetors of matrix M  which means solving the following equation

|M - \lambda I| = \left|\matrix{3 - \lambda & -5\\-5 & 3 - \lambda}\right| = 0.

Evaluating the determinant gives (3 - \lambda)^2 - (-5)^2 = \lambda^2 - 6\lambda - 16 = 0.  This is a quadratic equation with two real roots \lambda_1 = 8, \lambda_2 = -2.  Since substituting either into the matrix M - \lambda I  renders its determinant zero, the rows of the matrix are then linearly dependent giving us a freedom of choosing an eigenvector. Thus when solving (M - \lambda I){{x}\choose{y}}=0 we need only consider one row. For example, for \lambda = 8,  the first row becomes -5x - 5y = 0  giving a solution {1}\choose{-1} which we normalize to {{{1}/\sqrt{2}}\choose{{-1}/\sqrt{2}}}.  For the second eigenvalue \lambda = -2,   the first row is 5x - 5y = 0   resulting in a solution {1 \choose 1}.  This is normalized to {1/ \sqrt 2 \choose 1/ \sqrt 2}.  Use the two eigenvector to compose the transform matrix

P = \pmatrix{1/ \sqrt 2&-1/ \sqrt 2\\1/ \sqrt 2&1/ \sqrt 2}.

As the consequence of the normalization, |P| = 1  meaning that the matrix is orthogonal and the induced transformtion is a rotation around the origin. Naturally,

P^T M P = \pmatrix{8&0\\0&-2},

as expected. Thus the substitution {x \choose y} = P{x' \choose y'} leads to the equation

8x'^2 - 2y'^2 + 8\sqrt{2}x' + 6\sqrt{2}y' + 3 = 0.

Completing the equare (or rather squares, in this case) we obtain

8(x' + \frac{1}{\sqrt{2}})^2 - 2(y' - \frac{3}{\sqrt{2}})^2 + 8 = 0

which, more conventionally, is written as

\frac{(y' - \frac{3}{\sqrt{2}})^2}{4} - \frac{(x' + \frac{1}{\sqrt{2}})^2}{1}= 1.

This is an equation of a hyperbola. In the new coordinate system, its center is at {x \choose y} = {-1/\sqrt{2} \choose 3/\sqrt{2}}.   In the original coordinate system, the hyperbola is centered at

{x \choose y} = P{-1/\sqrt{2} \choose 3/\sqrt{2}} = {1 \choose 2}.

References

  • D. A. Brannan et al, Geometry Cambridge University Press, 2002