Linear Algebra - Fanyu Zhao

P.S. 3B1B in YouTube gives a complete new and clear understanding about the matrix transformation. Higly Highly Recommended.

For better understanding, we consider a vector as a combination between the direction and the distance.

Key Points

1. Properties

1.1 Dot Product

Dimension of One.

and, $\overrightarrow{v}\cdot \overrightarrow{u}=\overrightarrow{u}\cdot \overrightarrow{v}$$\vec{v}\cdot \vec{u}=\vec{u}\cdot \vec{v}$. The sequence is irrelevant.

1.2 Length

A vector has two important characteristics, one is the direction and the other is the length.

, where we define $||\overrightarrow{v}||$$||\vec{v}||$ As the lengh, and $\hat{\overrightarrow{v}}$$\hat{\vec{v}}$ as a unit vector, for which the length is one and is used to show the direction.

The length of $\overrightarrow{v}$$\vec{v}$ is coined as the norm,

The distance between two vectors

1.3 Two Vectors

We can solve the angle between two vectors, $\theta$$\theta$.

2. Matrix

2.1 Matrix Multiplication

, then an element of matrix $C$$C$ Is,

2.2 Transpose

2.3 Solve Linear Equations & Determinant

, where the matrix $A$$A$, and the vector $\overrightarrow{p}$$\vec{p}$ are known, we aim to calculate the unknow vector $\overrightarrow{x}$$\vec{x}$.

Three unknowns and three equations, we can solve the vector $\overrightarrow{x}$$\vec{x}$ if we say $A$$A$ is invertible. That is equivalent to say that $A$$A$ has a determinant of none zero.

The determinant of $A$$A$ Is,

Key implication of matrix:

1. Matrix, as shown as a set of combined equations, could represent the direction of linear transformation, $A$$A$, for a vector, $\overrightarrow{x}$$\vec{x}$. Solving $\overrightarrow{x}$$\vec{x}$ is like finding a unknown vector that, after applying the linear transformation, $A$$A$, we can get our target vector, $\overrightarrow{p}$$\vec{p}$.
2. The row of $A$$A$ is the number of equations, $M$$M$, and the dimension of $\overrightarrow{x}$$\vec{x}$ is the number of unknowns, $N$$N$, we are planing to solve with. If $M=N$$M=N$, then we have a solution for sure, and matrix $A$$A$ is squared. If $M>N$$M>N$, then there are more equations available to solve those unknows, we would also have a solution. The last case is $M<N$$M<N$, and in that case, there are more free variables.
3. The determinant is pretty much like collasping a matrix, $A$$A$, into dimension one, a number.
4. The determinant <=> collapse the dimension

To solve it,

• Step 1: append the target vector into the RHS of matrix A.
• Step 2: make matrix A an identity matrix, then the RHS would be our answer.

2.4 Matrix Inverse

Clearly, inverse of matrix requires the denominator to be non-zero, which is equivalent to having a non-zero determinant.

For a $m\times n$$m\times n$ matrix $A$$A$, the inverse is defined as,

, where $adjA=(-1{)}^{i+j}{M}_{ij}^T$$adj A= (-1)^{i+j}\ M_{ij}^T$, and $M_{ij}$$M_{ij}$ is the squared sub-matrix.

Why we calculate the inverse?

Because the computer is good at matrix calculation. By using the invese, we could quickly get the solution.

2.5 Orthogonal Matrix

The orthogonal matrix is defined as,

2.6 Eigenvalues and Eigenvectors

Recall that a matrix could be considered as a linear transformation. Exactly, it transfer a N-dimension space.

For example,

A matrix could be considered as a panel, with two unit axis. In a two-dimension space, the firtst column sates the x-axis, and the second states the y-axis. The unit is one.

Similarly,

There is another matrix as shown above, and could also represent a space, with the first column $\left(\begin{array}{c}2\\ 3\end{array}\right)$$\begin{pmatrix} 2\\3 \end{pmatrix}$ and the second $\left(\begin{array}{c}1\\ 4\end{array}\right)$$\begin{pmatrix} 1\\4 \end{pmatrix}$ consisting another space. The unit x-axis has a angle and length of $\sqrt{13}$$\sqrt{13}$, and the y-axis has also a different angle and length is 3.

Now, if we have a vector $\overrightarrow{v}$$\vec{v}$, and we want to fit it into that space, what we do is $M\cdot \overrightarrow{v}$$M \cdot \vec{v}$. Graphically, we may see it as a vector undergoes a linear transformation and have a different lengh and direction.

2.6.1 Eigenvalue

The $\lambda$$\lambda$ here is called the eigenvalue. ($A$$A$ needs to be squared)

To solve it,

, coz $\overrightarrow{v}$$\vec{v}$ is non-zero, $A-\lambda I$$A-\lambda I$ must be zero to let the equation hold. Therefore we would let

The above equation is also called, characterisitic polynomial.

For example, if $\begin{array}{c}A=\left(\begin{array}{cc}3& 1\\ 1& 3\end{array}\right)\end{array}$$A=\begin{pmatrix}3&1\\1&3\end{pmatrix}$

We solve it could get the solution, ${\lambda }_1=2,{\lambda }_2=4$$\lambda_1=2,\quad\lambda_2=4$.

Some Propertities of Eigenvalue

1. $\sum {\lambda }_i$$\sum \lambda_i$ is the sum of diagnoal elements.
2. $\prod {\lambda }_i=det(M)$$\prod \lambda_i=det(M)$

2.6.2 Eigenvector

The eigenvector is just the solution of our equation, $\overrightarrow{v}$$\vec{v}$, once we plug $\lambda$$\lambda$ inside.

For example, if $\begin{array}{c}A=\left(\begin{array}{cc}3& 1\\ 1& 3\end{array}\right)\end{array}$$A=\begin{pmatrix}3&1\\1&3\end{pmatrix}$

2.7 Diagonalisation and Powers of $A$$A$

Let matrix $Q$$Q$ be a matrix whose columns are the eigenvectors of $A$$A$.

Let $\mathrm{\Lambda }$$\Lambda$ ( \Lambda ) be a matrix whose diagnals are each eigenvalues.

We define that

The implication is that matrix $A$$A$ could be decomposed by its eigenvalue, $\mathrm{\Lambda }$$\Lambda$, and its eigenvector, $Q$$Q$.

Therefore,

The it could be represented by the recursive substitution, being useful under certain circumstances.

For example, (by simply applying the previous matrix)

By Fanyu Zhao