Complete Matrices Guide: Formulas, Rank, Inverse & Eigenvalues

Unit 1 — Matrices (8 Hours)

Matrices provide powerful tools for solving systems of equations and analyzing engineering problems. This unit covers transformations, rank, inverse, eigenstructures, and complex matrices with every essential formula.

1. What Are Matrices?

A matrix is a rectangular array of numbers, symbols, or expressions arranged in rows and columns.

If a matrix $A$ has $m$ rows and $n$ columns, it is of order $m \times n$ :

A = [\begin{array}{cccc} a_{11} & a_{12} & \dots & a_{1 n} \\ a_{21} & a_{22} & \dots & a_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ a_{m 1} & a_{m 2} & \dots & a_{m n} \end{array}] =

Why Matrices Matter in Engineering

Compact representation of linear systems
Efficient computation of transformations (graphics, robotics, signal processing)
Foundation for machine learning, control systems, and structural analysis

2. Elementary Transformations

2.1 Row and Column Transformations

Elementary Row Operations ( $R_i$ denotes row $i$ ):

Operation	Notation
Interchange two rows	$R_i \leftrightarrow R_j$
Multiply a row by non-zero scalar $k$

Elementary Column Operations ( $C_i$ denotes column $i$ ):

Operation	Notation
Interchange two columns	$C_i \leftrightarrow C_j$
Multiply a column by non-zero scalar $k$

2.2 Standard Forms

Any matrix $A$ of rank $r$ can be reduced by elementary operations to:

Row Echelon Form (REF):

All non-zero rows are above any rows of all zeros
The leading coefficient of a non-zero row is always strictly to the right of the leading coefficient of the row above it

Reduced Row Echelon Form (RREF):

Satisfies REF conditions
The leading entry in each non-zero row is $1$
Each leading $1$ is the only non-zero entry in its column

Normal Form:

\begin{bmatrix} I_r & 0 \\ 0 & 0 \end{bmatrix}

where $I_r$ is the identity matrix of order $r$ .

2.3 Elementary Matrices

An elementary matrix $E$ is obtained by performing a single elementary row operation on $I$ .

If $E$ is obtained from $I$ by an elementary row operation, then applying that same operation to $A$ gives:

EA = \text{transformed matrix}

3. Inverse of a Matrix

3.1 Definition and Properties

A square matrix $A$ of order $n$ is invertible (non-singular) if there exists a matrix $B$ such that:

AB = BA = I_n

$B$ is denoted as $A^{-1}$ .

Properties:

$(A^{-1})^{-1} = A$
$(AB)^{-1} = B^{-1}A^{-1}$

3.2 Finding Inverse Using Elementary Transformations

For a square matrix $A$ , augment it with $I$ :

[A \mid I]

Apply elementary row operations to transform $A \to I$ . The right side simultaneously transforms to $A^{-1}$ :

[I \mid A^{-1}]

Condition: $A$ is invertible if and only if $\det(A) \neq 0$ .

3.3 Inverse Using Adjugate

A^{-1} = \frac{1}{\det(A)} \cdot \text{adj}(A)

where $\text{adj}(A)$ is the transpose of the cofactor matrix:

\text{adj}(A) = [C_{ij}]^T

C_{ij} = (-1)^{i+j} M_{ij}

$M_{ij}$ is the minor (determinant of submatrix after removing row $i$ and column $j$ ).

4. Rank of a Matrix

4.1 Definition

The rank of a matrix $A$ , denoted $\rho(A)$ or $\text{rank}(A)$ , is:

The maximum number of linearly independent row vectors
The maximum number of linearly independent column vectors
The order of the highest non-zero minor

4.2 Finding Rank Using Elementary Transformations

Reduce $A$ to echelon form using elementary row operations. The rank equals the number of non-zero rows in the echelon form.

4.3 Rank Properties

$\rho(A) = \rho(A^T)$
$\rho(A + B) \leq \rho(A) + \rho(B)$

5. Solution of System of Linear Equations

5.1 Matrix Representation

A system of $m$ linear equations in $n$ unknowns:

\begin{cases} a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n = b_1 \\ a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n = b_2 \\ \vdots \\ a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n = b_m \end{cases}

In matrix form:

AX = B

where:

A = \begin{bmatrix} a_{ij} \end{bmatrix}_{m \times n}, \quad X = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}, \quad B = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{bmatrix}

5.2 Homogeneous Systems ( $AX = 0$ )

Always consistent (trivial solution $X = 0$ exists)
Non-trivial solutions exist if and only if $\det(A) = 0$ (for square $A$ )
If $\rho(A) = r < n$ , there are linearly independent solutions

5.3 Non-Homogeneous Systems ( $AX = B$ , $B \neq 0$ )

Consistency Conditions (Rouché-Capelli Theorem):

The system $AX = B$ is consistent if and only if:

\rho(A) = \rho([A \mid B])

where $[A \mid B]$ is the augmented matrix.

Nature of Solutions:

Condition	Result
$\rho(A) = \rho([A \mid B]) = n$	Unique solution
$\rho(A) = \rho([A \mid B]) < n$

5.4 Solution Using Matrix Inverse

If $A$ is square and $\det(A) \neq 0$ :

X = A^{-1}B

5.5 Cramer's Rule

For a system $AX = B$ with $n$ equations and $n$ unknowns, if $\det(A) \neq 0$ :

x_i = \frac{\det(A_i)}{\det(A)}

where $A_i$ is the matrix formed by replacing the $i$ -th column of $A$ with $B$ .

6. Characteristic Equation

6.1 Formation

For a square matrix $A$ of order $n$ , the characteristic equation is:

\det(A - \lambda I) = 0

or equivalently:

|A - \lambda I| = 0

Expanding this gives:

\lambda^n + c_1\lambda^{n-1} + c_2\lambda^{n-2} + \cdots + c_n = 0

6.2 Characteristic Roots (Eigenvalues)

The roots $\lambda_1, \lambda_2, \ldots, \lambda_n$ of the characteristic equation are the eigenvalues of .

Properties of Eigenvalues:

Sum of eigenvalues = trace of $A$ : $\sum_{i=1}^{n} \lambda_i = \text{tr}(A) = \sum_{i=1}^{n} a_{ii}$

7. Cayley-Hamilton Theorem

7.1 Statement

Every square matrix satisfies its own characteristic equation.

If the characteristic equation of $A$ is:

\lambda^n + c_1\lambda^{n-1} + c_2\lambda^{n-2} + \cdots + c_n = 0

Then:

A^n + c_1A^{n-1} + c_2A^{n-2} + \cdots + c_nI = 0

7.2 Finding Higher Powers of Matrices

From Cayley-Hamilton:

A^n = -c_1A^{n-1} - c_2A^{n-2} - \cdots - c_nI

This allows expressing any higher power $A^m$ ( $m \geq n$ ) in terms of $I, A, A^2, \ldots, A^{n-1}$ .

7.3 Finding Inverse Using Cayley-Hamilton

From the characteristic equation:

A^n + c_1A^{n-1} + \cdots + c_{n-1}A + c_nI = 0

Multiply by $A^{-1}$ (assuming $c_n = \det(A) \neq 0$ ):

A^{n-1} + c_1A^{n-2} + \cdots + c_{n-1}I + c_nA^{-1} = 0

Solving for $A^{-1}$ :

A^{-1} = -\frac{1}{c_n}\left(A^{n-1} + c_1A^{n-2} + \cdots + c_{n-1}I\right)

8. Linear Dependence and Independence of Vectors

8.1 Linear Combinations

A vector $v$ is a linear combination of vectors $v_1, v_2, \ldots, v_k$ if:

v = c_1v_1 + c_2v_2 + \cdots + c_kv_k

for some scalars $c_1, c_2, \ldots, c_k$ .

8.2 Linearly Dependent Vectors

Vectors $v_1, v_2, \ldots, v_k$ are linearly dependent if there exist scalars , not all zero, such that:

c_1v_1 + c_2v_2 + \cdots + c_kv_k = 0

Equivalent Conditions:

At least one vector can be written as a linear combination of the others
The matrix with these vectors as columns has rank $< k$
$\det(V) = 0$ if $V$ is a square matrix of these vectors

8.3 Linearly Independent Vectors

Vectors $v_1, v_2, \ldots, v_k$ are linearly independent if:

c_1v_1 + c_2v_2 + \cdots + c_kv_k = 0 \implies c_1 = c_2 = \cdots = c_k = 0

Equivalent Conditions:

No vector can be written as a linear combination of the others
The matrix with these vectors as columns has rank $= k$
$\det(V) \neq 0$ if $V$ is square

9. Eigenvalues and Eigenvectors

9.1 Finding Eigenvalues

Solve the characteristic equation:

\det(A - \lambda I) = 0

9.2 Finding Eigenvectors

For each eigenvalue $\lambda_i$ , solve:

(A - \lambda_i I)X = 0

The non-zero solutions $X$ are the eigenvectors corresponding to $\lambda_i$ .

If $v$ is an eigenvector for $\lambda$ , then any non-zero scalar multiple $kv$ is also an eigenvector.

9.3 Eigenspace

The set of all eigenvectors corresponding to $\lambda$ (plus the zero vector) forms a subspace called the eigenspace:

E_\lambda = \{ v \mid (A - \lambda I)v = 0 \}

9.4 Algebraic and Geometric Multiplicity

Algebraic multiplicity: Number of times $\lambda$ appears as a root of the characteristic equation
Geometric multiplicity: Dimension of the eigenspace $E_\lambda$ (number of linearly independent eigenvectors)

Always: Geometric Multiplicity $\leq$ Algebraic Multiplicity

9.5 Diagonalization

A matrix $A$ of order $n$ is diagonalizable if there exists an invertible matrix $P$ such that:

P^{-1}AP = D = \text{diag}(\lambda_1, \lambda_2, \ldots, \lambda_n)

Conditions for Diagonalization:

$A$ has $n$ linearly independent eigenvectors
Geometric multiplicity = Algebraic multiplicity for every eigenvalue

The columns of $P$ are the linearly independent eigenvectors of $A$ .

9.6 Engineering Applications

Vibration Analysis: Natural frequencies are eigenvalues; mode shapes are eigenvectors
Principal Component Analysis (PCA): Eigenvectors of covariance matrix define principal components
Stability Analysis: System stability determined by eigenvalue locations
PageRank: Google's algorithm uses dominant eigenvector of the web graph

10. Complex Matrices

10.1 Complex-Valued Matrices

A complex matrix has entries from $\mathbb{C}$ :

A = [a_{ij}] \quad \text{where} \quad a_{ij} \in \mathbb{C}

Conjugate Matrix:

\overline{A} = [\overline{a}_{ij}]

where $\overline{a}_{ij}$ is the complex conjugate of $a_{ij}$ .

Conjugate Transpose (Hermitian Adjoint):

A^* = A^\dagger = (\overline{A})^T = \overline{(A^T)}

11. Special Types of Complex Matrices

11.1 Hermitian Matrix

A square matrix $A$ is Hermitian if:

A = A^* \quad \text{or} \quad A = \overline{A}^T

Properties:

$a_{ij} = \overline{a}_{ji}$
Diagonal elements are real numbers
Eigenvalues are always real
Eigenvectors corresponding to distinct eigenvalues are orthogonal

11.2 Skew-Hermitian Matrix

A square matrix $A$ is skew-Hermitian if:

A = -A^* \quad \text{or} \quad A = -\overline{A}^T

Properties:

$a_{ij} = -\overline{a}_{ji}$
Diagonal elements are purely imaginary (or zero)
Eigenvalues are purely imaginary or zero

11.3 Unitary Matrix

A square matrix $A$ is unitary if:

A A^* = A^* A = I \quad \text{or} \quad A^{-1} = A^*

Properties:

Preserves inner products: $\langle Au, Av \rangle = \langle u, v \rangle$
Preserves norms: $\|Au\| = \|u\|$
Columns (and rows) form an orthonormal set
Eigenvalues have absolute value :

12. Applications to Engineering Problems

12.1 Matrix-Based Modeling

Circuit Analysis: Nodal admittance matrices for electrical networks
Structural Engineering: Stiffness matrices in finite element analysis
Control Systems: State-space representation $\dot{x} = Ax + Bu$

12.2 Systems of Linear Equations

Engineering systems modeled as $AX = B$ :

Network flow problems
Heat distribution (discretized Laplace equation)
Force equilibrium in truss structures

12.3 Engineering Computations

Signal Processing: Discrete Fourier Transform uses unitary matrices
Computer Graphics: Rotation, scaling, and projection matrices
Robotics: Transformation matrices for kinematics
Quantum Computing: Quantum states represented by complex vectors; operations by unitary matrices

13. Common Mistakes to Avoid

Mistake	Correction
Assuming $AB = BA$	Matrix multiplication is not commutative
Using $\det(A + B) = \det(A) + \det(B)$

14. Best Practices & Architecture

Always reduce to echelon form first when finding rank or solving systems
Verify $\det(A) \neq 0$ before computing $A^{-1}$
Check algebraic vs. geometric multiplicity before claiming diagonalizability
Use unitary/orthogonal transformations for numerical stability in computations
Leverage sparsity — engineering matrices are often sparse; use sparse matrix algorithms

15. Practice Problems

Problem 1: Find the rank of:

A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 7 \\ 3 & 6 & 10 \end{bmatrix}

Problem 2: Find the inverse of:

A = \begin{bmatrix} 2 & -1 \\ 1 & 3 \end{bmatrix}

using elementary transformations.

Problem 3: Solve the system:

\begin{cases} x + 2y + 3z = 6 \\ 2x + 4y + 7z = 14 \\ 3x + 6y + 10z = 19 \end{cases}

Problem 4: Find eigenvalues and eigenvectors of:

A = \begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix}

Problem 5: Verify Cayley-Hamilton theorem for:

A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}

and use it to find $A^{-1}$ .

16. Mini Project

Title: Structural Load Analysis Using Matrix Methods

Objective: Model a simple truss structure as a system of linear equations $KX = F$ , where $K$ is the stiffness matrix, $X$ is the displacement vector, and $F$ is the force vector.

Tasks:

Construct the stiffness matrix $K$ for a 3-node, 2-element truss
Compute $\det(K)$ to verify the structure is statically determinate
Solve for displacements using $X = K^{-1}F$
Calculate member forces from displacements
Verify your solution using rank/consistency checks

Deliverable: A Python/MATLAB script that accepts geometry and load inputs, builds the matrix system, solves it, and outputs member forces with validation checks.

Quick Reference Formula Sheet

Concept	Formula
Inverse (2×2)	$A^{-1} = \frac{1}{ad-bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$

End of Unit 1 — Matrices

Unit 1 — Matrices (8 Hours)

1. What Are Matrices?

Why Matrices Matter in Engineering

2. Elementary Transformations

2.1 Row and Column Transformations

2.2 Standard Forms

2.3 Elementary Matrices

3. Inverse of a Matrix

3.1 Definition and Properties

3.2 Finding Inverse Using Elementary Transformations

3.3 Inverse Using Adjugate

4. Rank of a Matrix

4.1 Definition

4.2 Finding Rank Using Elementary Transformations

4.3 Rank Properties

5. Solution of System of Linear Equations

5.1 Matrix Representation

5.2 Homogeneous Systems (AX=0AX = 0AX=0)

5.3 Non-Homogeneous Systems (AX=BAX = BAX=B, B≠0B \neq 0B=0)

5.4 Solution Using Matrix Inverse

5.5 Cramer's Rule

6. Characteristic Equation

6.1 Formation

6.2 Characteristic Roots (Eigenvalues)

7. Cayley-Hamilton Theorem

7.1 Statement

7.2 Finding Higher Powers of Matrices

7.3 Finding Inverse Using Cayley-Hamilton

8. Linear Dependence and Independence of Vectors

8.1 Linear Combinations

8.2 Linearly Dependent Vectors

8.3 Linearly Independent Vectors

9. Eigenvalues and Eigenvectors

9.1 Finding Eigenvalues

9.2 Finding Eigenvectors

9.3 Eigenspace

9.4 Algebraic and Geometric Multiplicity

9.5 Diagonalization

9.6 Engineering Applications

10. Complex Matrices

10.1 Complex-Valued Matrices

11. Special Types of Complex Matrices

11.1 Hermitian Matrix

11.2 Skew-Hermitian Matrix

11.3 Unitary Matrix

12. Applications to Engineering Problems

12.1 Matrix-Based Modeling

12.2 Systems of Linear Equations

12.3 Engineering Computations

13. Common Mistakes to Avoid

14. Best Practices & Architecture

15. Practice Problems

16. Mini Project

Quick Reference Formula Sheet

5.2 Homogeneous Systems ( $AX = 0$ )

5.3 Non-Homogeneous Systems ( $AX = B$ , $B \neq 0$ )