# Matrix Algebra

The FEM is a numerical approach which results in systems of equations often involving thousands and thousands of unknowns. The use of matrix algebra turns out to be very convenient when dealing with FEM systems as we will see later on (Assembly of systems etc.).

In our third chapter of the FEM course I will show you some basic fundamentals of matrix algebra. However no proofs will be given otherwise I would never come to an end typing in all the LaTeX expressions. But if you want to learn more about matrix algebra in general I recommend you to watch the Linear Algebra course by Gilbert Strang (math professor @MIT):

So what we are doing is basically recall the elementary results of matrix algebra.

Let’s start!

# 1. Definitions

A **matrix** consists of so called **components**. These components are ordered in **rows** and **columns**. We can say that if the number of rows or the columns is equal to one, the matrix is **one-dimensional**, otherwise we have a **two-dimensional matrix**.

**Example:** We consider now a column matrix, where the number of columns is equal to one.

We write vectors of this type with a lower-case letter in bold type.

Where for instance a_1,a_2 and a_3 are the so called components of the matrix \mathbf{a}.

The specific dimension of a matrix is given by the number of rows and columns.

Matrix \mathbf{a} and \mathbf{b} is 3 x 1 but the dimension of matrix \mathbf{c} is 4 x 1.

You first indicate the **row** **THEN** the **column**.

### Transpose

The transpose of our matrix \mathbf{a} is \mathbf{a}^\intercal

which now is the dimension 1 x 3 (remember: first row then column).

### 2-D Matrix

We write a two-dimensional matrix usually with an upper-case letter in bold:

\mathbf{B} is said to be a **square matrix** since the number of rows and the number of columns is equal. (Dimension of \mathbf{B} is 3x3 and \mathbf{C} is 4 x 2)

How do we get the transpose of this matrix now? Is this hard? Not at all! Just interchange rows and columns!

Another property of our matrix \mathbf{B} is symmetry. \mathbf{B} = \mathbf{B}^\intercal.

We will often encounter symmetric matrices because if the continuum body is in static equilibrium it can be demonstrated that the components of the Cauchy stress tensor in every material point in the body satisfy the equilibrium equations (Cauchy’s equations of motion for zero acceleration).

At the same time, according to the principle of conservation of angular momentum, equilibrium requires that the summation of moments with respect to an arbitrary point is zero, which leads to the conclusion that the stress tensor is symmetric, thus having only six independent stress components, instead of the original nine

### Diagonal Matrix

If only the diagonal of a matrix is different from zero, our matrix is termed a **diagonal matrix**.

### Unit Matrix

If all the diagonal components of a diagonal matrix are qual to 1, the matrix is a so called **unit matrix**.

We often write a unit matrix as \mathbf{I}.

### Zero Matrix

If all components of a matrix are zero we call the matrix a zero matrix \mathbf{0}

# 2. Addition and Subtraction

If two matrices have the **same dimension** the can be added and subtracted.

For transposed matrices we have

and for a two-dimensional matrix we have

Some rules follow:

Moreover, we have the following rules

# 3. Multiplication

A matrix can be multiplied by a number *c* and this implies that each component is multiplied by the number *c*.

The same rule applies to two-dimensional matrices.

### Scalar product

The *scalar product* of two matrices \mathbf{a} and \mathbf{b} having the **same** dimension is defined according to

where n is the number of rows. It follows that

With \mathbf{a} and \mathbf{b} given from the beginning we obtain

The *length* of \mathbf{a} or \mathbf{a}^\intercal is denoted by \vert \mathbf{a} \vert and defined by

Let us now generalize the multipliation rule (14) to more complicated situations.

Let \mathbf{A} have the dimension *m x n* and \mathbf{x} the dimension *n x 1*; then the matrix product \mathbf{Ax} defines a row matrix \mathbf{c} with the dimension *m x 1* according to

In a similar manner let now \mathbf{x} have the dimension *m x 1* and \mathbf{A} the dimension *m x n* , then the matrix product \mathbf{x}^\intercal \mathbf{A} defines a row matrix \mathbf{c}^\intercal with the dimension *1 x n* according to

An example would be

Finally, we can carry out matrix multiplication of two-dimensional matrices. If \mathbf{A} has the dimension *m x n* and \mathbf{B} the dimension *n x p*, the matrix product \mathbf{AB} defines the matrix \mathbf{C} with the dimension *m x p* according to

See the following example

We keep in mind that matrix multiplication is only defined if the matrices possess the correct dimensions!

Moreover if \mathbf{A} has the dimension *m x n* and \mathbf{B} the dimension *n x m* both multiplications \mathbf{AB} and \mathbf{BA} are defined **BUT**

Why? Because \mathbf{AB} defines a matrix with the dimension *m x m* , whereas \mathbf{BA} defines a matrix with the dimension *n x n*.

Even if m = n (24) will (in general) hold true.

and

### Some definitions

We also keep in mind that

(28) implies

we also have

Moreover, it follows trivially that if *c* denotes a scalar (number) then

Finally, we recall the distribution law from ordinary algebra which also holds for matrices!

Tip: If you want to get the resulting dimension of a matrix write down both matrix dimensions. For instance matrix \mathbf{A} has the dimension *1 x m* and matrix \mathbf{B} has the dimension *m x n*. We write

In the next step we erase the following components

**The leftover of our trick is 1 x n which is the new matrix dimension, so simple! **

### Excursus

We often name matrices tensors. The order/rank of the tensor tells us what kind of matrix we have.

Do tensors of the order \geq{3} exist? Yes!

A simple example would be a tensor of order 4 namely the Young’s modulus having 81 components.

Hooke’s law in tensor notation looks like this

Our Young’s modulus has to be a tensor of order four so that every component of \sigma can depend of every component of \epsilon.

# 4. Determinant

For the square matrix \mathbf{A}, it is possible to calculate the so called *determinant* det \mathbf{A} of \mathbf{A}. Let \mathbf{A} be of dimension *n x n* . If n = 1 then, by definition, det \mathbf{A} = A_{11}. Before it is possible to determine det \mathbf{A} for n \geq 2 some definitions have to be made.

If row number i and column number k are deleted, a new square matrix with dimension (n - 1) x (n - 1) emerges. The determinant of this matrix is called a **minor** of \mathbf{A} and is denoted by det M_{ik}. The **cofactor** of \mathbf{A} is denoted by A^c_{ik} and is defined by

According to the **expansion formula of Laplace** we then have

where i indicates any row number by the range 1 \leq i \leq n .

As an example consider \mathbf{A} given by

It follows that

Suppose we use formula (34) and chose i = 1; then with formula (33) and (36) we obtain the following result

We can verify that the same result is obtained if we chose i = 2 instead of 1 in (34).

(34) expresses the determinant as a certain linear combination of all components A_{ik} in the arbitrary row i. The expansion formula of Laplace can also be written as a linear combination of all components in an arbitrary column. We have

where j indicates any column number in the range 1 \leq j \leq n. As an example consider \mathbf{A} given as in (35) and choose j = 1. Then with (38),(33) and (36) we obtain

in accordance with (37).

In practice, the direct use of the expansion formulae of Laplace in numerical calculations is unsuitable since many operations are necessary.

In theory, these expansion formulae are very important as they enable one to establish a number or properties for determinants.

It can be shown

For later purposes it is convenient to recall that a row (or column) operation is an operation where all components of one row (or column) are multiplied by a certain factor and then added to another row (or column). We then have

Finally, we note that

whereas

# 5. Inverse Matrix

The *inverse* \mathbf{A}^{-1} of a square matrix \mathbf{A} is defined by

By means of the cofactor A^c_{ik} of \mathbf{A} as defined by (33), it is possible to present an explicit expression for the inverse matrix \mathbf{A}^{-1}.

For all values of i and k, we are able to construct a square matrix of dimension n x n by means of (33). If this matrix is transposed, we obtain the so called ** adjoint matrix** of \mathbf{A} given by

The inverse matrix is given by

Please remember that \mathbf{A}^{-1} only exists if det \mathbf{A} \neq 0. A square matrix \mathbf{A} for which det \mathbf{A} = 0 is called a **singular matrix**. We recall that

and the latter expression implies

We note that if

then the matrix is called an * orthogonal matrix*. If \mathbf{A} is orthogonal, we have \mathbf{A^\intercal A} = \mathbf{A A^\intercal} = I

## 6. Linear equations: Number of equations is equal to number of unknowns

Consider the following system of equations

where \mathbf{A} is a square matrix (*n x n*) and \mathbf{x} and \mathbf{b} both have the dimension (*n x 1*).

If \mathbf{b} = 0 then (49) is called a * homogeneous system* of equations otherwise we call it an

*of equations.*

**inhomogeneous system**Assuma that

then by premultiplying (49) by \mathbf{A}^{-1} we get

which, due to (42), gives us

For the homogeneous system of equations

for which det \mathbf{A} only has the trivial solution \mathbf{x} = \mathbf{0}. The non-trivial solution of (52) requires that det \mathbf{A} = 0.

`Homogeneous systems of equaions are of importance, for instance in buckling and vibration problems.`

For homogeneous systems of equations, we have

For inhomogeneous systems of equations, we have

Depending on the specific \mathbf{b}-matrix we may have no solution or an infinite amount of solutions if det \mathbf{A} = 0.

In practical numerical calculations the direct establishment of the inverse \mathbf{A}^{-1} is too cumbersome and other approaches like the common and efficient ** Gauss elimination** are used.

In Gauss elimination, the equations are combined in such a manner that the lower left part of the new coefficient matrix (\mathbf{A}) consists of zeros. To get there, a multiple of the first equation is added to the second one such that the first component of the second row in the new coefficient matrix becomes zero. A multiple of the first equation is then added to the third equation such that the first component of the third row in the new coefficient matrix becomes zero.

Equation (54) is then transformed into the form

We say \mathbf{A'} and \mathbf{b} to emphasize that the new coefficient matrix and right-hand side have changed as a result of our elimination process. \mathbf{A'x} has the form

As you can see the lower left part of our matrix consists of zeros. We can also say that the system of equations has been * triangularized*. The diagonal elements A'_{11},A'_{22},…,A'_{nn} are called

*.*

**pivot elements**Easy-peasy now! From the last equation,we obtain

This solution is substituted into equation number *n - 1* to provide x_{n - 1} and we continue in this manner until we got all \mathbf{x}-components. We call this process **back-substitution**

## 7. Linear equations: number of equations is different from number of unknowns

We consider the linear homogeneous system of equations \mathbf{Ax} = \mathbf{0} where \mathbf{A} has the dimension *m x n*, whereas \mathbf{x} has the dimension *n x 1* and \mathbf{0} the dimension *m x 1* (furthermore n > m).

# 8. Quadratic forms and positive definiteness

For a square matrix \mathbf{A} consider the quantity \mathbf{x^\intercal Ax}, which is a scalar (a number). This quantity is called a * quadratic form*. If

then the matrix \mathbf{A} is said to be * positive definite*. We will see later on why positive definite matrices are very important!

From (59) follows that the determinant of a positive definite matrix is different from zero. Assume that (59) holds and that det \mathbf{A} = 0.

Since det \mathbf{A} = 0, it is possible to determine a non-trivial solution to the homogeneous system of equations \mathbf{Ax} = \mathbf{0}, which means that our equation (59) is violated. This means det \mathbf{A} \neq 0 must hold for positive definite matrices,id est

The inverse argument does not hold! If det \mathbf{A} \neq 0 we cannot conclude that \mathbf{A} is positive definite.

Also keep in mind that the square matrix \mathbf{A} is called positive * semi-definite* if

# 9. Partitioning

To facilitate matrix multiplications, it may be useful to *partition* a matrix into so called ** submatrices**. The idea is demonstrated below where the lines show the partitioning

Using this partitioning the matrix \mathbf{A} can be written in the following form

where

# 10. Differentiation and integration

Consider the matrix \mathbf{A} given by

If the components of the matrix depend on a variable x, we define * matrix differentiation* as

So all components are differentiated with respect to x. If we have a system of equations

where \mathbf{A} and \mathbf{f} depend on x, whereas \mathbf{b} is constant. Differentiation of (70) yields

* Integration of the matrix* \mathbf{A} given by (68) is defined as an integration of each component, i.e.

**Sources:**

• Introduction to the Finite Element Method - Niels Ottosen & Hans Petersson

• Cauchy stress tensor - Wikipedia

• Tensor Rank -- from Wolfram MathWorld

• Gaussian elimination - Wikipedia

This was the third part for FEM fundamentals. The following chapter will be about the direct approach for discrete systems.

If you find any mistakes or have wishes for the next chapters please let me know.

First chapter: The Finite Element Method - Fundamentals - Introduction [1]

Second chapter: The Finite Element Method - Fundamentals - Physical Problems [2]

Do not hesitate to ask questions. Avoid writing me a private message because your question(s) might also be useful for others.