19 Linear transformations with fastmath.matrix - DRAFT π
In this tutorial we discuss the notion of linear transformations and show how they can be represented as matrices.
Specifically, we will use fastmath.vector for vectors and fastmath.matrix for matrices.
19.1 Setup
(ns noj-book.fastmath-matrix-intro
(:require [fastmath.vector :as vec]
[fastmath.matrix :as mat]
[fastmath.core :as math]))19.2 Vectors
Recall that vectors are abstract entities that live in vector spaces (or linear spaces).
We often represent vectors them as arrays of numbers. The fastmath.vector API supports a few datatypes of this kind.
(vec/->Vec2 3 4)[3.0 4.0](vec/->Vec3 3 4 -1)[3.0 4.0 -1.0](vec/vec->RealVector
[1 3 9 -3])#object[org.apache.commons.math3.linear.ArrayRealVector 0x70ab88b7 "{1; 3; 9; -3}"]What we care about in linear algebra is mostly the structure of vector spaces, which is defined by the operations of addition and multiplication by a βscalarβ (that is, a single number). These operations are just functions which are assumed to satisfy certain axioms.
When we represent vectors as arrays of numbers, there is a standard way to define such operations: element-by-element.
(vec/add (vec/->Vec2 3 1)
(vec/->Vec2 2 -2))[5.0 -1.0](vec/mult (vec/->Vec2 3 1)
5)[15.0 5.0]Sometimes, vector spaces are endowed by additional structure, which is defined as additional functions over these spaces.
E.g., we may add a notion of magnitude of vectors, so called norm. When we represent vectors as arrays of numbers, one standard way to define a norm is to compute the distance of the corresponding points from zero.
(vec/mag
(vec/->Vec2 3 4))5.0(You may verify that 5 is the distance between (0,0) to (3,4) in the plane using Pythagoras theorem.)
Another additional operation that is a scalar product operation between vectors. This is a function that takes two vectors and returns a scalar (a number). When we represent vectors as arrays of numbers, one standard way to define it is by multiplying element by element and summing up. This is the so-called dot-product.
(vec/dot
(vec/->Vec3 1 10 100)
(vec/->Vec3 1 2 3))321.019.3 Matrices as transformations
Matrices are arrays of numbers of rectangular shape:
(mat/->Mat2x2 0 1 2 0)#mat2x2 [[0.0, 1.0]
[2.0, 0.0]](mat/rows->RealMatrix
[[1 3 9]
[8 -9 7]])#object[org.apache.commons.math3.linear.Array2DRowRealMatrix 0x4a7ab0cd "Array2DRowRealMatrix{{1.0,3.0,9.0},{8.0,-9.0,7.0}}"]Conceptually, we think about matrices as functions, also called mappings or transformations or operators, between vector spaces. As weβll see below, the act as a specific kind of functions, which are linear.
The way to apply the matrix as a function to a vector is called βmultiplying the vector by the matrixβ. We multiply from the left β algebraically, the matrix will be written to the left of the vector.
The multiplication of a \(k \times l\) matrix \(M\) with an \(l\)-dimensional vector \(v\) is an \(k\)-dimensional vector \(Mv\).
Each element of \(Mv\) is a dot product (as defined above) of the corresponding row of \(M\) with \(v\).
19.4 Example: 2x3
- M - 2x3 matrix
- v - 3-dimensional vector
- Mv - 2-dimensional vector
(mat/mulv (mat/rows->RealMatrix
[[1 1 1]
[1 0 -1]])
(vec/vec->RealVector
[10 20 30]))#object[org.apache.commons.math3.linear.ArrayRealVector 0x23811656 "{60; -20}"]Concretely, its elements will be:
(vec/dot (vec/->Vec3 1 1 1)
(vec/->Vec3 10 20 30))60.0(vec/dot (vec/->Vec3 1 0 -1)
(vec/->Vec3 10 20 30))-20.0You see, multiplying this 2x3 matrix by a vector of dimension 3 resulted in a vector of dimension 2. This matrix acts as follows:
- The first new element is the sum of three old elements.
- The second new element is the difference of the first and last old elements.
19.4.1 Example: 1x3
The case of a matrix with one row is basically dot product. (but considering the result as a 1-dim vector rather than a number).
(mat/mulv (mat/rows->RealMatrix
[[1 0 -1]])
(vec/vec->RealVector
[10 20 30]))#object[org.apache.commons.math3.linear.ArrayRealVector 0x4d0ff718 "{-20}"](vec/dot (vec/vec->RealVector
[1 0 -1])
(vec/vec->RealVector
[10 20 30]))-20.019.4.2 Example: 3x2
(mat/mulv (mat/rows->RealMatrix
[[1 1]
[1 0]
[0 1]])
(vec/vec->RealVector
[10 20]))#object[org.apache.commons.math3.linear.ArrayRealVector 0x2f348c04 "{30; 10; 20}"]19.4.3 Application
Assume that our vectors are 2 dimensional and represent our sales: (apples,oranges). We wish to add the income. Assume we receive 20 cents for an apple and 30 cents for an orange.
Example sales: 10 apples, 100 oranges.
(mat/mulv (mat/rows->RealMatrix
[[1 0]
[0 1]
[20 30]])
(vec/vec->RealVector
[10 100]))#object[org.apache.commons.math3.linear.ArrayRealVector 0x57ca8054 "{10; 100; 3,200}"]a lot of cents!!
19.4.4 Example: 2x2
A square-shaped matrix can be seen as a transformation from a vector space to itself. For example, a 2x2 matrix takes vectors of dimension 2 to vectors of dimension 2.
(mat/mulv (mat/->Mat2x2
1 1
1 0)
(vec/->Vec2
10 20))[30.0 10.0]19.4.5 Multiplying by a matrix is a linear transformation
We still need to explain what linear transformations are, and how they can be implemented with matrices.
19.5 Special cases
19.5.1 Identity matrix
This matrix has ones on the main diagonal and zeros elsewhere.
(mat/->Mat2x2 1 0 0 1)#mat2x2 [[1.0, 0.0]
[0.0, 1.0]]Multiplying by this matrix does nothing (as the identity function in Clojure!).
(mat/mulv
(mat/->Mat2x2 1 0 0 1)
(vec/->Vec2 3 4))[3.0 4.0](identity
(vec/->Vec2 3 4))[3.0 4.0]19.5.2 Permutation matrix
This is similar to the identity, but the order of rows is changes.
(mat/->Mat2x2 0 1 1 0)#mat2x2 [[0.0, 1.0]
[1.0, 0.0]]It acts by changing the order of coordinates.
(mat/mulv
(mat/->Mat2x2 0 1 1 0)
(vec/->Vec2 3 4))[4.0 3.0]19.6 Operations on matrices
fastmath.vector offers a rich collection of operations that act on matrices themselves.
For example, transposition reverses the roles of columns and rows:
(mat/->Mat2x2 0 1 2 0)#mat2x2 [[0.0, 1.0]
[2.0, 0.0]](mat/transpose
(mat/->Mat2x2 0 1 2 0))#mat2x2 [[0.0, 2.0]
[1.0, 0.0]]19.7 What are linear transformations?
Given a matrix \(M\), the function \(v \mapsto M v\), or in Clojure, (fn [v] (mat/mul m v)), is of a special kind. It is \(linear\).
Let us discuss what this means.
Given a set of vectors, their linear combinations are all the other vectors one may get from them using addition and multiplication by scalar.
For example, (3,4) is a linear combination of (1,0) and (0,1):
(vec/add (vec/mult (vec/->Vec2 1 0) 3)
(vec/mult (vec/->Vec2 0 1) 4))[3.0 4.0]3 * (1,0) + 4 * (0,1) = (3,4)
Linear transformations are functions vectors->vectors which respect linear combinations.
Example:
(defn T [v]
[(+ (* -1 (v 0))
(* 9 (v 1)))
(+ (* 2 (v 0))
(* -5 (v 1)))])(T [1 0])[-1 2](T [0 1])[9 -5](T
(vec/add (vec/mult (vec/->Vec2 1 0) 3)
(vec/mult (vec/->Vec2 0 1) 4)))[33.0 -14.0](vec/add (vec/mult (T (vec/->Vec2 1 0)) 3)
(vec/mult (T (vec/->Vec2 0 1)) 4))[33.0 -14.0]You see, it does not matter whether we apply T before or after taking a linear combination. In other words, it respects the linear structure of our vector space.
Another example - rotating a vector by 90 degrees (or Pi/2 radians):
(defn R [v]
(vec/rotate v (/ math/PI 2)))(R
(vec/add (vec/mult (vec/->Vec2 1 0) 3)
(vec/mult (vec/->Vec2 0 1) 4)))[-4.0 3.0000000000000004](vec/add (vec/mult (R (vec/->Vec2 1 0)) 3)
(vec/mult (R (vec/->Vec2 0 1)) 4))[-4.0 3.0000000000000004]Another example - multiplying by a matrix:
(defn M [v]
(mat/mulv
(mat/->Mat2x2 -1 9 2 -5)
v))(M
(vec/add (vec/mult (vec/->Vec2 1 0) 3)
(vec/mult (vec/->Vec2 0 1) 4)))[33.0 -14.0](vec/add (vec/mult (M (vec/->Vec2 1 0)) 3)
(vec/mult (M (vec/->Vec2 0 1)) 4))[33.0 -14.0]Actually, note that by the definition of multiplication between matrices and vectors, T and M are actually the same function.
19.8 The standard basis
The standard basis of the vector space of \(n\)-dimensional arrays of numbers is the set of vectors which are all 0 except for one 1.
In dimension \(n=3\), for example:
#{(vec/->Vec3 1 0 0)
(vec/->Vec3 0 1 0)
(vec/->Vec3 0 0 1)}#{[0.0 0.0 1.0] [1.0 0.0 0.0] [0.0 1.0 0.0]}Every vector in these spaces can be expressed as a linear combination of standard basis elements.
For example:
(vec/->Vec3 2 3 4)[2.0 3.0 4.0](-> (vec/mult (vec/->Vec3 1 0 0) 2)
(vec/add (vec/mult (vec/->Vec3 0 1 0) 3))
(vec/add (vec/mult (vec/->Vec3 0 0 1) 4)))[2.0 3.0 4.0]The standard basis vectors are linearly independent: Each of them cannot be expressed as a linear combination of the others.
So: Every vector can be expressed (in a unique way!) as a linear combination of standard basis elements, but we need all of them for that.
E.g., in dimension 3, we need all 3.
19.9 Representing linear transformations as matrices
We have already claimed that matrices act as linear transformations.
Actually, if we represent our vectors as arrays of numbers, any linear transformation \(T\) can be represented as a matrix.
The reason is that, given any vector, we can represent it as a linear combination of standard basis elements, so we can infer the way it is transformed by \(T\), if we look into what \(T\) does to the standard basis elements. But this information can be encoded in a matrix, which acts the same way as \(T\).
19.10 Matrix multiplication
Matrix multiplication is one important operation:
(mat/->Mat2x2 0 1 1 0)#mat2x2 [[0.0, 1.0]
[1.0, 0.0]](mat/->Mat2x2 2 0 3 0)#mat2x2 [[2.0, 0.0]
[3.0, 0.0]](mat/mulm
(mat/->Mat2x2 0 1 1 0)
(mat/->Mat2x2 2 0 3 0))#mat2x2 [[3.0, 0.0]
[2.0, 0.0]]The multiplication \(MN\) of a \(k \times l\) matrix \(M\) with an \(l \times m\) matrix \(N\) is defined as a \(k \times m\) matrix. Each of its columns is the matrix-vector multiplication of \(M\) by the corresponding column of \(N\), seen as a vector.
Importantly, if we see matrices as transformations as suggested above, then multiplication is just composition of functions. At the moment, we will not try to explain why this is true.
Matrix multiplication is non-commutative β the order matters:
(mat/mulm
(mat/->Mat2x2 0 1 1 0)
(mat/->Mat2x2 2 0 0 3))#mat2x2 [[0.0, 3.0]
[2.0, 0.0]](mat/mulm
(mat/->Mat2x2 2 0 0 3)
(mat/->Mat2x2 0 1 1 0))#mat2x2 [[0.0, 2.0]
[3.0, 0.0]]