16 Intro to Linear Algebra - DRAFT π
In this tutorial, we introduce basic linear algebra concepts and the way they can be computed using Fastmath.
16.1 What is it about?
Linear algebra focuses on vectors and certain kinds of transformations of them, called linear transformations.
In our context here, our vectors are ordered collections of floating-point numbers.
This is a concrete special case of a more absract and more general notion of elements of vector spaces. As usual, abstraction can be useful for our reasoning. We recommend learning abouthe more general ideas of linear algebra. Probably, for Clojurians who appreciate simplicity and functional composition, those ideas can be attractive.
Linear transformations are transformations which are simple, in a certain sense which can be made precise. They are often useful when mixed and composed with nonlinear transformations, and of course, the Fastmath API offers both kinds, as we will see below.
Implementation-wise, vectors can be represented in many ways. Clojureβs persistent vectors are one way (assuming they contain only numbers). It is a bit heavy in time and space, since all numbers are boxed in objects. Java arrays are a more lightweight representatin.
16.2 Recommended reading
16.2.1 Applied
Introduction to Applied Linear Algebra β Vectors, Matrices, and Least Squares by Stephen Boyd & Lieven Vandenberghe, Cambridge University Press, UK, 2018.
Numerical Linear Algebra for Programmers - An Interactive Tutorial with GPU, CUDA, OpenCL, MKL, Java, and Clojure by Dragan Djurich, 2019.
Mathematics for Machine Learning by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon, Cambridge University Press, 2020.
16.2.2 Abstract
- Linear Algebra Done Right 4th by Sheldon Axler, Springer, 2024.
16.3 Setup
(ns noj-book.linear-algebra-intro
(:require
[clojure.math :as math]
[fastmath.vector :as vec]
[scicloj.kindly.v4.api :as kindly]))Define a utility variable for demonstration purposes: a Java double array of length 5.
(def java-double-array (double-array 5))16.4 Linear Algebra Concepts with fastmath.vector
Below are examples and explanations of common operations using the fastmath.vector library.
16.4.1 What is a Linear Transformation?
A linear transformation is a mathematical function that maps vectors from one vector space to another (or to itself) while preserving the operations of vector addition and scalar multiplication. In simpler terms, it is a transformation that maintains the structure of a vector space.
16.4.2 Vector Addition
Adding two vectors element-wise. This operation combines corresponding elements of two vectors.
(vec/add [1 9]
[0 -3])[1.0 6.0]16.4.3 Scalar Multiplication
Multiplying each element of a vector by a scalar value. This operation scales the vector proportionally.
(vec/mult [1 9]
1000)[1000.0 9000.0]16.4.4 Vector Subtraction
16.4.4.1 Case 1: Single vector passed to vec/sub
Negates the vector by multiplying each element by -1.0.
(vec/sub [10 5])[-10.0 -5.0]16.4.4.2 Case 2: Two vectors passed to vec/sub
Performs element-wise subtraction between the first and second vectors.
(vec/sub [10 5]
[8 4])[2.0 1.0]16.4.5 Dot Product
The dot product of two vectors is a scalar that measures the similarity of their directions. It is calculated as: a Β· b = aβbβ + aβbβ + ... + aβbβ.
(vec/dot [10 5]
[8 4])100.016.4.6 Converters
The following examples demonstrate various conversions between vector types and formats.
16.4.6.1 Convert a vector to a Java array of doubles [D.
(vec/vec->array [10 5])[10.0, 5.0](type (vec/vec->array [10 5]))double/116.4.6.2 Convert a Java double array back to a Clojure sequence.
(identity java-double-array)[0.0, 0.0, 0.0, 0.0, 0.0](type (vec/vec->seq java-double-array))clojure.lang.ArraySeq$ArraySeq_double16.4.6.3 Convert a vector or Java array to an Apache Commons Math RealVector.
(type (vec/vec->RealVector [10 5]))org.apache.commons.math3.linear.ArrayRealVector(identity java-double-array)[0.0, 0.0, 0.0, 0.0, 0.0](type (vec/vec->RealVector java-double-array))org.apache.commons.math3.linear.ArrayRealVector16.4.6.4 Convert a Clojure vector or Java array to a primitive vector Vec.
(vec/vec->Vec [10 5])[10.0 5.0](type (vec/vec->Vec [10 5]))clojure.core.Vec(identity java-double-array)[0.0, 0.0, 0.0, 0.0, 0.0](type (vec/vec->Vec java-double-array))clojure.core.Vec16.4.7 Specialized Operations
16.4.7.1 Transform elements from one vector to match the count of another.
This operation extracts the same number of elements from the second vector as there are in the first.
(vec/as-vec [10 2] [5 10 15])[5 10]16.4.7.2 Create a zero vector matching the size of the input vector.
(vec/as-vec [5 10 15])[0.0 0.0 0.0]16.4.8 Vector Magnitude
Calculates the magnitude (length) of a vector using the Pythagorean theorem.
(vec/mag [3 4])5.016.4.9 Approximation
Rounds each value in the vector to the specified number of decimal places.
(vec/approx [math/PI])[3.14](vec/approx [math/PI math/PI math/PI] 5)[3.14159 3.14159 3.14159]16.4.10 Equality Tolerance
16.4.10.1 Element-wise Equality with Tolerance
Checks if elements of two vectors are equal within a given absolute (and/or relative) tolerance.
(vec/edelta-eq [math/PI] (vec/approx [math/PI] 4))false(vec/edelta-eq [math/PI] (vec/approx [math/PI] 4) 0.001)true(vec/edelta-eq [math/PI] (vec/approx [math/PI] 4) 0.000001)false16.4.10.2 Vector Equality with Tolerance
Similar to edelta-eq, but compares entire vectors for equality with tolerance.
(vec/delta-eq [math/PI] (vec/approx [math/PI] 4))false(vec/delta-eq [math/PI] (vec/approx [math/PI] 4) 0.000001)false16.4.11 Normalization
Scales the vector to have a magnitude of 1, keeping its direction. This is often used to create unit vectors.
(vec/normalize [3 4])[0.6000000000000001 0.8]16.4.12 Vector Projection
Projects one vector onto another. The result is the component of the first vector in the direction of the second vector.
(def first-real-vec
(vec/vec->RealVector [3 4]))(def second-real-vec
(vec/vec->RealVector [1 0]))(vec/project [3 4] [1 0])[3.0 0.0](vec/project first-real-vec second-real-vec)#object[org.apache.commons.math3.linear.ArrayRealVector 0x1c4e08ef "{3; 0}"]16.4.13 Vector Rotation
Rotates a 2D or 3D vector by a given angle/angles (in radians). Useful for geometric transformations.
(vec/rotate (vec/vec2 [1 2]) (/ math/PI 2))[-2.0 1.0000000000000002](vec/rotate (vec/vec3 [1 2 3]) 0.5 0.5 0.5)[1.3669567839387822 0.6801518465116336 3.4159658394853065]16.4.14 Linear Interpolation
Performs linear interpolation between two vectors based on a parameter t (0 β€ t β€ 1). Great for smooth transitions between points.
(vec/lerp (vec/vec2 [1 1]) (vec/vec2 [4 4]) 0.5)[2.5 2.5]16.4.15 Cross Product
Computes the cross product of two vectors. The result is a vector perpendicular to both input vectors.
(vec/cross (vec/vec3 [1 0 0]) (vec/vec3 [0 1 0]))[0.0 0.0 1.0](vec/cross (vec/vec2 [1 0]) (vec/vec2 [0 1]))1.016.4.16 Angle Between
Calculates the angle (in radians) between two vectors. Useful for determining directional differences.
(vec/angle-between (vec/vec2 [1 0]) (vec/vec2 [0 1]))1.570796326794896616.4.17 Vector Distance
Computes the Euclidean distance between two vectors (points in space).
(vec/distances (vec/vec2 [1 1]) (vec/vec2 [4 5]))[4.0 5.0]16.4.18 Clamp
Restricts each component of a vector to lie within a specified range.
(vec/clamp (vec/vec3 [-10 3 7]) 0 5)[0.0 3.0 5.0]16.4.19 Absolute value
Computes the absolute value of each component of the vector.
(vec/abs [-10 -5 3])[10.0 5.0 3.0]16.4.20 Maximum value
Returns largest element value from vector.
(vec/mx (vec/vec3 [-10 3 7]))7.016.4.21 Maximum value index
Returns largest element index from vector.
(vec/maxdim (vec/vec3 [-10 3 7]))216.4.22 Minimum value
Returns smallest element value from vector.
(vec/mn (vec/vec3 [-10 3 7]))-10.016.4.23 Minimum value index
Returns smallest element index from vector.
(vec/mindim (vec/vec3 [-10 3 7]))016.4.24 Vector Set Magnitude
Rescales a vector to have a specific magnitude without requiring normalization first.
(vec/set-mag (vec/vec2 [10 3]) 4)[3.8313051408846057 1.1493915422653815]16.4.25 Vector Limit Magnitude
Restricts the magnitude of a vector to a specified maximum value.
(vec/limit (vec/vec2 [3 4]) 2)[1.2000000000000002 1.6]16.4.26 Distance Squared
Calculates the squared distance between two vectors
(vec/dist-sq [1 1] [4 5])25.0