1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
|
#set document(
title: [Lecture 1],
author: "Gidon Rosalki",
date: auto,
)
#set heading(numbering: "1.")
#set text(font: "New Computer Modern")
#set page(margin: (
bottom: 0.5cm,
right: 1.5cm,
left: 1.5cm,
top: 1.5cm,
))
#set list(indent: 10pt)
#set enum(indent: 10pt)
#show title: set align(center)
#show link: set text(rgb("005eff"))
#show link: underline
#show math.equation.where(block: true): set block(breakable: true)
#title()
#align(center)[
Gidon Rosalki \
2026-10-13
]
#let colourmaths(x, color) = text(fill: color)[$#x$]
= Introduction
Since everyone comes with a different background in mathematics to their masters in CompSci, this course exists to
ensure everyone knows a standard foundation. \
The course is split into 4 main parts, Linear Algebra, Infinitesimal Calculus, Optimisation, and Probability. Aside from
the optimisation section, it will all be content already taught in one's bachelors degree. The purpose is to give us
skills that we may use in our research.
= Linear Algebra
We will be focusing on slightly less abstract concepts than in the bachelors courses. We will only work over the field
$RR$, real matrices, and vector spaces, and so on, rather than abstract fields, spotlighting applicative concepts.
== Reminder
=== Matrices
Let there be a matrix $M in "Mat"_(n times m) (RR)$. The vectors will all be column vectors (unless stated otherwise) $v
in RR^(n)$. We may transpose our matrix $M : M^T in "Mat"_(m times n) (RR)$ defined such that $M^T_(i, j) = M_(j, i)$.
If $M$ is a square matrix: $M in "Mat"_(n times n) (RR)$, it is called *invertible* if $exists M^(-1) in M_(n) (RR) : M
dot M^(-1) = I_n$.
=== Eigenvalues and Eigenvectors
Let there be $M in "Mat"_(n) (RR)$.
*Definition* (eigenvalue): $lambda in RR$ is called an *eigenvalue* of $M$ if $exists v in RR^n != 0 : M dot v = lambda
dot v$. The eigenvalues of a matrix are called its _spectrum_.
*Definition* (eigenvector): Given an eigenvalue $lambda$ of matrix $M$, where $M dot v = lambda dot v$, then $v in RR^n
!= 0$ is called an *eigenvector*.
*Definition* (eigenspace): The *eigenspace* of $lambda$ is defined as: $V_lambda = {v in RR^n : M v = lambda v}$. \
It is easy to show that $V_lambda$ is a linear subspace of $RR^n$. \
$dim (V_lambda)$ is the (geometric) multiplicity of $lambda$.
*Fact*:
$
sum_(lambda "eigenvalue of" M)^() dim (V_lambda) <= n
$
*Definition* (diagonalisable): $M$ is diagonalisable if there is an invertible matrix $P$, and a diagonal matrix $D$
such that $M = P D P^(-1)$, where $P, D in M_(n)$.
Let $v_i$ be the $i$-th column of $P$. Then
$
M dot v_i & = P D P^(-1) dot v_i \
P dot P^(-1) ==> quad & = P D e_i \
& = P dot D_(i i) dot e_i \
& = D_(i, i) P e_i \
& = D_(i, i) v_i \
$
This implies that $D_(i, i)$ is an eigenvalue, with eigenvector $v_i$. From this we may conclude:
*Theorem*: $M$ is diagonalisable $<==>$ there is a basis $v_1, dots, v_n$ of $RR^n$ of eigenvectors of $M$.
=== Orthogonal Diagonalisation
We will firstly discuss the standard inner product. There are many different inner products, but we will focus on the
standard one:
$
x, y in RR^n ==> lr(chevron.l x, y chevron.r) = sum_(i = 1)^(n) x_i dot y_i
$
Let $0 != v_1, dots, v_n in RR^n$.
*Definition* (orthogonal): $v_1, dots, v_n$ are _orthogonal_ if $forall i != j quad lr(chevron.l v_i, v_j chevron.r) = 0$
*Definition* (orthonormal): $v_1, dots, v_n$ are _orthonormal_ if they are orthogonal, and \
$forall i quad norm(v_i) = sqrt(lr(chevron.l v_i, v_i chevron.r)) = 1$ (unit vectors)
*Definition* (orthogonal matrix): An $n times n$ matrix $V$ is called _orthogonal_ if $V^T dot V = I$. Equivalent to:
+ The columns of $V$ are orthonormal
+ The rows of $V$ are orthonormal
+ $V^(-1) = V^T$
+ $forall x in RR^n quad norm(V x) = norm(x)$
*Definition* (orthogonally diagonalisable): A matrix $M$ is _orthogonally diagonalisable_ if there exist orthogonal matrix
$V$, diagonal matrix $D$ such that $M = V dot D dot V^T$.
*Theorem* (Spectral theorem for symmetric matrices): If $M$ is symmetric ($M = M^T$), then $M$ is orthogonally
diagonalisable, i.e. there is an orthonormal basis of eigenvectors. In other words $M = V dot D dot V ^ (T)$, where $V$
is orthogonal, and $D$ is diagonal.
|
