ΘρϵηΠατπ

Matrices

Matrix

A matrix is a collection of numbers that can be indexed in a particular way, this indexing mimics a two dimensional grid. The dimensions of a matrix are stated as r×c where r,c1 when it has m rows and n columns.

Suppose that A is a matrix, then we can access the value at the i-th row and j-th column by writing Ai,j, we can graphically represent this information by introducing variables for the coefficients and writing them like this:

A=[a1,1a1,2a1,na2,1a2,2a2,nam,1am,2am,n]

Matrices are denoted by capital letters such as A,B,M,N.

Set of Matrices
Suppose that r,c1 then we denote the set of all r×c matrices with cofficients from the set T as Mr×c(T)
Row Matrix
Any matrix RM1×c(T) for some c1 is known as a row matrix.
Row of a Matrix

Suppose that A is a matrix, then if we want to extract the ith row of A we can write Ai,.

Note that this is a function of the form Mr×c(T)M1×c(T)
The notation Ai, intends to mimic freezing at a certain row and then sweeping over all columns.
Row Matrix Representation
Given a matrix A, there is an associated row matrix representation, of the following form
A=[A1,A2,Am,]

Where the arrows are there to remind you that they expand horizontally. Also observe that this matrix is an element of Mr×1(M1×c())

Column Matrix
Any matrix CMr×1(T) for some r1 is known as a row matrix.

We will denote column or row matrices with a lower case boldfaced character, like 𝐱

Column of a Matrix

Suppose that A is a matrix, then if we want to extract the j-th row of A we can write A|,j.

Note that this is a function of the form Mr×c(T)Mr×1(T)
The | denotes that the row variable may be varied vertically, while the column index is fixed in place. If we want to represent the matrix as columns graphically, we may do so like this
Column Matrix Representation
Given a matrix A, there is an associated column matrix representation, of the following form A=[A|,1A|,2A|,n]

Where the arrows are there to remind you that they expand graphically expand vertically. Also observe that this matrix is an element of M1×c(Mr×1()).

We can connect some of the previous definitions here to say that R is a column matrix whose entries are row matrices.
Matrix Rank
Let AMr×c(). The rank of A, denoted rank(A), is the dimension of the span of the columns of A, considered as vectors in r.
Determinant
The determinant is a function det:Mn×n() defined recursively as follows. If A=[a], then detA=a. If n>1, let A1,j be the (n1)×(n1) matrix obtained from A by deleting the first row and the j-th column. Then detA=j=1n(1)1+ja1,jdetA1,j.
Invertible Matrix
A square matrix AMn×n() is invertible if the function LA:nn,LA(x)=Ax, is an invertible function.
Nonzero Determinant iff Invertible
A square matrix AMn×n() is an invertible matrix if and only if detA0.
The determinant measures the signed volume-scaling factor of the linear map LA(x)=Ax. This factor is nonzero exactly when LA sends the standard basis to a basis of n. By the matrix with basis columns is invertible theorem, this is exactly the condition that A be invertible.
Matrix with Basis Columns is Invertible
Let AMn×n(). If the columns of A form a basis of n, then A is an invertible matrix.
Write the columns of A as a1,,an. For any x=(x1,,xn)n, matrix multiplication gives Ax=x1a1++xnan. Since a1,,an form a basis of n, every vector yn has a unique representation y=x1a1++xnan by basis implies unique representation. Therefore there is a unique xn such that Ax=y. Thus LA(x)=Ax is bijective, so it is invertible. Hence A is an invertible matrix.
Matrix Multiplication

Suppose rA,cA,rB,cB1 such that cA=rB and that we have two matrices A,BMrA×cA(),MrB×cB(), defined as

A=[a1,1a1,2a1,cAa2,1a2,2a2,cAarA,1arA,2arA,cA],B=[b1,1b1,2b1,cBb2,1b2,2b2,cBbrB,1brB,2brB,cB]

Then we define the matrix multiplication of A and B as the matrix PMrA×cB

P=[c1,1c1,2c1,cPc2,1c2,2c2,cPcrP,1crP,2crP,cP]

Where:

ci,j:=ai,1b1,j+ai,2b2,j++ai,cAbcA,j=k=1cAai,kbk,j
Matrix Times Standard Basis Vector is a Column
Let AMm×n(), and let ej be the j-th vector in the standard basis of n. Then Aej=A|,j, where A|,j denotes the j-th column of A, viewed as a vector in m.
Write A=[ai,k]. By the definition of matrix multiplication, the i-th coordinate of Aej is k=1nai,k(ej)k. Since (ej)k=0 for kj and (ej)j=1, this sum is ai,j. Therefore every coordinate of Aej equals the corresponding coordinate of the j-th column of A, so Aej=A|,j.
Linear Transformation has a Standard Matrix
Let T:nm be a linear transformation. There exists a unique matrix AMm×n() such that T(x)=Ax for every xn. This matrix is the matrix of T with respect to the standard bases, and is denoted [T]EmEn, where En and Em are the standard bases of n and m. Equivalently, [T(x)]Em=[T]EmEn[x]En. When m=n, this is often abbreviated to [T]E.
Let e1,,en be the standard basis of n. Define A to be the matrix whose j-th column is T(ej): A=[T(e1)T(e2)T(en)]. If x=(x1,,xn)n, then by the standard basis expansion, x=x1e1++xnen. Since T is linear, T(x)=x1T(e1)++xnT(en). By matrix times standard basis vector is a column, the same expression is exactly Ax. Thus T(x)=Ax for every x.

To prove uniqueness, suppose BMm×n() also satisfies T(x)=Bx for every x. Then Bej=T(ej)=Aej for every j. By matrix times standard basis vector is a column, the j-th column of B equals the j-th column of A for every j. Therefore A=B.

Matrix of a Linear Transformation in Bases
Let T:VW be a linear transformation, let B be an ordered basis of V, and let C be an ordered basis of W. The matrix of T from the B-basis to the C-basis, denoted [T]CB, is the matrix satisfying [T(v)]C=[T]CB[v]B for every vV, where [v]B and [T(v)]C are coordinate vectors in bases. The arrow records that the matrix takes B-coordinates as input and returns C-coordinates as output.
Matrix Addition and Scalar Multiplication
Let A,BMr×c() and λ. We define: (A+B)i,j:=Ai,j+Bi,j,(λA)i,j:=λAi,j. We also define AB:=A+(1)B.
Identity Matrix
For n1, the n×n identity matrix is: (In)i,j:={1i=j0ij.

Block Matrices

Block Matrix
A block matrix is a matrix written as a rectangular array of smaller matrices, called blocks. For example, if the block sizes fit together, then M=[ABCD] denotes the ordinary matrix obtained by placing A and B side by side, placing C and D side by side, and then stacking the two block rows.

The block notation is valid only when blocks in the same block row have the same number of rows, and blocks in the same block column have the same number of columns.

Two by Two Block Matrix Multiplication
Suppose the block sizes are compatible for matrix multiplication. Then [ABCD][EFGH]=[AE+BGAF+BHCE+DGCF+DH].
Each entry of the product is computed by taking a row of the first matrix and a column of the second matrix. If the rows and columns are grouped into blocks, the same calculation groups the resulting sums into blocks. The upper-left block receives the contributions AE and BG, the upper-right block receives AF and BH, the lower-left block receives CE and DG, and the lower-right block receives CF and DH. This is exactly the displayed block product.
Matrix Transpose
Let AMr×c(). The transpose of A is the matrix ATMc×r() defined by: (AT)i,j:=Aj,i.
Reflection Matrix About a Line Through the Origin
Let θ, and let Lθ2 be the line through the origin in the direction (cos(θ),sin(θ)). The matrix which reflects vectors across Lθ is: [cos(2θ)sin(2θ)sin(2θ)cos(2θ)].

Let v=(x,y)2, and write v in polar form: v=(x,y)=(rcos(α),rsin(α)). Suppose v is the reflection of v across the line Lθ, and write: v=(rcos(γ),rsin(γ)).

Let ϕ=θα. Since reflection across Lθ places v on the other side of the line by the same angle, γ=θ+ϕ, and hence: γ=θ+(θα)=2θα. Therefore: v=(rcos(2θα),rsin(2θα)).

Using the trigonometric angle addition formulas: v=(rcos(2θ+(α)),rsin(2θ+(α)))=(r(cos(2θ)cos(α)sin(2θ)sin(α)),r(sin(2θ)cos(α)+cos(2θ)sin(α))). By the fact that sine is odd and cosine is even: v=(r(cos(2θ)cos(α)+sin(2θ)sin(α)),r(sin(2θ)cos(α)cos(2θ)sin(α)))=(cos(2θ)rcos(α)+sin(2θ)rsin(α),sin(2θ)rcos(α)cos(2θ)rsin(α)).

Since x=rcos(α) and y=rsin(α), we have: v=(cos(2θ)x+sin(2θ)y,sin(2θ)xcos(2θ)y). Equivalently, using matrix multiplication: [xy]=[cos(2θ)sin(2θ)sin(2θ)cos(2θ)][xy]. Therefore the reflection matrix about Lθ is: [cos(2θ)sin(2θ)sin(2θ)cos(2θ)].

Two Dimensional Rotation Matrix
For θ, the matrix which rotates vectors in 2 counterclockwise by angle θ is: Rθ=[cos(θ)sin(θ)sin(θ)cos(θ)].
Composition of Two Reflections is a Rotation
Let Lα and Lβ be lines through the origin in 2, where Lα has direction (cos(α),sin(α)), Lβ has direction (cos(β),sin(β)), and θ=βα. Reflecting first across Lα, and then across Lβ, is the same as rotating by 2θ: SβSα=R2θ.

By the reflection matrix formula, the reflection matrices across Lα and Lβ are: Sα=[cos(2α)sin(2α)sin(2α)cos(2α)],Sβ=[cos(2β)sin(2β)sin(2β)cos(2β)].

Using matrix multiplication: SβSα=[cos(2β)sin(2β)sin(2β)cos(2β)][cos(2α)sin(2α)sin(2α)cos(2α)]=[cos(2β)cos(2α)+sin(2β)sin(2α)cos(2β)sin(2α)sin(2β)cos(2α)sin(2β)cos(2α)cos(2β)sin(2α)sin(2β)sin(2α)+cos(2β)cos(2α)].

By the trigonometric angle addition formulas, this simplifies to: SβSα=[cos(2β2α)sin(2β2α)sin(2β2α)cos(2β2α)]=[cos(2θ)sin(2θ)sin(2θ)cos(2θ)]. By the definition of the two dimensional rotation matrix, this is R2θ.

Composition of Two Plane Reflections is a Rotation
Let P1,P23 be planes through the origin, and suppose that A:=P1P2 is a line. Let A be the plane perpendicular to A. Then P1A and P2A are lines through the origin in A. If the oriented angle from P1A to P2A is θ, then reflecting first across P1, and then across P2, is the same as rotating around the axis A by 2θ.

Every vector x3 can be written uniquely as x=xA+x, where xAA and xA.

Since AP1 and AP2, reflection across either plane fixes every vector in A. Thus the xA component is unchanged by both reflections.

It remains to understand what happens to x. Inside the plane A, the intersections L1:=P1A,L2:=P2A are lines through the origin. Reflection across P1 restricts on A to reflection across L1, and reflection across P2 restricts on A to reflection across L2.

By the two-dimensional composition of reflections theorem, reflecting first across L1, and then across L2, rotates A by 2θ. Therefore the composition fixes the component along A and rotates the perpendicular component by 2θ, which is exactly rotation around the axis A by 2θ.

Vector as a Column Matrix
For each n1, define a function to_col:nMn×1() by: to_col((a1,,an)):=[a1an].
Vector as a Row Matrix
For each n1, define a function to_row:nM1×n() by: to_row((a1,,an)):=[a1an].

The notation to_col and to_row is overloaded across dimensions: if xn, then the dimension n determines which function is being used and what size matrix is produced.

Matrix Functions
Suppose that 𝐌Mr,c(T) and that f:TS is a function, then we augment this function to 𝐟:Mr,c(T)Mr,c(S) as follows: 𝐟([m1,1m1,2m1,cMm2,1m2,2m2,cMmrM,1mrM,2mrM,cM])=[f(m1,1)f(m1,2)f(m1,cM)f(m2,1)f(m2,2)f(m2,cM)f(mrM,1)f(mrM,2)f(mrM,cM)]
Packing Function
Suppose that x, then we define the following function pack:M1×1() as pack(x):=[x]
Unpacking Function
We define unpack:M1×1() by: unpack([x]):=x.
The Packing and Unpacking Functions are Inverses
TODO
One by one Matrices are Isomorphic to R
and M1,1() are isomorphic.
Row and Column Matrices are Isomorphic to n
Suppose that n1, then M1,n() and Mn,1() are isomorphic to n
Nested Matrix Multiplication Unpacking

Given two matrices AMa×b()BMc×d() then product is equal to

unpack([A1,B|,1A1,B|,2A1,B|,dA2,B|,1A2,B|,2A2,B|,dAa,B|,1Aa,B|,2Aa,B|,d])

With Ai, B|,j and unpack is being used as a matrix function

Matrix Multiplication with Rows and Columns

Suppose that A,B are matrices, and that AR,BC are their row matrix and column matrix representations, then

unpack(ARBC)=AB where unpack is being used as a matrix function
column matrix multiplication
suppose that C=[c1c2ck] is a column matrix and MMn×k(), then
MC=i=1kciM|,i}Mn×1()

By the definition of matrix multiplication the answer should be equal to

[m1,1m1,2m1,km2,1m2,2,m2,kmj,1mj,2mj,k] [c1c2ck] =[c1m1,1+c2m1,2++ckm1,kc1m2,1+c2m2,2++ckm2,kc1mj,1+c2mj,2++ckmj,k] =c1M|,1}+c2M|,2}++ckM|,k}Mn×1()
unit column matrix
ei is a column matrix who's i-th row is 1 and the rest are 0
column extraction
suppose MMj×k then the n-th column of M is given by M multiplied by en (where 1nk)

Due to the way that column matrix multiplication works, we can see that Men=i=0k(en)iM|,i}=M|,n} this is clear since only the n-th component of en is 1 and the rest are 0.

Complex Transpose
We denote the complex transpose of a matrix as A:=AT
Hermitian Matrix
A matrix AMn×n() is said to be hermitian diff A=A where we've applied the complex-transpose

Note that the a Hermitian Matrix is also called symmetric or self-adjoint.

Normal
We say that a matrix is normal if AA=AA