Matrices

Matrix

A matrix is a collection of numbers that can be indexed in a particular way, this indexing mimics a two dimensional grid. The dimensions of a matrix are stated as \( r \times c \) where \( r, c \in \mathbb{ N } _ 1 \) when it has \( m \) rows and \( n \) columns.

Suppose that \( A \) is a matrix, then we can access the value at the \( i \)-th row and \( j \)-th column by writing \( A_{i , j} \), we can graphically represent this information by introducing variables for the coefficients and writing them like this:

\[ A = \begin{bmatrix} a_{1 , 1} & a_{1 , 2} & \ldots & a_{1 , n} \\ a_{2 , 1} & a_{2 , 2} & \ldots & a_{2 , n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m , 1} & a_{m , 2} & \ldots & a_{m , n} \end{bmatrix} \]

Matrices are denoted by capital letters such as \( A , B , M , N \).

Set of Matrices
Suppose that \( r, c \in \mathbb{ N } _ 1 \) then we denote the set of all \( r \times c \) matrices with cofficients from the set \( T \) as \[ M _ { r \times c } \left( T \right) \]
Row Matrix
Any matrix \( R \in M _ { 1 \times c } \left( T \right) \) for some \( c \in \mathbb{ N } _ 1 \) is known as a row matrix.
Row of a Matrix

Suppose that \( A \) is a matrix, then if we want to extract the \( i-th \) row of \( A \) we can write \( A_{i , -} \).

Note that this is a function of the form \( M _ { r \times c } \left( T \right) \rightarrow M _ {1 \times c} \left( T \right) \)
The notation \( A _ {i, - } \) intends to mimic freezing at a certain row and then sweeping over all columns.
Row Matrix Representation
Given a matrix \( A \), there is an associated row matrix representation, of the following form
\( A = \left [ \begin{matrix} \leftarrow & A_{1 , -} & \rightarrow \\ \leftarrow & A_{2 , -} & \rightarrow \\ \leftarrow & \vdots & \rightarrow \\ \leftarrow & A_{m , -} & \rightarrow \end{matrix} \right ] \)

Where the arrows are there to remind you that they expand horizontally. Also observe that this matrix is an element of \( M _ { r \times 1 } \left( M _ { 1 \times c } \left( \mathbb{ R } \right) \right) \)

Column Matrix
Any matrix \( C \in M _ { r \times 1 } \left( T \right) \) for some \( r \in \mathbb{ N } _ 1 \) is known as a row matrix.

We will denote column or row matrices with a lower case boldfaced character, like \( \mathbf{x} \)

Column of a Matrix

Suppose that \( A \) is a matrix, then if we want to extract the \( j \)-th row of \( A \) we can write \( A_{ \mid , j} \).

Note that this is a function of the form \( M _ { r \times c } \left( T \right) \rightarrow M _ {r \times 1} \left( T \right) \)
The \( \mid \) denotes that the row variable may be varied vertically, while the column index is fixed in place. If we want to represent the matrix as columns graphically, we may do so like this
Column Matrix Representation
Given a matrix \( A \), there is an associated column matrix representation, of the following form \[ A = \begin{bmatrix} \uparrow & \uparrow & \uparrow & \uparrow \\ A_{ \mid, 1} & A_{\mid , 2} & \ldots & A_{\mid, n} \\ \downarrow & \downarrow & \downarrow & \downarrow \end{bmatrix} \]

Where the arrows are there to remind you that they expand graphically expand vertically. Also observe that this matrix is an element of \( M _ { 1 \times c } \left( M _ { r \times 1 } \left( \mathbb{ R } \right) \right) \).

We can connect some of the previous definitions here to say that \( R \) is a column matrix whose entries are row matrices.
Matrix Multiplication

Suppose \( r _ A, c _ A, r _ B, c _ B \in \mathbb{ N } _ 1 \) such that \( c _ A = r _ B \) and that we have two matrices \( A, B \in M _ { r _ A \times c _ A } \left( \mathbb{ R } \right), M _ { r _ B \times c _ B } \left( \mathbb{ R } \right) \), defined as

\[ A = \begin{bmatrix} a_{1 , 1} & a_{1 , 2} & \ldots & a_{1 , c_A} \\ a_{2 , 1} & a_{2 , 2} & \ldots & a_{2 , c_A} \\ \vdots & \vdots & \ddots & \vdots \\ a_{r_A , 1} & a_{r_A , 2} & \ldots & a_{r_A , c_A} \end{bmatrix}, B = \begin{bmatrix} b_{1 , 1} & b_{1 , 2} & \ldots & b_{1 , c_B} \\ b_{2 , 1} & b_{2 , 2} & \ldots & b_{2 , c_B} \\ \vdots & \vdots & \ddots & \vdots \\ b_{r_B , 1} & b_{r_B , 2} & \ldots & b_{r_B , c_B} \end{bmatrix} \]

Then we define the matrix multiplication of \( A \) and \( B \) as the matrix \( P \in M _ { r _ A \times c _ B } \)

\[ P = \begin{bmatrix} c_{1 , 1} & c_{1 , 2} & \ldots & c_{1 , c_P} \\ c_{2 , 1} & c_{2 , 2} & \ldots & c_{2 , c_P} \\ \vdots & \vdots & \ddots & \vdots \\ c_{r_P , 1} & c_{r_P , 2} & \ldots & c_{r_P , c_P} \end{bmatrix} \]

Where:

\[ c _ { i, j } := a _ { i, 1 } b _ { 1, j } + a _ { i, 2 } b _ { 2, j } + \cdots + a _ { i, c _ A } b _ { c _ A, j } = \sum _ { k = 1 } ^ { c _ A } a _ { i, k } b _ { k, j } \]
Matrix Functions
Suppose that \( \mathbf{M} \in M _ { r, c } \left( T \right) \) and that \( f: T \rightarrow S \) is a function, then we augment this function to \( \mathbf{f}: M _ { r, c } \left( T \right) \rightarrow M _ { r, c } \left( S \right) \) as follows: \[ \mathbf{ f } \left( \begin{bmatrix} m_{1 , 1} & m_{1 , 2} & \ldots & m_{1 , c_M} \\ m_{2 , 1} & m_{2 , 2} & \ldots & m_{2 , c_M} \\ \vdots & \vdots & \ddots & \vdots \\ m_{r_M , 1} & m_{r_M , 2} & \ldots & m_{r_M , c_M} \end{bmatrix} \right) = \begin{bmatrix} f \left( m_{1 , 1} \right) & f \left( m_{1 , 2} \right) & \ldots & f \left( m_{1 , c_M} \right) \\ f \left( m_{2 , 1} \right) & f \left( m_{2 , 2} \right) & \ldots & f \left( m_{2 , c_M} \right) \\ \vdots & \vdots & \ddots & \vdots \\ f \left( m_{r_M , 1} \right) & f \left( m_{r_M , 2} \right) & \ldots & f \left( m_{r_M , c_M} \right) \end{bmatrix} \]
Packing Function
Suppose that \( x \in \mathbb{ R } \), then we define the following function \( \operatorname{ pack } : \mathbb{ R } \rightarrow M _ { 1 \times 1 } \left( \mathbb{ R } \right) \) as \[ \operatorname{ pack } \left( x \right) := \begin{bmatrix} x \\ \end{bmatrix} \]
Unpacking Function
Suppose that \( \begin{bmatrix} x \\ \end{bmatrix} \in M _ { 1 \times 1 } \left( \mathbb{ R } \right) \), then we define the following function \( \operatorname{ unpack } : \mathbb{ R } \rightarrow M _ { 1 \times 1 } \left( \mathbb{ R } \right) \) as \[ \operatorname{ unpack } \left( \begin{bmatrix} x \\ \end{bmatrix} \right) := x \]
The Packing and Unpacking Functions are Inverses
TODO
One by one Matrices are Isomorphic to R
\( \mathbb{ R } \) and \( M _ { 1, 1 } \left( \mathbb{ R } \right) \) are isomorphic.
Row and Column Matrices are Isomorphic to \( \mathbb{ R } ^ n \)
Suppose that \( n \in \mathbb{ N } _ 1 \), then \( M _ { 1, n } \left( \mathbb{ R } \right) \) and \( M _ { n, 1 } \left( \mathbb{ R } \right) \) are isomorphic to \( \mathbb{ R } ^ n \)
Nested Matrix Multiplication Unpacking

Given two matrices \( A \in M_{a \times b} \left ( \mathbb{R} \right ) B \in M_{c \times d} \left ( \mathbb{R} \right ) \) then product is equal to

\[ \mathbf{ \operatorname{ unpack } } \left( \begin{bmatrix} A_{1, -} \cdot B _ { \mid, 1 } & A_{1, -} \cdot B _ { \mid, 2 } & \ldots & A_{1, -} \cdot B _ { \mid, d } \\ A_{2, -} \cdot B _ { \mid, 1 } & A_{2, -} \cdot B _ { \mid, 2 } & \ldots & A_{2, -} \cdot B _ { \mid, d } \\ \vdots & \vdots & \ddots & \vdots \\ A_{a, -} \cdot B _ { \mid, 1 } & A_{a, -} \cdot B _ { \mid, 2 } & \ldots & A_{a, -} \cdot B _ { \mid, d } \\ \end{bmatrix} \right) \]

With \( A_{i , -} \) \( B_{\mid , j}\) and \( \mathbf{ \operatorname{ unpack } } \) is being used as a matrix function

Matrix Multiplication with Rows and Columns

Suppose that \( A, B \) are matrices, and that \( A_R, B_C \) are their row matrix and column matrix representations, then

\[ \mathbf{ \operatorname{ unpack } } \left( A_R B_C \right) = A B \] where \( \mathbf{ \operatorname{ unpack } } \) is being used as a matrix function
column matrix multiplication
suppose that \( C = \left [ \begin{matrix} c_{1} \\ c_{2} \\ \vdots \\ c_{k} \end{matrix} \right ] \) is a column matrix and \( M \in M_{n \times k} \left ( \mathbb{R} \right ) \), then
\( M C = \sum_{i = 1}^{k} c_{i} M_{\left . \right |} , i \rbrace \in M_{n \times 1} \left ( \mathbb{R} \right ) \)

By the definition of matrix multiplication the answer should be equal to

\( \left [ \begin{matrix} m_{1 , 1} & m_{1 , 2} & \ldots & m_{1 , k} \\ m_{2 , 1} & m_{2 , 2 ,} & \ldots & m_{2 , k} \\ \vdots & \vdots & \ddots & \vdots \\ m_{j , 1} & m_{j , 2} & \ldots & m_{j , k} \end{matrix} \right ] \) \( \left [ \begin{matrix} c_{1} \\ c_{2} \\ \vdots \\ c_{k} \end{matrix} \right ] \) \( = \left [ \begin{matrix} c_{1} \cdot m_{1 , 1} + c_{2} \cdot m_{1 , 2} + \ldots + c_{k} \cdot m_{1 , k} \\ c_{1} \cdot m_{2 , 1} + c_{2} \cdot m_{2 , 2} + \ldots + c_{k} \cdot m_{2 , k} \\ \vdots \\ c_{1} \cdot m_{j , 1} + c_{2} \cdot m_{j , 2} + \ldots + c_{k} \cdot m_{j , k} \end{matrix} \right ] \) \( = c_{1} M_{\left . \right |} , 1 \rbrace + c_{2} M_{\left . \right |} , 2 \rbrace + \ldots + c_{k} M_{\left . \right |} , k \rbrace \in M_{n \times 1} \left ( \mathbb{R} \right ) \)
unit column matrix
\( e_{i} \) is a column matrix who's i-th row is \( 1 \) and the rest are \( 0 \)
column extraction
suppose \( M \in M_{j \times k} \) then the \( n \)-th column of \( M \) is given by \( M \) multiplied by \( e_{n} \) (where \( 1 \le n \le k \))

Due to the way that column matrix multiplication works, we can see that \( M e_{n} = \sum_{i = 0}^{k} \left ( e_{n} \right )_{i} M_{\left . \right |} , i \rbrace = M_{\left . \right |} , n \rbrace \) this is clear since only the \( n \)-th component of \( e_{n} \) is \( 1 \) and the rest are \( 0 \).