Vector Spaces

BOLD IT ALL OUT!
A vector space over a field is a set on which two operations, vector addition (+) and scalar multiplication¹ () are defined such that:

1. forms an abelian group².
2. The operation is defined between scalars and vectors such that
3. and (associativity)
4. and
5. and
6. unitary law: such that
   If all of these properties hold, we say that is a vector space.
   some examples:

• consider the vector space over the field of real numbers.
  • Testing the first property: let’s take two vectors, in . Let’s perform addition here: we get , thus we can say that under addition is “closed”. Also, . Thus the operation is also commutative in nature. Similarly we can also verify that associativity holds such that for any three vectors we can say that . Also, an identity element exists which is . Lastly, we can have the additive inverse which enables us to have . Thus this is a group and an abelian group because of the commutative property.
  • Testing the second property: let ; then . Thus the second property holds.
  • Testing the third property: let . Then
  • Testing the fourth property: let . Then
  • Testing the fifth property: let . Then
  • Testing the sixth property: indeed, if
    Thus, is a vector space.
• other examples include forming a vector space over ; forming a vector space over , = set of all polynomials in variable x and coefficient from ; etc.
• example: . Take n = 3, and take two vectors, . Then as n 3. Thus this is not a vector space because it’s not closed under vector addition, which is one of the fundamental properties of a group.

Euclidean space: A vector space over equipped with notions of distance, length, and angles.

Geometric interpretation of

For n = 1, 2, 3 the space has a very useful geometric interpretation.

• n = 1
  • . Thus this can be represented as a vector on the number line:
    
    In the image above, vectors and are shown.
• n = 2
  • . This can be represented in the x-y plane. This is your standard vector from high school physics.
    
  • Yes, vectors are generally represented as a column, like and not . There are multiple forms, like row-vector and column-vector form, but in machine learning the column vector form is seen more over tuples. MML suggests: “By default, we write to denote a column vector, and a row vector is denoted by , the transpose of .”
• n = 3
  • . This can be represented in a three dimensional plane.
    
• Linear independence of three vectors means that and are not collinear, and does not lie in the plane of and .

---

Footnotes

1. also known as the dot product ↩

2. is the set of vectors here. Recall that the condition for a group was that needs to be closed under the addition operation here, or that addition of two vectors produces another vector. Another condition was the identity element: there must be such that , which holds for the zero vector. The third condition was for the inverse element which also holds in addition - every vector has an additive inverse which can be added to get the zero vector, the identity element here. Note that for an abelian group we need to have the condition that for any two pairs which certainly holds as additive groups are abelian. ↩