Hyperreality

Basis and dimension

Properties1
tags—

5 min read

BOLD IT ALL OUT!
In many machine learning algorithms, we need to represent feature vectors as a linear combination of the set of linearly independent vectors or orthogonal vectors. These vectors form a basis for the feature space. More relevance in dimensional reduction and dictionary learning algorithms.

Basis

Let 𝑉(𝐹) be a vector space. Then a basis for 𝑉 is a set of linearly independent vectors in 𝑉 which span the vector space 𝑉.

  • the space 𝑉 is finite dimensional if it has finite basis
  • a vector in basis is called a basis vector

𝐵={𝑣1,…,𝑣𝑛} is a basis of an n-dimensional vector space 𝑉 if:

  1. {𝑣1,….,𝑣𝑛} is linearly independent
  2. for any vector 𝑣∈𝑉 we have 𝑣=𝛼1𝑣1+…+𝛼𝑛𝑣𝑛:𝛼1,…,𝛼𝑛∈𝐹
    Note that the zero vector can NOT be a basis vector because then the second condition would be trivial.

examples:

  • if 𝑉=ℝ2(ℝ) then {(1,0),(0,1)} forms the basis as we can take an arbitrary vector (𝛼,𝛽)∈ℝ,(𝛼,𝛽)=𝛼(1,0)+𝛽(0,1). Thus we can see that the basis is linearly independent (you can also check via the Gaussian elimination method, it’s trivial here), and also we can represent any vector in ℝ2 using the vectors in the basis. Hence we can say that it “spans” the vector space 𝑉.
  • now when 𝑉=ℝ3(ℝ) the basis is {(1,0,0),(0,1,0),(0,0,1)}
  • generally, when 𝑉=ℝ𝑛(ℝ), basis is {(1,0,…,0),(0,1,…,0),…,(0,0,…,1)}
  • 𝑉={𝑀2×2|set of all 2x2 real matrices}, then 𝑉(ℝ) forms a vector space. Here the basis would be {(1000),(0010),(0100),(0001)}
  • 𝑉={𝑀2×2|set of all 2x2 real symmetric matrices} forms a vector space 𝑉(ℝ). Here the basis would be {(1000),(0110),(0001)}

Dimension

The number of vectors in the basis of 𝑉(𝐹) is called the dimension of the vector space 𝑉(𝐹).
We can have multiple bases for a vector space, but each would have the same number of vectors, that is the dimension.
examples:

  • the dimension of the vector space ℝ2(ℝ) is 2
  • the dimension of the vector space ℝ𝑛(ℝ) is n
  • the dimension of the vector space 𝑀𝑚×𝑛(ℝ) is m x n
  • dimension of the vector space 𝑃𝑛(ℝ) is n+1
  • dimension of vector space 𝑃 is ∞, where 𝑃 is the set of all polynomials with coefficients from ℝ.

some results:

  1. If 𝐵={𝑣1,𝑣2,…,𝑣𝑛} be a basis of a given vector space 𝑉(𝐹) and 𝑆 be a subset of vectors having more than n vectors, then 𝑆 is linearly dependent. It’s because if 𝐵 is the basis, it spans the entire vector space 𝑉 and every single vector in a subset 𝑆 can be built using a linear combination of the n vectors in 𝐵.
  2. If 𝑆={𝑣1,…,𝑣𝑘} is a linearly independent subset of an n-dimensional vector space 𝑉(𝐹) where k < n, then 𝑆 can be extended to a basis of 𝑉(𝐹). This one is also simple when you think about how there can be multiple bases for a single vector space, and how all of them would have the same number of vectors.
  3. If 𝑉(𝐹) is a finite dimensional vector space (finite basis), then any two bases of 𝑉 have the same number of vectors, the dimension. Ex. in ℝ3 we have 𝐵1={(1,0,0),(0,1,0),(0,0,1)} and 𝐵2={(1,0,0),(1,1,0),(1,1,1)}
  4. If 𝑉(𝐹) is a finite dimensional vector space, and 𝑆1 and 𝑆2 are two subspaces of 𝑉, then dim(𝑆1)+dim(𝑆2)=dim(𝑆1+𝑆2)+dim(𝑆1∩𝑆2)

example:

  • find the basis and dimension for 𝑆={(𝑥1,𝑥2,𝑥3)∈ℝ3|𝑥1+𝑥2−𝑥3=0}, where 𝑆 is a subspace of 𝑉(𝐹).

    • 𝑆={(𝑥1,𝑥2,𝑥3)∈ℝ3|𝑥1+𝑥2=𝑥3} and thus 𝑆={(𝑥1,𝑥2,𝑥1+𝑥2)∈ℝ3}∵𝑥1+𝑥2=𝑥3
    • now 𝑆={𝑥1(1,0,1)+𝑥2(0,1,1)} and hence 𝐵={(1,0,1),(0,1,1)}.
    • dim(𝑆)=2

  • find the basis and dimension for 𝑊={(𝑥1,𝑥2,𝑥3)∈ℝ3|𝑥1=𝑥2=𝑥3} where 𝑆 is a subspace of 𝑉(𝐹)

    • 𝑊={(𝑥1,𝑥1,𝑥1)∈ℝ3}∵𝑥1=𝑥2=𝑥3 which is 𝑆={𝑥1(1,1,1)} and hence 𝐵={(1,1,1)}
    • dim(𝑊)=1

  • find the basis and dimension for 𝑆∩𝑊

    • we have 𝑆∩𝑊={(𝑥1,𝑥2,𝑥3)∈ℝ3|𝑥1+𝑥2−𝑥3=0∧𝑥1=𝑥2=𝑥3}
    • 𝑆∩𝑊={(𝑥1,𝑥2,𝑥1+𝑥2)|𝑥1=𝑥2=𝑥3}∵𝑥3=𝑥1+𝑥2
    • 𝑆∩𝑊={(𝑥1,𝑥1,2𝑥1)}∵𝑥1=𝑥2=𝑥3 where we can’t have a basis like (1,1,2) since all three are equal. Therefore the only valid element is (0,0,0) which can NOT be a basis element (see the basis section).
    • ∴𝐵=∅
    • dim(𝑆∩𝑊)=0 as there are no elements in the basis.

Trick (sort of): when you are trying to find the basis vector and you have an equation like (𝑥,2𝑥+3𝑧+𝑧) just do 𝑥(𝑎,𝑏,𝑐)+𝑦(𝑑,𝑒,𝑓)+𝑧(𝑔,ℎ,𝑖) where the letters inside brackets are nothing but the appearance of the attached letter; ex. in (𝑥,2𝑥+3𝑧,𝑧), 𝑥 appears 1, 2, and 0 times respectively. Similarly 𝑦 appears 0, 0, 0 times and z appears 0, 3, 1 times.

Graph View

Backlinks