A vector is a mathematical object that encodes length and direction. Formally, these are closed under an addition rule and a rule for multiplication by scalars - this is what we call a vector space, and it’s covered later. Scalars’ main use is for “scaling” a vector.
The study of vectors and rules to manipulate them is called linear algebra.
In general, vectors are special objects that can be added together and multiplied by scalars to produce another object of the same kind. From an abstract view, even polynomials are vectors because they satisfy those conditions. More examples include audio signals, geometrical vectors, and the type described below.
We represent vectors by a one dimensional array, referred to as a components, and is displayed in column or row form.
Geometrically, vectors represent coordinates within an n-dimensional space.
ex. , assuming
Here is the number of components of a vector, or the number of dimensions.
As an example, take n = 2. Then that represents the vector space . A vector that belongs to this vector space is where the component along the first axis is 1 while on the second axis it’s two. Similarly, a vector belonging to vector space would have components, as seen in the above example.
Vector Algebra
some operations using vectors.
Addition and Subtraction
We can add or subtract vectors if they have the same dimensions.
more generally,
Dot product
For two vectors and ,
or generally,
Keep in mind that the dot product is a scalar.
Length or magnitude of a vector
If a vector is defined as then the length or the magnitude of v is a scalar, that is:
Angle between two vectors
The angle between two vectors is given as:
Linear combination of vectors
Consider a set , then a new vector is called a linear combination of where are scalars.
As an example, let’s try to create a linear combination from the given vectors.
ex.
Linearly dependent and independent vectors
A set of vectors is linearly independent if the vector equation
which should hold only when
if not, is linearly dependent and the vectors in have a linear relationship between them.
ex, in
another ex. in
- In a set of more than vectors is linearly dependent.
- Any set of vectors containing the zero vector is linearly dependent.
Orthogonal vectors
We say that a set of vectors are mutually (pairwise) orthogonal if for (dot product is zero).
ex. in
is a set of vectors, and
thus they are orthogonal.
Orthonormal vectors
A set of orthogonal vectors is orthonormal if each vector has length 1.
- a set of orthogonal vectors is linearly independent.
Feature vectors in machine learning
Recall that features are nothing but the predictors or attributes of a data set.
some basic vector operations using numpy
import numpy as np
v_1 = np.array([1, -1, 2])
v_2 = np.array([2, 4, 8])
# sum of the vectors
print(v_1 + v_2)
# length of v_1
print(np.linalg.norm(v_1))
# dot product
print(np.dot(v_1, v_2))