Chapter 3: Vectors

Topics: vectors and linear combinations, norms, dot product, projecting onto a single vector

Problems below are collected from past exams; each links back to its full exam. Solutions are in the dropdowns.

Problems


FA25 MT1 · Problem 4: Mission Impossible 12 pts 🎥 Walkthrough

From FA25 MT1

a)

6 pts Suppose \(\vec u, \vec v \in \mathbb{R}^n\) are non-zero vectors, and suppose that

$$ | \vec u \cdot \vec v | = \lVert \vec u \rVert \lVert \vec v \rVert $$

For each statement below, determine whether it is impossible, possible, or guaranteed to be true, given the above assumptions. Select exactly one option from each row. The first statement has been done for you as an example.

 statementimpossible?possible?guaranteed?
\(i\)\(\lVert \vec u \rVert = 5\)
\(ii\)\(\vec u\) and \(\vec v\) are orthogonal
\(iii\)\(\lVert \vec u - \vec v \rVert = 0\)
\(iv\)\(\vec u\) and \(\vec v\) span a 1-dimensional subspace of \(\mathbb{R}^n\)
\(v\)\(\vec u\) and \(\vec v\) span a 2-dimensional subspace of \(\mathbb{R}^n\)
\(vi\)\(\lVert \vec u + \vec v \rVert = \lVert \vec u \rVert + \lVert \vec v \rVert\)
Solution

Remember that for any two vectors \(\vec u\) and \(\vec v\),

$$ \vec u \cdot \vec v = \lVert \vec u \rVert \lVert \vec v \rVert \cos \theta $$

The fact that we’re told that

$$ | \vec u \cdot \vec v | = \lVert \vec u \rVert \lVert \vec v \rVert $$

tells us that \(\cos \theta = 1\) or \(\cos \theta = -1\), which means that the angle between \(\vec u\) and \(\vec v\) is \(0^\circ\) or \(180^\circ\), which means that \(\vec u\) and \(\vec v\) are scalar multiples of each other. (They may point in the same or opposite directions.) This is the key insight to assessing each of the statements.

b)

6 pts Suppose \(\vec w, \vec z \in \mathbb{R}^n\). Given that \(\lVert \vec w \rVert = \lVert \vec z \rVert = \lVert \vec w - \vec z \rVert = 1\), find \(\lVert \vec w + \vec z \rVert\). Show your work, and \(\boxed{\text{circle}}\) your final answer, which should be a number with no variables.

Solution

\(\lVert \vec w + \vec z \rVert = \sqrt{3}\).

We’re asked to find \(\lVert \vec w + \vec z \rVert\). To do so, let’s expand out \(\lVert \vec w + \vec z \rVert^2\) as we’ve done in the past, and see how to utilize what we were given.

$$ \begin{align*} \lVert \vec w + \vec z \rVert^2 &= (\vec w + \vec z) \cdot (\vec w + \vec z) \\\\ &= \vec w \cdot \vec w + 2 \vec w \cdot \vec z + \vec z \cdot \vec z \\\\ &= \lVert \vec w \rVert^2 + 2 \vec w \cdot \vec z + \lVert \vec z \rVert^2 \\\\ &= 1 + 2 \vec w \cdot \vec z + 1 \\\\ &= 2 + 2 \vec w \cdot \vec z \end{align*} $$

Above, we’ve plugged in \(\lVert \vec w \rVert^2 = 1\) and \(\lVert \vec z \rVert^2 = 1\). We need to know \(\vec w \cdot \vec z\), which we don’t yet know.

But, we have enough information to find it, if we expand out \(\lVert \vec w - \vec z \rVert^2\), which we were told is equal to 1.

$$ \begin{align*} \lVert \vec w - \vec z \rVert^2 &= (\vec w - \vec z) \cdot (\vec w - \vec z) \\\\ 1 &= \vec w \cdot \vec w - 2 \vec w \cdot \vec z + \vec z \cdot \vec z \\\\ 1 &= \lVert \vec w \rVert^2 - 2 \vec w \cdot \vec z + \lVert \vec z \rVert^2 \\\\ 1 &= 1 - 2 \vec w \cdot \vec z + 1 \\\\ 1 &= 2 - 2 \vec w \cdot \vec z \end{align*} $$

Solving the above gives us \(\vec w \cdot \vec z = \frac{1}{2}\). This gives

$$ \lVert \vec w + \vec z \rVert^2 = 2 + 2 \vec w \cdot \vec z = 2 + 2 \cdot \frac{1}{2} = 3 $$

And so,

$$ \lVert \vec w + \vec z \rVert = \sqrt{3} $$

FA25 MT1 · Problem 8: Worst-Case Scenario 8 pts 🎥 Walkthrough

From FA25 MT1

Suppose \(a, b, c, d, e\) are positive real numbers. Find the largest real number \(T\) such that it’s guaranteed that

$$ (a + b + c + d + e) \left( \frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d} + \frac{1}{e} \right) \geq T $$

Think of \(T\) as the “best possible lower bound”. For instance, we know that the expression on the left-hand side above must be greater than or equal to 0, since \(a, b, c, d, e\) are all positive, but \(T = 0\) is not the answer, since there’s a larger value of \(T\) that also guarantees the inequality holds.

Show your work, and \(\boxed{\text{circle}}\) your final answer, which should be a number with no variables.

Hint: Use the Cauchy-Schwarz inequality.

Solution

\(T = 25\).

Recall, the Cauchy-Schwarz inequality states that for any two vectors \(\vec u\) and \(\vec v\),

$$ |\vec u \cdot \vec v| \leq \lVert \vec u \rVert \lVert \vec v \rVert $$

Let’s define two vectors \(\vec u\) and \(\vec v\) and then apply the Cauchy-Schwarz inequality to them.

$$ \vec u = \begin{bmatrix} \sqrt{a} \\\\ \sqrt{b} \\\\ \sqrt{c} \\\\ \sqrt{d} \\\\ \sqrt{e} \end{bmatrix}, \quad \vec v = \begin{bmatrix} \frac{1}{\sqrt{a}} \\\\ \frac{1}{\sqrt{b}} \\\\ \frac{1}{\sqrt{c}} \\\\ \frac{1}{\sqrt{d}} \\\\ \frac{1}{\sqrt{e}} \end{bmatrix} $$

Let’s compute the three quantities involved in the inequality.

  • \(\lVert \vec u \rVert = \sqrt{a + b + c + d + e}\)

  • \(\lVert \vec v \rVert = \sqrt{\frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d} + \frac{1}{e}}\)

  • \(|\vec u \cdot \vec v| = |\sqrt{a} \cdot \frac{1}{\sqrt{a}} + \sqrt{b} \cdot \frac{1}{\sqrt{b}} + \sqrt{c} \cdot \frac{1}{\sqrt{c}} + \sqrt{d} \cdot \frac{1}{\sqrt{d}} + \sqrt{e} \cdot \frac{1}{\sqrt{e}}| = 5\)

So, we have that

$$ 5 \leq \sqrt{a + b + c + d + e} \cdot \sqrt{\frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d} + \frac{1}{e}} $$

Squaring both sides of the inequality gives us

$$ 25 \leq (a + b + c + d + e) \left( \frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d} + \frac{1}{e} \right) $$

This means that for any positive values of \(a, b, c, d, e\), it’s impossible for \((a + b + c + d + e) \left( \frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d} + \frac{1}{e} \right)\) to be less than 25. Finding a value equal to 25 is doable if we set \(a = b = c = d = e = 1\). So, \(T = 25\) is the largest possible value of \(T\) that guarantees the inequality holds.


FA25 Final · Problem 3 16 pts MT1 Redemption

From FA25 Final

Consider the vectors \(\vec u = \begin{bmatrix} 3 \\ 3 \\ 6 \end{bmatrix}\) and \(\vec v = \begin{bmatrix} 1 \\ 0 \\ c \end{bmatrix}\), where \(c \in \mathbb{R}\) is some constant.

In parts a) and b), if there are multiple possible values of \(c\), give just one.

a)

3 pts Suppose \(\vec u\) and \(\vec v\) are orthogonal. Find \(c\). Give your answer as a number with no variables.

\(c = \_\_\_\_\_\_\)

Solution

Since \(\vec u\) and \(\vec v\) are orthogonal, their dot product is 0.

$$ \begin{bmatrix} 3 \\\\ 3 \\\\ 6 \end{bmatrix} \cdot \begin{bmatrix} 1 \\\\ 0 \\\\ c \end{bmatrix} = 0 $$
$$ 3 + 0 + 6c = 0 $$
$$ 6c = -3 $$
$$ c = -1/2 $$
b)

3 pts Suppose \(\lVert \vec v \rVert = 4\). Find \(c\). Give your answer as a number with no variables.

\(c = \_\_\_\_\_\_\)

Solution

Since \(\lVert \vec v \rVert = 4\), we have

$$ \sqrt{1^2 + 0^2 + c^2} = 4 $$
$$ 1 + c^2 = 16 $$
$$ c^2 = 15 $$
$$ c = \sqrt{15} $$
c)

6 pts Suppose the projection of \(\vec v\) onto \(\vec u\) is \(\begin{bmatrix} 1.5 \\ 1.5 \\ 3 \end{bmatrix}\). What is the value of \(c\)? Select one of the answers below, then justify your answer in the box provided.

  1. Answer:

    \(1/2\) \(3/2\) \(2\) \(4\) \(6\) \(6 + \sqrt{41}\) \(27\)
  2. Justify your answer in the box below.

    Solution
    \(1/2\) \(3/2\) \(2\) \(4\) \(6\) \(6 + \sqrt{41}\) \(27\)

    The projection of \(\vec v\) onto \(\vec u\) is given by

    $$ \vec p = \frac{\vec v \cdot \vec u}{\vec u \cdot \vec u} \vec u $$

    Since we’re told that \(\vec p = \begin{bmatrix} 1.5 \\ 1.5 \\ 3 \end{bmatrix}\), this means that \(p = \frac{1}{2} \begin{bmatrix} 3 \\ 3 \\ 6 \end{bmatrix} = \frac{1}{2} \vec u\). So,

    $$ \frac{\vec v \cdot \vec u}{\vec u \cdot \vec u} = \frac{1}{2} $$

    Substituting in \(\vec v = \begin{bmatrix} 1 \\ 0 \\ c \end{bmatrix}\) and \(\vec u = \begin{bmatrix} 3 \\ 3 \\ 6 \end{bmatrix}\) gives us

    $$ \frac{1 \cdot 3 + 0 \cdot 3 + c \cdot 6}{3^2 + 3^2 + 6^2} = \frac{1}{2} \implies \frac{3 + 6c}{54} = \frac{1}{2} \implies 3 + 6c = 27 \implies \boxed{c = 4} $$

Recall from the previous page that \(\vec u = \begin{bmatrix} 3 \\ 3 \\ 6 \end{bmatrix}\) and \(\vec v = \begin{bmatrix} 1 \\ 0 \\ c \end{bmatrix}\), where \(c \in \mathbb{R}\) is some constant.

d)

4 pts Suppose \(\text{span}(\lbrace\vec u, \vec v\rbrace)\) is the plane \(2x + 4y - 3z = 0\). Find \(c\). Show your work, and \(\boxed{\text{circle}}\) your final answer, which should be a number with no variables. Hint: While you could compute the cross product, there is no need to — there is a much quicker solution.

Solution

One way to find the equation of the plane \(ax + by + cz = 0\) spanned by \(\vec u\) and \(\vec v\) in \(\mathbb{R}^3\) is to take the cross product of the two vectors, and setting \(a\) to the first component of the cross product, \(b\) to the second component, and \(c\) to the third component. We could compute the cross product in terms of \(c\), and solve for where it is equal to \(\begin{bmatrix} 2 \\ 4 \\ -3 \end{bmatrix}\).

But this is overly complicated, and there’s an easier solution: if this plane is spanned by \(\vec u\) and \(\vec v\), then \(\vec v\) needs to satisfy the equation of the plane, which is \(2x + 4y - 3z = 0\).

Substituting in \(\vec v = \begin{bmatrix} 1 \\ 0 \\ c \end{bmatrix}\) gives us

$$ 2 \cdot 1 + 4 \cdot 0 - 3 \cdot c = 0 \implies 2 - 3c = 0 \implies \boxed{c = 2/3} $$

WN26 MT1 · Problem 4 12 pts

From WN26 MT1

Suppose \(\vec u = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}\) and \(\vec u + \vec v + \vec w = \begin{bmatrix} 4 \\ 0 \\ 0 \end{bmatrix}\). Assume that none of \(\vec u\), \(\vec v\), or \(\vec w\) are the zero vector, \(\vec 0\).

For each statement below, identify whether it is impossible, possible, or guaranteed, and provide a brief explanation in the box provided.

a)

4 pts \(\vec u\) and \(\vec v\) are orthogonal.

Impossible Possible Guaranteed
Solution
Impossible Possible Guaranteed

This is possible.

There is nothing stopping \(\vec u\) and \(\vec v\) from being orthogonal. For example, let \(\vec v = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}\). Then, \(\vec u \cdot \vec v = 0 \cdot 0 + 0 \cdot 1 + 0 \cdot 0 = 0\), so \(\vec u\) and \(\vec v\) are orthogonal, and we can still find a \(\vec w\) such that \(\vec u + \vec v + \vec w = \begin{bmatrix} 4 \\ 0 \\ 0 \end{bmatrix}\). For example, let \(\vec w = \begin{bmatrix} 4 \\ 0 \\ 0 \end{bmatrix} - \vec u - \vec v = \begin{bmatrix} 3 \\ -1 \\ 0 \end{bmatrix}\).

However, it’s not guaranteed: \(\vec v = \begin{bmatrix} 2 \\ 0 \\ 0 \end{bmatrix}\) and \(\vec w = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}\) satisfy \(\vec u + \vec v + \vec w = \begin{bmatrix} 4 \\ 0 \\ 0 \end{bmatrix}\), but \(\vec u\) and \(\vec v\) are not orthogonal.

So, it is possible for \(\vec u\) and \(\vec v\) to be orthogonal.

b)

4 pts The set \(\lbrace\vec u, \vec v, \vec w\rbrace\) is linearly dependent.

Impossible Possible Guaranteed
Solution
Impossible Possible Guaranteed

This is guaranteed.

$$ \vec u + \vec v + \vec w = \begin{bmatrix} 4 \\\\ 0 \\\\ 0 \end{bmatrix} $$

Since \(\vec u = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}\), we can subtract \(4 \vec u\) from both sides to get

$$ \vec u + \vec v + \vec w - 4 \vec u = \vec w - 3 \vec u = \begin{bmatrix} 4 \\\\ 0 \\\\ 0 \end{bmatrix} - 4 \begin{bmatrix} 1 \\\\ 0 \\\\ 0 \end{bmatrix} $$

Or, equivalently,

$$ - 3 \vec u + \vec v + \vec w = \vec 0 $$

This is a non-trivial linear combination of \(\vec u\), \(\vec v\), and \(\vec w\) that equals the zero vector, so the set \(\lbrace\vec u, \vec v, \vec w\rbrace\) is linearly dependent. Equivalently, we could say \(\vec w = 3 \vec u - \vec v\), which means \(\vec w\) is a linear combination of \(\vec u\) and \(\vec v\), which also means the set is linearly dependent.

c)

4 pts \(\vec u\), \(\vec v\), and \(\vec w\) all have the same norm (length).

Impossible Possible Guaranteed
Solution
Impossible Possible Guaranteed

This is impossible.

Recall that the triangle inequality states that for any two vectors \(\vec v\) and \(\vec w\),

$$ \lVert \vec v + \vec w \rVert \leq \lVert \vec v \rVert + \lVert \vec w \rVert $$

We know that \(\lVert \vec u \rVert = 1\), so in order for the statement to be possible, we’d need both \(\lVert \vec v \rVert = 1\) and \(\lVert \vec w \rVert = 1\). But, \(\vec v + \vec w = \begin{bmatrix} 3 \\ 0 \\ 0 \end{bmatrix}\), so \(\lVert \vec v + \vec w \rVert = \sqrt{3^2 + 0^2 + 0^2} = \sqrt{9} = 3\). In the triangle inequality, this would mean

$$ \lVert \vec v + \vec w \rVert \leq \lVert \vec v \rVert + \lVert \vec w \rVert \implies 3 \leq 2 $$

This is a contradiction, so it is impossible for both \(\vec v\) and \(\vec w\) to have a norm of 1, and therefore impossible for all three vectors to have the same norm.


WN26 Final · Problem 3 9 pts MT1 Redemption

From WN26 Final

a)

5 pts Suppose \(\vec a = \begin{bmatrix} 0 \\ 3 \\ 6 \end{bmatrix}\) and that \(\vec b\) is another vector in \(\mathbb{R}^3\) such that:

  • \(\vec a\) and \(\vec b\) are orthogonal, and

  • the plane spanned by \(\vec a\) and \(\vec b\) is

$$ 4x - 2y + z = 0 $$

There are infinitely many possible vectors \(\vec b\) that satisfy the given conditions. State one of them. Show your work, and write your final answer in the box provided. Your answer should be a vector with no variables.

$$ \text{one possible }\vec b = \_\_\_\_\_\_ $$
Solution

Let \(\vec b = \begin{bmatrix} x \\ y \\ z \end{bmatrix}\).

Since \(\vec b\) lies in the given plane,

$$ 4x-2y+z=0 $$

Since \(\vec a\) and \(\vec b\) are orthogonal,

$$ \vec a \cdot \vec b = 3y+6z=0 $$

The second equation gives \(y=-2z\). Plugging this into the first equation gives

$$ 4x+4z+z=0 \implies x=-\frac{5}{4}z $$

There are infinitely many solutions for \(x\), \(y\), and \(z\); they all lie on a line. To state one, let’s just fix a value of \(z\). Arbitrarily choosing \(z = 4\) gives

$$ \vec b = \boxed{\begin{bmatrix}-5\\\\-8\\\\4\end{bmatrix}} $$

Here’s another solution: really, the question is asking for a vector that is orthogonal to both \(\vec a\) and \(\begin{bmatrix} 4 \\ -2 \\ 1 \end{bmatrix}\). Such a vector would be orthogonal to \(\vec a\) and would lie in the plane \(4x-2y+z=0\). So, all we need to do is take the cross product of \(\vec a\) and \(\begin{bmatrix} 4 \\ -2 \\ 1 \end{bmatrix}\).

$$ \underbrace{\begin{bmatrix} 0 \\\\ 3 \\\\ 6 \end{bmatrix}}_{\vec a} \times \begin{bmatrix} 4 \\\\ -2 \\\\ 1 \end{bmatrix} = \begin{bmatrix} 3 \cdot 1 - 6 \cdot (-2) \\\\ 6 \cdot 4 - 0 \cdot 1 \\\\ 0 \cdot (-2) - 3 \cdot 4 \end{bmatrix} = \boxed{\begin{bmatrix} 15 \\\\ 24 \\\\ -12 \end{bmatrix}} $$

Note that this is just \(-3\) times the vector we found above. Indeed, any scalar multiple of \(\begin{bmatrix} -5 \\ -8 \\ 4 \end{bmatrix}\) is also a solution.

b)

4 pts This part is unrelated to the previous part. Suppose \(\vec u, \vec v \in \mathbb{R}^n\), and that:

  • \(\vec u\) is a unit vector,

  • \(\cos(\theta) = 2/3\), where \(\theta\) is the angle between \(\vec u\) and \(\vec v\),

  • the projection of \(\vec v\) onto \(\vec u\) is \(6 \vec u\).

What is the value of \(\lVert \vec v \rVert\)?

\(1\) \(3\) \(4\) \(6\) \(9\)
Solution
\(1\) \(3\) \(4\) \(6\) \(9\)

Since \(\vec u\) is a unit vector,

$$ \vec p = \frac{\vec v \cdot \vec u}{\vec u \cdot \vec u} \vec u = (\vec v \cdot \vec u)\vec u $$

But this projection is also \(6 \vec u\), so

$$ \vec u \cdot \vec v = 6 $$

Now, let’s use the fact that \(\cos(\theta) = 2/3\), where \(\theta\) is the angle between \(\vec u\) and \(\vec v\), and plug in the values we know.

$$ \begin{align*} \cos \theta &= \frac{\vec u \cdot \vec v}{\lVert \vec u \rVert \lVert \vec v \rVert} \\\\ \frac{2}{3} &= \frac{6}{1 \cdot \lVert \vec v \rVert} \\\\ \lVert \vec v \rVert &= 9 \end{align*} $$

So, \(\boxed{\lVert \vec v \rVert = 9}\).


SP26 MT1 · Problem 4 8 pts

From SP26 MT1

Let \(\vec u,\vec v\in\mathbb R^n\) be vectors satisfying

$$ \|\vec v\|=5,\qquad \|\vec u+\vec v\|=10,\qquad \|\vec u-\vec v\|=6 $$

Find \(\lVert \vec u \rVert^2\) (not \(\lVert \vec u \rVert\)). Show your work, and write your final answer in the box provided. Your answer should be a number with no variables.

$$ \lVert \vec u \rVert^2 = \_\_\_\_\_\_ $$
Solution

We have

$$ 10^2=\|\vec u+\vec v\|^2=\|\vec u\|^2+2\vec u\cdot\vec v+\|\vec v\|^2 $$

and

$$ 6^2=\|\vec u-\vec v\|^2=\|\vec u\|^2-2\vec u\cdot\vec v+\|\vec v\|^2 $$

Notice that the expressions on the right-hand side are similar, except for the signs of \(2 \vec u \cdot \vec v\). So, adding these equations gives

$$ 136=2\|\vec u\|^2+2\|\vec v\|^2=2\|\vec u\|^2+50 $$

so

$$ \lVert \vec u \rVert^2 = \frac{136 - 50}{2} = \boxed{43} $$

SP26 MT1 · Problem 5 13 pts

From SP26 MT1

Suppose \(\vec u,\vec v\in\mathbb R^n\) are non-zero vectors and \(k\) is a scalar. Let

$$ f(k) = \lVert \vec u - k \vec v \rVert^2 + C k^2 $$

where \(C \geq 0\) is a non-negative constant.

a)

6 pts In this part only, suppose \(C=0\), \(\vec u = \begin{bmatrix} 1 \\ 2 \end{bmatrix}\), and \(\vec v = \begin{bmatrix} 3 \\ 1 \end{bmatrix}\). Find the value of \(k\) that minimizes \(f(k)\). Show your work, and write your final answer in the box provided. Your answer should be a number with no variables.

$$ \text{minimizer of } f(k) = \_\_\_\_\_\_ $$
Solution

There are several ways to think about this problem. What I expected most students to see is that when \(C = 0\), this is really asking for the orthogonal projection of \(\vec u\) onto \(\vec v\); the minimizer of \(f(k)\) is the value of \(k\) that makes \(\vec u - k \vec v\) orthogonal to \(\vec v\).

Using that logic, we know from Chapter 3.4 that the orthogonal projection of \(\vec u\) onto \(\vec v\) is given by

$$ \vec p = k^* \vec v = \left( \frac{\vec u \cdot \vec v}{\vec v \cdot \vec v} \right) \vec v $$

So,

$$ k^* = \frac{\vec u \cdot \vec v}{\vec v \cdot \vec v} = \frac{1 \cdot 3 + 2 \cdot 1}{3^2 + 1^2} = \frac{5}{10} = \boxed{\frac{1}{2}} $$

There’s another way to approach this problem, which is to simplify \(f(k)\) and treat this like a calculus problem.

$$ f(k)=\left\|\begin{bmatrix}1\\\\2\end{bmatrix}-k\begin{bmatrix}3\\\\1\end{bmatrix}\right\|^2=(1-3k)^2+(2-k)^2 $$

Expanding,

$$ f(k)=10k^2-10k+5 $$

so

$$ f'(k)=20k-10 $$

Setting \(f’(k)=0\) gives \(k^{\ast} = \frac{1}{2}\) as well.

b)

4 pts Note that \(f(k)\) almost looks like the squared norm of the vector \(\vec u - k \vec v\), but with an extra term \(C k^2\). Let’s try and rewrite \(f(k)\) so that it is the squared norm of another related vector.

Define two new vectors, \(\vec U, \vec V \in \mathbb R^{n+1}\) by appending the scalar \(a\) to the end of \(\vec u\) and the scalar \(b\) to the end of \(\vec v\).

$$ \vec U = \begin{bmatrix} u_1 \\\\ u_2 \\\\ \vdots \\\\ u_n \\\\ a\end{bmatrix}, \quad \vec V = \begin{bmatrix} v_1 \\\\ v_2 \\\\ \vdots \\\\ v_n \\\\ b\end{bmatrix} $$

Select values of \(a\) and \(b\) so that \(f(k) = \lVert \vec U - k \vec V \rVert^2\), for all possible non-negative values of \(C\).

  1. What is the value of \(a\)?

    0 \(C\) \(C^2\) \(\sqrt{C}\)
  2. What is the value of \(b\)?

    0 \(C\) \(C^2\) \(\sqrt{C}\)
    Solution
    0 \(C\) \(C^2\) \(\sqrt{C}\)

    First, let’s try and get a better sense of how \(\lVert \vec U - k \vec V \rVert^2\) works.

    $$ \begin{align*} \lVert \vec U - k \vec V \rVert^2 &= \left\lVert \begin{bmatrix} u_1 \\\\ u_2 \\\\ \vdots \\\\ u_n \\\\ a\end{bmatrix} - k \begin{bmatrix} v_1 \\\\ v_2 \\\\ \vdots \\\\ v_n \\\\ b\end{bmatrix} \right\rVert^2 \\\\ &= \left\lVert \begin{bmatrix} u_1 - kv_1 \\\\ u_2 - kv_2 \\\\ \vdots \\\\ u_n - kv_n \\\\ a - kb\end{bmatrix} \right\rVert^2 \\\\ &= \sum_{i=1}^n (u_i - kv_i)^2 + (a - kb)^2 \\\\ &= \lVert \vec u - k \vec v \rVert^2 + (a - kb)^2 \\\\ \end{align*} $$

    Our job is to find \(a\) and \(b\) so that \(f(k)\), which we were told is defined as

    $$ f(k) =\lVert \vec u - k \vec v \rVert^2 + C k^2 $$

    is also equal to

    $$ \lVert \vec U - k \vec V \rVert^2 = \lVert \vec u - k \vec v \rVert^2 + (a - kb)^2 $$

    If we set \(f(k) = \lVert \vec U - k \vec V \rVert^2\), we see that this boils down to finding \(a\) and \(b\) such that

    $$ (a - kb)^2 = C k^2 $$

    Notice the right-hand side of the expression above is just \(Ck^2\), not \(Ck^2 + \text{some constant} \cdot k + \text{some other constant}\). This means that \(a = 0\), and that forces \(b = \sqrt{C}\):

    $$ (0 - k\sqrt{C})^2 = Ck^2 $$

    So, the correct answers are \(\boxed{a=0}\) and \(\boxed{b=\sqrt{C}}\).

c)

3 pts As \(C\) increases, what happens to the value of \(k\) that minimizes \(f(k)\)? Explain your reasoning.

Solution

There are a couple of ways to think about this. First, if we use the interpretation provided in part b), the vectors \(\vec U\) and \(\vec V\) “bake in” the value of \(C\):

$$ \vec U = \begin{bmatrix} u_1 \\\\ u_2 \\\\ \vdots \\\\ u_n \\\\ 0\end{bmatrix}, \quad \vec V = \begin{bmatrix} v_1 \\\\ v_2 \\\\ \vdots \\\\ v_n \\\\ \sqrt{C}\end{bmatrix} $$

Increasing \(C\) keeps the dot product of \(\vec U\) and \(\vec V\) fixed, but increases the norm of \(\vec V\). Why is this relevant? Since \(f(k) = \lVert \vec U - k \vec V \rVert^2\), the minimizer \(k^{\ast}\) of \(f(k)\) is equal to

$$ k^* = \frac{\vec U \cdot \vec V}{\vec V \cdot \vec V} $$

So, as \(C\) increases, the denominator of \(k^{\ast}\) increases, so \(k^{\ast}\) moves toward \(0\), though this may happen either from the left or the right, since \(\vec U \cdot \vec V\) may be positive or negative.

If you’d prefer, you could just expand the original definition of \(f(k)\), take the derivative to find the closed-form expression for the minimizing \(k^{\ast}\) for an arbitrary \(C\), and look at what happens to \(k^{\ast}\) as \(C\) increases.

Recall, the original definition of \(f(k)\) is \(f(k)=\lVert \vec u - k \vec v \rVert^2 + C k^2\), so

$$ f(k)=\vec u \cdot \vec u - 2k(\vec u\cdot\vec v)+k^2\vec v \cdot \vec v+Ck^2 $$

Therefore,

$$ f'(k)=-2(\vec u\cdot\vec v)+2k(\vec v \cdot \vec v+C) $$

so the minimizer is

$$ k^*=\frac{\vec u\cdot\vec v}{\vec v \cdot \vec v+C} $$

As \(C\) increases, the denominator increases (but \(\vec u\) and \(\vec v\) are fixed — notice these are the original \(\vec u, \vec v\), not the new \(\vec U, \vec V\)), so \(k^{\ast}\) moves toward \(0\).


SP26 MT1 · Problem 9 10 pts

From SP26 MT1

a)

7 pts Suppose \(x\) and \(y\) are non-negative numbers. Using the Cauchy-Schwarz inequality, prove that

$$ \frac{(x+y)^2}{2}\le x^2+y^2 $$

Solutions that do not use the Cauchy-Schwarz inequality will not receive credit.

Solution

Recall, the Cauchy-Schwarz inequality states that for any two vectors \(\vec u, \vec v \in \mathbb{R}^n\),

$$ |\vec u \cdot \vec v| \leq \|\vec u\| \|\vec v\| $$

Applying Cauchy-Schwarz to the vectors \(\vec u=\begin{bmatrix}x\\y\end{bmatrix}\) and \(\vec v=\begin{bmatrix}1\\1\end{bmatrix}\) gives

$$ |x + y| \leq \sqrt{x^2 + y^2} \sqrt{1^2 + 1^2} = \sqrt{2(x^2 + y^2)} $$

Squaring both sides gives

$$ (x + y)^2 \leq 2(x^2 + y^2) $$

and finally, dividing both sides by \(2\) gives

$$ \frac{(x+y)^2}{2}\le x^2+y^2 $$

as needed.

b)

3 pts Now suppose \(x\), \(y\), and \(z\) are non-negative numbers. Which inequality is guaranteed to be true?

\(\displaystyle \frac{(x+y+z)^2}{2}\le x^2+y^2+z^2\) \(\displaystyle \frac{(x+y+z)^2}{3}\le x^2+y^2+z^2\) \(\displaystyle \frac{(x+y+z)^2}{2}\le x^3+y^3+z^3\) \(\displaystyle \frac{(x+y+z)^3}{3}\le x^3+y^3+z^3\) None of the above
Solution

None of the above

The Cauchy-Schwarz inequality directly implies one of the options, and the other options are all not guaranteed to be true. Extending our argument from part a), let’s now apply Cauchy-Schwarz to the vectors \(\vec u=\begin{bmatrix}x\\y\\z\end{bmatrix}\) and \(\vec v=\begin{bmatrix}1\\1\\1\end{bmatrix}\). This gives

$$ |x+y+z|\le \sqrt{x^2+y^2+z^2} \sqrt{1^2+1^2+1^2} = \sqrt{3(x^2+y^2+z^2)} $$

Squaring both sides and dividing by \(3\) gives

$$ \frac{(x+y+z)^2}{3}\le x^2+y^2+z^2 $$

which is the second option.


SP26 Final · Problem 3 10 pts MT1 Redemption

From SP26 Final

Let \(\vec x = \begin{bmatrix} 2 \\ 1 \\ 1 \end{bmatrix}\) and \(\vec z = \begin{bmatrix} 3 \\ 9 \\ 3 \end{bmatrix}\), and suppose \(\vec y \in \mathbb{R}^3\) is such that

the projection of \(\vec x\) onto \(\vec y\) is \(\vec 0\) and that \(\vec y \cdot \vec y = \vec y \cdot \vec z = 45\).

a)

4 pts Find the projection of \(\vec z\) onto \(\vec x\). Show your work, and write your final answer in the box provided. Give your answer as a vector with no variables.

$$ \text{projection of }\vec z\text{ onto }\vec x = \_\_\_\_\_\_ $$
Solution

Using the projection formula from Chapter 3.4,

$$ \vec p = \frac{\vec{z}\cdot\vec{x}}{\vec{x}\cdot\vec{x}}\vec{x} $$

Here,

$$ \vec{z}\cdot\vec{x}=3(2)+9(1)+3(1)=18, \qquad \vec{x}\cdot\vec{x}=2^2+1^2+1^2=6 $$

so

$$ \vec p = \frac{18}{6}\vec{x} = 3\begin{bmatrix}2\\\\1\\\\1\end{bmatrix} = \begin{bmatrix}6\\\\3\\\\3\end{bmatrix} $$
b)

6 pts Write \(\vec z\) as a linear combination of \(\vec x\) and \(\vec y\). Show your work, and fill in each box with a number with no variables. Hint: What is the relationship between \(\vec x\) and \(\vec y\)?

Solution

Since the projection of \(\vec{x}\) onto \(\vec{y}\) is \(\vec{0}\) and \(\vec{y}\cdot\vec{y}=45\), \(\vec{y}\) is nonzero and \(\vec{x}\cdot\vec{y}=0\). In other words, \(\vec{x}\) and \(\vec{y}\) are orthogonal.

Suppose

$$ \vec{z}=a\vec{x}+b\vec{y} $$

Taking dot products with \(\vec{x}\) gives

$$ \vec{z}\cdot\vec{x}=a(\vec{x}\cdot\vec{x})+b(\vec{y}\cdot\vec{x}) $$

Using the work from part a), \(\vec{z}\cdot\vec{x}=18\) and \(\vec{x}\cdot\vec{x}=6\). Since \(\vec{y}\cdot\vec{x}=0\),

$$ 18 = 6a $$

so \(a=3\).

Now take dot products with \(\vec{y}\):

$$ \vec{z}\cdot\vec{y}=a(\vec{x}\cdot\vec{y})+b(\vec{y}\cdot\vec{y}) $$

The problem tells us that \(\vec{z}\cdot\vec{y}=\vec{y}\cdot\vec{y}=45\), and \(\vec{x}\cdot\vec{y}=0\), so

$$ 45=45b $$

and therefore \(b=1\). So,

$$ \vec{z}=3\vec{x}+\vec{y} $$