(2.5 pts) 10 is the value of \(w\) that minimizes...
Fall 2025 Midterm 1
This page is meant to give you quick access to problems and their solutions. Refer to the original exam PDF, linked above, for test-taking instructions and formatting. Note that we’ve kept the problem text identical, which is why you may see things like “write your answer in the box below” despite there not being a box on this page.
Problems
- Problem 1: Consider the Following...
- Problem 2: Absolute Madness
- Problem 3: Spreading Your Wings
- Problem 4: Mission Impossible
- Problem 5: Back to Normal
- Problem 6: Needed Me
- Problem 7: High Definition
- Problem 8: Worst-Case Scenario
Problem 1: Consider the Following... 15 pts
Consider the following dataset of \(n = 9\) values.
| \(y_1\) | \(y_2\) | \(y_3\) | \(y_4\) | \(y_5\) | \(y_6\) | \(y_7\) | \(y_8\) | \(y_9\) |
|---|---|---|---|---|---|---|---|---|
| \(7\) | \(8\) | \(10\) | \(10\) | \(11\) | \(13\) | \(14\) | \(17\) | \(27\) |
Suppose we’d like to find the optimal parameter, \(w^{\ast}\), for the constant model \(h(x_i) = w\), given this dataset of 9 values.
In parts a) through f), choose the empirical risk function \(R(w)\) that the given value of \(w^{\ast}\) is the minimizer of, for this particular dataset. If you believe the given value of \(w^{\ast}\) does not minimize any of the five options, select N/A.
Option 1: \(\displaystyle R(w) = \frac{1}{n} \sum_{i = 1}^n (y_i - w)^2\)
Option 2: \(\displaystyle R(w) = \frac{1}{n} \sum_{i = 1}^n (27y_i - 13w)^2\)
Option 3: \(\displaystyle R(w) = \frac{1}{n} \sum_{i = 1}^n 13|y_i - w|\)
Option 4: \(\displaystyle R(w) = \frac{1}{n} \sum_{i = 1}^n \begin{cases} 13 & \text{if } y_i = w \\ 27 & \text{if } y_i \neq w \end{cases}\)
Option 5: \(\displaystyle R(w) = \lim_{p \rightarrow \infty} \frac{1}{n} \sum_{i = 1}^n |y_i - w|^p\)
(2.5 pts) 11 is the value of \(w\) that minimizes...
Solution
(2.5 pts) 12 is the value of \(w\) that minimizes...
Solution
(2.5 pts) 13 is the value of \(w\) that minimizes...
Solution
(2.5 pts) 17 is the value of \(w\) that minimizes...
Solution
(2.5 pts) 27 is the value of \(w\) that minimizes...
Solution
Problem 2: Absolute Madness 17 pts
Consider a dataset of \(n = 8\) values, where
and \(4 < \alpha < 20\).
As usual, let \(R_\text{abs}(w)\) represent the mean absolute error of a constant prediction \(w\) on this dataset of 8 values.
3 pts Is the value of \(w^{\ast}\), the minimizer of \(R_\text{abs}(w)\), unique? Select and fill out one option below.
The value of \(w^{\ast}\) is unique, and is equal to ______.
The value of \(w^{\ast}\) is not unique; any value between ______ and ______ is a minimizer.
Solution
The value of \(w^{\ast}\) is unique, and is equal to \(\boxed{\alpha}\).
The value of \(w^{\ast}\) is not unique; any value between ______ and ______ is a minimizer.
6 pts Find the value of \(R_\text{abs}(\alpha)\), for any valid choice of \(\alpha\). Show your work, and \(\boxed{\text{circle}}\) your final answer, which should be an expression involving \(\alpha\) and other constants, but no other variables, and no summation notation.
Solution
\(R_\text{abs}(\alpha) = \frac{\alpha + 31}{8}\).
Let’s start with the definition of \(R_\text{abs}(w)\) and plug in \(w = \alpha\).
Since \(\alpha > 1\) and \(\alpha > 4\), we know that \(|1 - \alpha| = \alpha - 1\) and \(|4 - \alpha| = \alpha - 4\). Similarly, since \(\alpha < 20\), we have \(|20 - \alpha| = 20 - \alpha\).
Recall,
where \(4 < \alpha < 20\).
8 pts Let the minimum possible value of \(R_\text{abs}(w)\) be \(M\). Given that
find the value of \(\alpha\). Show your work, and \(\boxed{\text{circle}}\) your final answer, which should be a number with no variables.
Hint: It’s possible to answer this without using your answer from the previous part.
Solution
\(\alpha = 11\).
Since \(\alpha\) minimizes \(R_\text{abs}(w)\), we know that \(R_\text{abs}(\alpha) = M\). In the previous part, we found an expression for \(R_\text{abs}(\alpha)\). One common solution was to find another expression for \(R_\text{abs}(20)\) (which is also a function of \(\alpha\)), and then to solve for the \(\alpha\) such that
Here’s another solution. Since \((\alpha, M)\) is the vertex of \(R_\text{abs}(w)\), we know that the slope to the left of it is negative and the slope to the right of it is positive.
The slope on the line segment between \((\alpha, M)\) and \((20, M + \frac{9}{2})\) is
So, now we know that on the line segment between \((\alpha, M)\) and \((20, M + \frac{9}{2})\), the slope is \(\frac{1}{2}\). This is all we need to solve for \(\alpha\). Since the slope of a line segment is its change in \(y\) over its change in \(x\), we have:
Solving for \(\alpha\), we get:
Problem 3: Spreading Your Wings 12 pts
Consider a dataset of \(n\) points, \((x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)\), where
the means of \(x_1, x_2, \ldots, x_n\) and \(y_1, y_2, \ldots, y_n\) are 15 and 5, respectively
the variances of \(x_1, x_2, \ldots, x_n\) and \(y_1, y_2, \ldots, y_n\) are \(\sigma_x^2\) and \(\sigma_y^2\), respectively
the correlation coefficient between \(x_1, x_2, \ldots, x_n\) and \(y_1, y_2, \ldots, y_n\) is \(r\)
We define a new set of values, \(z_1, z_2, \ldots, z_n\), as follows:
4 pts Suppose we fit a simple linear regression line to the dataset \((x_1, z_1), (x_2, z_2), \ldots, (x_n, z_n)\) by minimizing mean squared error. Note that \(z\) is the variable being predicted, not \(y\). Let \(h(x_i)\) represent the corresponding line.
What is the value of \(h(15)\)? Your answer should be a number with no variables.
\(h(15) =\) ______
Solution
\(h(15) = 40\).
The key fact being assessed here is that the line that minimizes mean squared error always passes through
Normally this is stated as the line passing through the point \((\bar{x}, \bar{y})\), but here the output variable is \(z\), not \(y\).
The mean of \(z\) is \(3 \bar{x} - \bar{y}\), as we explored in a homework problem, and this is
8 pts \(\sigma_z^2\), the variance of \(z_1, z_2, \ldots, z_n\), can be written in the form \(\sigma_z^2 = 9 \sigma_x^2 + \sigma_y^2 + C\).
What is the value of \(C\)?
Show your work in the box below. English explanations are not enough.
Solution
We’ll find the answer by expanding out the definition of \(\sigma_z^2\) and simplifying.
$$ \begin{align*} \sigma_z^2 &= \frac{1}{n} \sum_{i=1}^n (z_i - \bar{z})^2 \\\\ &= \frac{1}{n} \sum_{i=1}^n (3x_i - y_i - (3\bar{x} - \bar{y}))^2 \\\\ &= \underbrace{\frac{1}{n} \sum_{i=1}^n (3x_i - 3\bar{x} - y_i + \bar{y})^2}_\text{distributed the negative sign and rearranged} \\\\ &= \frac{1}{n} \sum_{i=1}^n \underbrace{(3(x_i - \bar{x}) - (y_i - \bar{y}))^2}_\text{treat this as }(a - b)^2 \\\\ &= \frac{1}{n} \sum_{i=1}^n \left( 9(x_i - \bar{x})^2 - 6(x_i - \bar{x})(y_i - \bar{y}) + (y_i - \bar{y})^2 \right) \\\\ &= 9 \left(\frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 \right) - 6 \left(\frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})\right) + \left(\frac{1}{n} \sum_{i=1}^n (y_i - \bar{y})^2 \right) \\\\ &= 9 \sigma_x^2 + \sigma_y^2 - 6 \left(\frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})\right) \\\\ \end{align*} $$So,
$$ C = -6 \left(\frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})\right) $$But, recall that
$$ r = \frac{1}{n} \sum_{i=1}^n \left( \frac{x_i - \bar{x}}{\sigma_x} \right) \left( \frac{y_i - \bar{y}}{\sigma_y} \right) $$which means that
$$ r \sigma_x \sigma_y = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y}) $$So,
$$ C = -6 \left(\frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})\right) = -6r \sigma_x \sigma_y $$
Problem 4: Mission Impossible 12 pts 🎥 Walkthrough
6 pts Suppose \(\vec u, \vec v \in \mathbb{R}^n\) are non-zero vectors, and suppose that
For each statement below, determine whether it is impossible, possible, or guaranteed to be true, given the above assumptions. Select exactly one option from each row. The first statement has been done for you as an example.
| statement | impossible? | possible? | guaranteed? | |
|---|---|---|---|---|
| \(i\) | \(\lVert \vec u \rVert = 5\) | |||
| \(ii\) | \(\vec u\) and \(\vec v\) are orthogonal | |||
| \(iii\) | \(\lVert \vec u - \vec v \rVert = 0\) | |||
| \(iv\) | \(\vec u\) and \(\vec v\) span a 1-dimensional subspace of \(\mathbb{R}^n\) | |||
| \(v\) | \(\vec u\) and \(\vec v\) span a 2-dimensional subspace of \(\mathbb{R}^n\) | |||
| \(vi\) | \(\lVert \vec u + \vec v \rVert = \lVert \vec u \rVert + \lVert \vec v \rVert\) |
Solution
Remember that for any two vectors \(\vec u\) and \(\vec v\),
The fact that we’re told that
tells us that \(\cos \theta = 1\) or \(\cos \theta = -1\), which means that the angle between \(\vec u\) and \(\vec v\) is \(0^\circ\) or \(180^\circ\), which means that \(\vec u\) and \(\vec v\) are scalar multiples of each other. (They may point in the same or opposite directions.) This is the key insight to assessing each of the statements.
6 pts Suppose \(\vec w, \vec z \in \mathbb{R}^n\). Given that \(\lVert \vec w \rVert = \lVert \vec z \rVert = \lVert \vec w - \vec z \rVert = 1\), find \(\lVert \vec w + \vec z \rVert\). Show your work, and \(\boxed{\text{circle}}\) your final answer, which should be a number with no variables.
Solution
\(\lVert \vec w + \vec z \rVert = \sqrt{3}\).
We’re asked to find \(\lVert \vec w + \vec z \rVert\). To do so, let’s expand out \(\lVert \vec w + \vec z \rVert^2\) as we’ve done in the past, and see how to utilize what we were given.
Above, we’ve plugged in \(\lVert \vec w \rVert^2 = 1\) and \(\lVert \vec z \rVert^2 = 1\). We need to know \(\vec w \cdot \vec z\), which we don’t yet know.
But, we have enough information to find it, if we expand out \(\lVert \vec w - \vec z \rVert^2\), which we were told is equal to 1.
Solving the above gives us \(\vec w \cdot \vec z = \frac{1}{2}\). This gives
And so,
Problem 5: Back to Normal 12 pts 🎥 Walkthrough
Consider the orthogonal vectors \(\vec u_1 = \begin{bmatrix} 13 \\ -3 \\ 2 \end{bmatrix}\), \(\vec u_2 = \begin{bmatrix} 0 \\ 4 \\ 6 \end{bmatrix}\), and \(\vec u_3 = \begin{bmatrix} 1 \\ 3 \\ -2 \end{bmatrix}\).
4 pts Find the equation of the plane spanned by \(\vec u_2\) and \(\vec u_3\) in standard form, i.e. \(ax + by + cz + d = 0\). \(\boxed{\text{Circle}}\) your final answer.
Solution
Plane: \(13x - 3y + 2z = 0\) (or any scalar multiple of this equation).
Most students took the cross product of \(\vec u_2\) and \(\vec u_3\) to find a vector that is orthogonal to the plane spanned by \(\vec u_2\) and \(\vec u_3\), and then used that vector to define the plane.
But, we were already told that all three vectors are orthogonal to each other, which means that the vector orthogonal to the plane spanned by \(\vec u_2\) and \(\vec u_3\) is \(\vec u_1\). So, we can use \(\vec u_1\) to define the plane.
So, the equation of the plane spanned by \(\vec u_2\) and \(\vec u_3\) is \(13x - 3y + 2z = 0\) (or any scalar multiple of this equation).
8 pts There is one value of \(k\) such that the projection of \(\vec x = \begin{bmatrix} 7 \\ 3 \\ 1 \end{bmatrix}\) onto \(\vec u_k\) is just \(\vec u_k\) itself.
What is the value of \(k\)?
Show your work in the box below. English explanations are not enough.
Solution
We’re told that for one of the three provided vectors — \(\vec u_1\), \(\vec u_2\), or \(\vec u_3\) — the projection of \(\vec x\) onto that vector is just that vector itself.
Remember that the projection of \(\vec x\) onto \(\vec u_k\) is given by
$$ \text{proj}_{\vec u_k} \vec x = \frac{\vec x \cdot \vec u_k}{\vec u_k \cdot \vec u_k} \vec u_k $$So, we need to find the vector \(\vec u_k\) such that the scalar \(\frac{\vec x \cdot \vec u_k}{\vec u_k \cdot \vec u_k}\) is equal to 1, or equivalently, \(\vec x \cdot \vec u_k = \vec u_k \cdot \vec u_k\). We can check this equality for each of the three provided vectors.
\(x \cdot \vec u_1 = \begin{bmatrix} 7 \\ 3 \\ 1 \end{bmatrix} \cdot \begin{bmatrix} 13 \\ -3 \\ 2 \end{bmatrix} = 7 \cdot 13 + 3 \cdot (-3) + 1 \cdot 2 = 84\)
\(\vec u_1 \cdot \vec u_1 = 13^2 + (-3)^2 + 2^2 = 180\)
\(84 \neq 180\), so \(\vec u_1\) is not the vector we’re looking for.
\(x \cdot \vec u_2 = \begin{bmatrix} 7 \\ 3 \\ 1 \end{bmatrix} \cdot \begin{bmatrix} 0 \\ 4 \\ 6 \end{bmatrix} = 7 \cdot 0 + 3 \cdot 4 + 1 \cdot 6 = 18\)
\(\vec u_2 \cdot \vec u_2 = 0^2 + 4^2 + 6^2 = 52\)
\(18 \neq 52\), so \(\vec u_2\) is not the vector we’re looking for.
\(x \cdot \vec u_3 = \begin{bmatrix} 7 \\ 3 \\ 1 \end{bmatrix} \cdot \begin{bmatrix} 1 \\ 3 \\ -2 \end{bmatrix} = 7 \cdot 1 + 3 \cdot 3 + 1 \cdot (-2) = 14\)
\(\vec u_3 \cdot \vec u_3 = 1^2 + 3^2 + (-2)^2 = 14\)
\(14 = 14\), so \(\vec u_3\) is the vector we’re looking for.
Problem 6: Needed Me 11 pts 🎥 Walkthrough
Suppose \(\vec x = \begin{bmatrix} c \\ 1 \\ 0 \end{bmatrix}\), \(\vec y = \begin{bmatrix} 1 \\ c \\ 1 \end{bmatrix}\), and \(\vec z = \begin{bmatrix} 0 \\ 1 \\ c \end{bmatrix}\), where \(c \in \mathbb{R}\) is a constant.
8 pts Find a positive value of \(c\) such that \(\vec x\), \(\vec y\), and \(\vec z\) are linearly dependent. Show your work, and \(\boxed{\text{circle}}\) your final answer, which should be a positive number with no variables.
Solution
\(c = \sqrt{2}\).
For \(\vec x\), \(\vec y\), and \(\vec z\) to be linearly dependent, there must exist scalars \(a\), \(b\), and \(c\) such that
(or equivalently, \(a \vec x + b \vec y + d\vec z = \vec 0\), but the former approach involves one fewer variable to solve for).
Substituting in the given vectors, we have
As a system of equations, we have
The third equation gives us \(b = c\), and the second gives us \(a = 1 - cb = 1 - c^2\). Substituting these into the first equation gives us
This equation has three solutions for \(c\): \(c = 0\), \(c = \sqrt{2}\), and \(c = -\sqrt{2}\). We’re asked to find a positive value of \(c\), so \(c = \sqrt{2}\) for this part, and either \(0\) or \(-\sqrt{2}\) for the next part.
3 pts Provide one other value of \(c\) (that is, not your answer from the previous part) such that \(\vec x\), \(\vec y\), and \(\vec z\) are linearly dependent. Your answer should be a number with no variables.
other value of \(c =\) ______
Solution
other value of \(c =\) \(\boxed{\text{0 or }-\sqrt{2}}\)
Problem 7: High Definition 12 pts
Suppose \(\vec x_1, \vec x_2, \ldots \vec x_{12}\) are 12 non-zero vectors in \(\mathbb{R}^{7}\). Furthermore, suppose:
\(\vec x_1\), \(\vec x_2\), and \(\vec x_3\) span a 2-dimensional subspace of \(\mathbb{R}^{7}\).
\(\vec x_4\), \(\vec x_5\), and \(\vec x_6\) span the same 2-dimensional subspace of \(\mathbb{R}^{7}\) as \(\vec x_1\), \(\vec x_2\), and \(\vec x_3\), i.e.
4 pts Let \(r\) be the dimension of the subspace of \(\mathbb{R}^{7}\) spanned by \(\vec x_1, \vec x_2, \ldots \vec x_{12}\). What are the smallest and largest possible values of \(r\)? Your answers should be integers with no variables.
smallest possible value of \(r =\) ______ largest possible value of \(r =\) ______
Solution
smallest possible value of \(r =\) \(\boxed{2}\)
largest possible value of \(r =\) \(\boxed{7}\)
4 pts Which of the following could form a basis for \(\mathbb{R}^{7}\)? Select all that apply. Blank answers will receive no credit.
\(\lbrace\vec x_7, \vec x_8, \vec x_9, \vec x_{10}, \vec x_{11}, \vec x_{12}\rbrace\)
\(\lbrace\vec x_6, \vec x_7, \vec x_8, \vec x_9, \vec x_{10}, \vec x_{11}, \vec x_{12}\rbrace\)
\(\lbrace\vec x_1, \vec x_5, \vec x_8, \vec x_9, \vec x_{10}, \vec x_{11}, \vec x_{12}\rbrace\)
\(\lbrace\vec x_1, \vec x_2, \vec x_5, \vec x_9, \vec x_{10}, \vec x_{11}, \vec x_{12}\rbrace\)
\(\lbrace\vec x_1, \vec x_2, \vec x_8, \vec x_9, \vec x_{10}, \vec x_{11}, \vec x_{12}\rbrace\)
Solution
\(\lbrace\vec x_1, \vec x_2, \vec x_8, \vec x_9, \vec x_{10}, \vec x_{11}, \vec x_{12}\rbrace\)
The first choice only includes 6 vectors, but since the span of \(\vec x_1, \vec x_2, \ldots \vec x_{12}\) is 7-dimensional, it must include at least 7 vectors. So, the first choice is not a valid basis.
The fourth choice includes 7 vectors, but we know that \(\vec x_1, \vec x_2, \vec x_5\) are a linearly dependent set since they all lie on the same 2-dimensional subspace of \(\mathbb{R}^7\) (and you only need 2 vectors to uniquely define a 2-dimensional subspace), so the fourth choice is not a valid basis.
The other options all include 7 vectors that could be linearly independent, and so they could form a basis for \(\mathbb{R}^7\).
4 pts Suppose the intersection of \(\text{span}(\lbrace\vec x_1, \vec x_2\rbrace)\) and \(\text{span}(\lbrace \vec x_4, \vec x_5 \rbrace)\) is a line (i.e. a 1-dimensional subspace) in \(\mathbb{R}^{7}\). Which of the following must be true? Select all that apply. Blank answers will receive no credit.
Hint: Don’t forget the assumptions introduced at the start of the problem.
\(\vec x_2\), \(\vec x_4\), and \(\vec x_5\) can all be written as scalar multiples of \(\vec x_1\).
The set \(\lbrace \vec x_2, \vec x_4 \rbrace\) is linearly independent.
The set \(\lbrace \vec x_3, \vec x_4 \rbrace\) is linearly independent.
The set \(\lbrace \vec x_3, \vec x_6 \rbrace\) is linearly independent.
None of the above.
Solution
None of the above.
The intended answer to the problem was options 1 and 3. The scenario we had in mind was that \(\operatorname{span}(\lbrace\vec x_1, \vec x_2\rbrace) = \operatorname{span}(\lbrace\vec x_4, \vec x_5\rbrace) = \text{the same line}\). The two spans can’t both be different planes that happen to intersect in a line, since we’re told that \(\vec x_1, \vec x_2, \vec x_3\) span a 2-dimensional subspace of \(\mathbb{R}^7\) and \(\vec x_4, \vec x_5, \vec x_6\) span the same 2-dimensional subspace of \(\mathbb{R}^7\). So, if the two spans are planes, they’re the same plane, and they would intersect at a plane. Since the two spans intersect at a line, we thought they’d both have to be lines. If that was the case, then \(\vec x_2\), \(\vec x_4\), and \(\vec x_5\) would all be scalar multiples of \(\vec x_1\), and so \(\vec x_3\) would have to not be on that line (for \(\vec x_1, \vec x_2, \vec x_3\) to span a 2-dimensional subspace), which is why Options 1 and 3 were our originally intended answers.
But after releasing exam scores, a student brought up a possibility we hadn’t considered: it’s possible that \(\operatorname{span}(\lbrace\vec x_1, \vec x_2\rbrace)\) is a plane, and \(\operatorname{span}(\lbrace\vec x_4, \vec x_5\rbrace)\) is a line that is contained on that plane. That setup would satisfy all of the assumptions provided in the problem statement, but it would imply that none of the options are true.
So, retroactively, we gave full credit to everyone for this part.
Problem 8: Worst-Case Scenario 8 pts 🎥 Walkthrough
Suppose \(a, b, c, d, e\) are positive real numbers. Find the largest real number \(T\) such that it’s guaranteed that
Think of \(T\) as the “best possible lower bound”. For instance, we know that the expression on the left-hand side above must be greater than or equal to 0, since \(a, b, c, d, e\) are all positive, but \(T = 0\) is not the answer, since there’s a larger value of \(T\) that also guarantees the inequality holds.
Show your work, and \(\boxed{\text{circle}}\) your final answer, which should be a number with no variables.
Hint: Use the Cauchy-Schwarz inequality.
Solution
\(T = 25\).
Recall, the Cauchy-Schwarz inequality states that for any two vectors \(\vec u\) and \(\vec v\),
Let’s define two vectors \(\vec u\) and \(\vec v\) and then apply the Cauchy-Schwarz inequality to them.
Let’s compute the three quantities involved in the inequality.
\(\lVert \vec u \rVert = \sqrt{a + b + c + d + e}\)
\(\lVert \vec v \rVert = \sqrt{\frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d} + \frac{1}{e}}\)
\(|\vec u \cdot \vec v| = |\sqrt{a} \cdot \frac{1}{\sqrt{a}} + \sqrt{b} \cdot \frac{1}{\sqrt{b}} + \sqrt{c} \cdot \frac{1}{\sqrt{c}} + \sqrt{d} \cdot \frac{1}{\sqrt{d}} + \sqrt{e} \cdot \frac{1}{\sqrt{e}}| = 5\)
So, we have that
Squaring both sides of the inequality gives us
This means that for any positive values of \(a, b, c, d, e\), it’s impossible for \((a + b + c + d + e) \left( \frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d} + \frac{1}{e} \right)\) to be less than 25. Finding a value equal to 25 is doable if we set \(a = b = c = d = e = 1\). So, \(T = 25\) is the largest possible value of \(T\) that guarantees the inequality holds.