I'm studying a course (Mathematics for Machine Learning). There's a quiz that is designed to test knowledge on using a model to fit data to predict frequencies in a Normal (Gaussian) distribution. The model used basically uses two parameters (Mean (μ) and Standard deviation (σ)), which are tweaked to enable the model to fit the data (in the right bell curve shape) for optimised prediction. The two parameters can be in a vector p. The question is as follows:

--------------------- START OF QUESTION ---------------------

Since each parameter vector p represents a different bell curve, each with its own value for the sum of squared residuals, SSR, we can draw the surface of SSR values over the space spanned by p, such as μ and σ in this example.

Here is an illustration of this surface for our data.

enter image description here

Every point on this surface represents the SSR of a choice of parameters, with some bell curves performing better at representing the data than others.

We can take a ‘top-down’ view of the surface, and view it as a contour map, where each of the contours (in green here) represent a constant value for the SSR.

enter image description here

The goal in machine learning is to find the parameter set where the model fits the data as well as it possibly can. This translates into finding the lowest point, the global minimum, in this space.

--------------------- END OF QUESTION ---------------------

The question then provides some statements, where I have to choose true or false. But before that, I don't even understand certain things about the question itself.

1- I didn't understand what is causing the effect of the contours, and why is it in the shape of a contour and not any other shape?

2- I didn't understand the graph itself. Why are the two parameters: Mean (μ) and Standard deviation (σ) had cm as units of measurements, while SSR is in decimals going up in intervals: 0.01, 0.02 ...etc.

3- Is this the right interpretation to say: that if we get to the center of the hole in the surface, this means we have reached the global minima - the most optimised parameters, causing the model to predict correctly?

4- If my statement in the previous question is correct, then referring to the second diagram, is it correct to say then, that a Standard deviation of 8cm and a Mean of 178cm gives us the global minima? Or am I incorrectly interpreting the graph?

5- There's another question that follows, which is of similar nature, but again I didn't understand how to interpret the graph. The question is as follows:

--------------------- START OF QUESTION ---------------------

Often we can't see the whole parameter space, so instead of just picking the lowest point, we have to make educated guesses where better points will be.

We can define another vector, Δp, in the same space as p that tells us what change can be made to p to get a better fit.

For example, a model with parameters p' = p + Δp will produce a better fit to data, if we can find a suitable Δp.

The second course in this specialisation will detail how to calculate these changes in parameters, Δp.

Given the following contour map,

enter image description here

What Δp will give the best improvement in the model?

enter image description here

--------------------- END OF QUESTION ---------------------

I didn't understand how to read the graph. Why is Δp = [-2,2]?

Many thanks in advance

  • $\begingroup$ No answer... but that question with a multiple choice rubric just seems weird... $\endgroup$
    – David P
    Feb 19 '21 at 0:35
  • $\begingroup$ Hi David. The questions are from the Coursera course - Mathematics for Machine Learning. If you don't have an answer and find the questions weird, and you have a reputation of 11.2K in Mathematics, then what hope do I have? :) $\endgroup$
    – Hazzaldo
    Feb 20 '21 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.