The recipe is an algorithm, The recipe adjustments are parameters It's how you turn the 'knobs' or 'dials' to tweak how the model behaves. Because we all want to bake the perfect cake (or improve the model's performance). Imagine you're baking a cake. The recipe provides you with a set of instructions. - How much flour - How much sugar and milk - How many eggs These ingredients and their quantities are the data used to train the model. But you might need to make adjustments to the recipe to bake your perfect cake. - Adjust the oven temperature - Change the baking time - Add more sugar These adjustments you make to the recipe are like the parameters in a machine learning algorithm. ---- ### Too many vs too few A model's performance is often evaluated **based on its ability to make accurate predictions on unseen data.** Think of unseen data as a new chocolate cake recipe handed to you by a friend that you've never seen before. You use your previous experience to adapt to the new recipe. **More parameters** can allow a model to **capture more patterns.** But too many can be a problem. Just like if you play with the dials too much. The oven becomes too hot. You risk burning the cake. ### Overfitting Overfitting happens when the model becomes too complex and is trained too much on the training data. If a model is trained heavily on legal documents it may become too specialized in legal jargon and struggle to generate text on sports or fashion. If you use **too much data** you might get mixed signals about what makes a good cake. The results may not be reliable. ### Underfitting On the other hand, you might underfit the data. Underfitting happens when the model is **not trained enough resulting** in it not capturing the patterns in the data. This makes it unable to generalise to new, unseen data. In the case of our cake-baking analogy, underfitting could happen if you only bake one or two cakes. If you have too little data you might miss important insights that could lead to baking better cakes. > You need to find the right balance between overfitting and underfitting to achieve the best possible result. You need to bake enough cakes to capture patterns about what makes a delicious cake. Without overdoing it & getting lost in the weeds by adding that extra gram of sugar. ---- ### Other definitions of Parameters Parameters are **descriptive measures of an entire population that may be used as the inputs for a probability** distribution function (PDF) to generate distribution curves. Parameters are usually signified by Greek letters to distinguish them from sample statistics. Parameters are fixed constants, that is, they do not vary like variables. However, their values are usually unknown because it is infeasible to measure an entire population. Because of this, you can take a random sample from the population to obtain **parameter estimates.** One goal of statistical analyses is to obtain estimates of the population parameters along with the amount of error associated with these estimates. These estimates are also known as sample statistics. There are several types of parameter estimates: - Point estimates are the single, most likely value of a parameter. For example, the point estimate of population mean (the parameter) is the sample mean (the parameter estimate). - Confidence intervals are a range of values likely to contain the population parameter