Understanding Linear Regression

Although [Vitor Fróis] is explaining linear regression because it relates to machine learning, the post and, indeed, the topic have wide applications in many things that we do with electronics …read more

May 9, 2025 - 03:27

Although [Vitor Fróis] is explaining linear regression because it relates to machine learning, the post and, indeed, the topic have wide applications in many things that we do with electronics and computers. It is one way to use independent variables to predict dependent variables, and, in its simplest form, it is based on nothing more than a straight line.

You might remember from school that a straight line can be described by: y=mx+b. Here, m is the slope of the line and b is the y-intercept. Another way to think about it is that m is how fast the line goes up (or down, if m is negative), and b is where the line “starts” at x=0.

[Vitor] starts out with a great example: home prices (the dependent variable) and area (the independent variable). As you would guess, bigger houses tend to sell for more than smaller houses. But it isn’t an exact formula, because there are a lot of reasons a house might sell for more or less. If you plot it, you don’t get a nice line; you get a cloud of points that sort of group around some imaginary line.

There are mathematical ways to figure out what line you should imagine, but you can often eyeball it, too. The real trick is evaluating the quality of that imaginary line.

To do that, you need an error measure. If you didn’t know better, you’d probably think expressing the error in terms of absolute value would be best. You know, “this is 10 off” or whatever. But, as [Vitor] explains, the standard way to do this is with a squared error term R². Why? Read the post and find out.

For electronics, linear regression has many applications, including interpreting sensor data. You might also use it to generalize a batch of unknown components, for example. Think of a batch of transistors with different Beta values at different frequencies. A linear regression will help you predict the Beta and the error term will tell you if it is worth using the prediction or not. Or, maybe you just want to make the perfect cup of coffee.