Hey there! Sign in to join this conversationNew here? Join for free

Least Squares Deriviation Watch

Announcements
    • Thread Starter
    Offline

    2
    ReputationRep:
    I've got the derivation of least squares in my notes here, but I'm confused about part of it. I'll post everything, in case I missed something.

    (Assuming the errors in x are much smaller than the errors in y)

    Let us represent the N data points, with i going from 1 to N, by (x_i, y_i). We then need to define S, the sum of the vertical squared distances of all the points from the straight line y = mx+c. This is given by

    S = \displaystyle\sum_{i=1}^N (y_i - mx_i - c)^2

    To find the values of c and m which minimise S we partially differentiate with respect to c, and to m. The derivatives are set to equal zero. The values of c and m which satisfy the resulting pair of equations are the values which minimise S.

    \displaystyle \frac{\partial S}{\partial c} = \displaystyle\sum_{i=1}^N -2(y_i - mx_i - c) = 0

    \displaystyle \frac{\partial S}{\partial m} = \displaystyle\sum_{i=1}^N -2x_i(y_i - mx_i - c) = 0

    The first equation gives us:

    \displaystyle\sum_{i=1}^N y_i -m \displaystyle\sum_{i=1}^N x_i - Nc = 0

    and the second:

    \displaystyle\sum_{i=1}^N x_i y_i -m\displaystyle\sum_{i=1}^N x^2_i - c\displaystyle\sum_{i=1}^N x_i = 0

    Solving these simultaenously gives us the values of m and c, which allows us to write the equation of the line of best fit for the data points.

    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    My question: where do the 2s in the derivitives go?

    I'm probably being obtuse, but it's actually confusing me lol. I figure once I fully understand the process for the line of best fit, adapting to a curve shouldn't be difficult. (It's just one more equation to solve simultaenously, right? :P)
    Offline

    4
    ReputationRep:
    If you have the equation 2*(whatever) = 0, then it's true that (whatever) = 0. They've done this because it makes it easier to look at.
    • Thread Starter
    Offline

    2
    ReputationRep:
    (Original post by spex)
    If you have the equation 2*(whatever) = 0, then it's true that (whatever) = 0. They've done this because it makes it easier to look at.
    *facepalm*

    I feel like an actual idiot now. I knew it was something obvious. Thanks!

    EDIT: I guess an actual derivation of this method would require looking at the second derivitives to prove that m and c are actually minimised, right?
    Offline

    4
    ReputationRep:
    You could do that, or you could think about it a bit more. There's not going to be a maximum distance for how far a line can be from the set of points, because you can always move it further. So the critical point where the derivatives are 0 must be a minimum.
    • Thread Starter
    Offline

    2
    ReputationRep:
    (Original post by spex)
    You could do that, or you could think about it a bit more. There's not going to be a maximum distance for how far a line can be from the set of points, because you can always move it further. So the critical point where the derivatives are 0 must be a minimum.
    ...That also makes perfect sense. Already +rep'd you.
 
 
 
  • See more of what you like on The Student Room

    You can personalise what you see on TSR. Tell us a little about yourself to get started.

  • Poll
    Will you be richer or poorer than your parents?
    Useful resources

    Make your revision easier

    Maths

    Maths Forum posting guidelines

    Not sure where to post? Read the updated guidelines here

    Equations

    How to use LaTex

    Writing equations the easy way

    Student revising

    Study habits of A* students

    Top tips from students who have already aced their exams

    Study Planner

    Create your own Study Planner

    Never miss a deadline again

    Polling station sign

    Thinking about a maths degree?

    Chat with other maths applicants

    Can you help? Study help unanswered threads

    Groups associated with this forum:

    View associated groups
  • See more of what you like on The Student Room

    You can personalise what you see on TSR. Tell us a little about yourself to get started.

  • The Student Room, Get Revising and Marked by Teachers are trading names of The Student Room Group Ltd.

    Register Number: 04666380 (England and Wales), VAT No. 806 8067 22 Registered Office: International House, Queens Road, Brighton, BN1 3XE

    Quick reply
    Reputation gems: You get these gems as you gain rep from other members for making good contributions and giving helpful advice.