klosovic2
Badges: 6
Rep:
?
#1
Report Thread starter 2 years ago
#1
I was doing some exploratory data analysis, and I found a couple of outliers. In what circumstance is it correct to remove them before fitting the linear models? They are not misrecorded but they are definitely affected by abnormal factors which are not measured in the data (for example natural disasters and wars).
0
reply
Gregorius
Badges: 14
Rep:
?
#2
Report 2 years ago
#2
(Original post by klosovic2)
I was doing some exploratory data analysis, and I found a couple of outliers. In what circumstance is it correct to remove them before fitting the linear models? They are not misrecorded but they are definitely affected by abnormal factors which are not measured in the data (for example natural disasters and wars).
Whenever you fit a statistical model, you assume (explicitly or implicitly) a particular probability model. In the case of linear regression, you assume that the outcome is normally distributed with constant standard deviation, conditional on a mean defined by the linear predictor.

As you've already identified that your outliers are not data errors, then what you're telling me is that these anomalous points don't fit your probability model, and you've identified that they are probably outliers because the lack of particular measurements concerning them does not allow them to be incorporated into your probability model.

Excluding such data points would be a valid way forward, provided that their exclusion was carefully documented in the write-up!
0
reply
X

Quick Reply

Attached files
Write a reply...
Reply
new posts
Back
to top
Latest
My Feed

See more of what you like on
The Student Room

You can personalise what you see on TSR. Tell us a little about yourself to get started.

Personalise

Do you think receiving Teacher Assessed Grades will impact your future?

I'm worried it will negatively impact me getting into university/college (171)
44.19%
I'm worried that I’m not academically prepared for the next stage in my educational journey (43)
11.11%
I'm worried it will impact my future career (32)
8.27%
I'm worried that my grades will be seen as ‘lesser’ because I didn’t take exams (83)
21.45%
I don’t think that receiving these grades will impact my future (36)
9.3%
I think that receiving these grades will affect me in another way (let us know in the discussion!) (22)
5.68%

Watched Threads

View All
Latest
My Feed