The Student Room Group
Uni students - Complete our survey >>
Uni students - Complete our survey >>
Uni students - Complete our survey >>

Ukcat technical report

Has anyone looked at the technical report just published?

i'm no statistician and cant't query their statement on reliability of tests. However.....

Table 3 scale score reliability gives the mean score for each version of the subtlests. By my calculations for VR, the average varies by 19 points, for QR it's 17 points, for AR it's 23 and for DA it's 3 points.

I don't know how they mix and match the tests but if it is completely random you could have been unlucky and sat the test with the lowest average. This would give an average of 62 points less than if you sat the ones with the highest average.

Maybe it's not a huge number, but with absolute cut offs and decile scoring, could it make the difference between an offer/interview or not?


Also, no mention of SJT in the report.
Reply 1
Avatar for Gibber96
Gibber96
9
Is one of the subtests harder in every aspect? If it's harder in some sections but easier in others then the overall effect might be negligible. Interesting, though.
Reply 2
Avatar for Paradox.
Paradox.
7
Original post by Gibber96
Is one of the subtests harder in every aspect? If it's harder in some sections but easier in others then the overall effect might be negligible. Interesting, though.

It doesn't work like that - each section has 2 or 3 versions and everyone who sits the test is randomly assigned one of the versions for each section. So there are actually loads of different "subtests" because there are so many different combinations of these versions.

It also means that someone could theoretically get the hardest version for all 4 sections - and as OP said putting them down a massive amount on someone who got all 4 easiest, which could easily lead to them missing out on an interview, especially at UKCAT cut-off unis like Newcastle and Southampton.

Not a fair way of marking at all IMO.
Reply 3
Avatar for Gibber96
Gibber96
9
Original post by Paradox.
It doesn't work like that - each section has 2 or 3 versions and everyone who sits the test is randomly assigned one of the versions for each section. So there are actually loads of different "subtests" because there are so many different combinations of these versions.

It also means that someone could theoretically get the hardest version for all 4 sections - and as OP said putting them down a massive amount on someone who got all 4 easiest, which could easily lead to them missing out on an interview, especially at UKCAT cut-off unis like Newcastle and Southampton.

Not a fair way of marking at all IMO.


Oh, gotcha. Yeah that's a pretty big problem.

Quick Reply

Related discussions

Latest

Trending

Trending

Articles for you