Turn on thread page Beta
    • Thread Starter
    Offline

    19
    ReputationRep:
    So even though it seems ages ago, seeing lots of people on the forum fret about their UKCAT and BMAT scores has made me wonder - I know they use them for selection, but I've always wondered how well they actually predict performance. Assuming that's their purpose!

    Is ditching a load of candidates on the basis of one score or another is really effective, or just a numbers game to reduce the applicant pool, for instance. Given that there are both BMAT and UKCAT universities, I suppose it doesn't make such a big impact (except that there are way more UKCAT than BMAT!) but then is it possible to do badly in both when actually you would've been totally fine at medical school?
    Does it not make more sense to say GCSEs are actually the best predictors of long term performance and go with those?

    I'd be curious to know what other people think from their own experience of doing the exams and now being on the other side of it.
    Offline

    10
    ReputationRep:
    I spoke to an academic a year or so back who said that it had been fairly extensively evaluated and did, in their medical school at least, predict a good portion of the variance in preclinical scores.

    GCSEs are a long time before UCAS, especially when a gap year or another degree is thrown in. I think a more recent test is desirable.

    DOI: when I applied the UKCAT was required by many places but not used in selection.
    Offline

    20
    ReputationRep:
    (Original post by seaholme)
    So even though it seems ages ago, seeing lots of people on the forum fret about their UKCAT and BMAT scores has made me wonder - I know they use them for selection, but I've always wondered how well they actually predict performance. Assuming that's their purpose!

    Is ditching a load of candidates on the basis of one score or another is really effective, or just a numbers game to reduce the applicant pool, for instance. Given that there are both BMAT and UKCAT universities, I suppose it doesn't make such a big impact (except that there are way more UKCAT than BMAT!) but then is it possible to do badly in both when actually you would've been totally fine at medical school?
    Does it not make more sense to say GCSEs are actually the best predictors of long term performance and go with those?

    I'd be curious to know what other people think from their own experience of doing the exams and now being on the other side of it.
    As an example, for the first year of Cambridge's preclinical medicine course, the correlations of various factors with exam results are:

    AS UMS: 0.38
    GCSE: 0.27
    BMAT Section 2: 0.26
    BMAT Section 1: 0.19

    And for the second year of Cambridge's preclinical medicine course:

    AS UMS: 0.37
    GCSE: 0.25
    BMAT Section 2: 0.21
    BMAT Section 1: 0.14
    Offline

    4
    ReputationRep:
    All forms of grading are simply ways to cut people out. I think it's fairly obvious that selection tests such as the UKCAT and BMAT are used to make the job of the university easier - I.e. They have it interview less candidates! By the admission of the UKCAT Consortium itself, they know that it cannot accurately predict who will and will not be good medical students and eventually doctors. However, the same can be said for GCSE and other academic cutoffs - the smartest candidate is not necessarily the best potential doctor. Ultimately, I think that cognitive tests like the UKCAT and BMAT are more useful than academic grades than academic tests. They examine traits of a person that academic tests don't, and the question therefore is who makes a better doctor: the candidate who was able to spot patterns well in the UKCAT but only got 89% for A Level Chemistry, or the candidate who was awful at spotting patterns but got 90% in A Level Chemistry? All methods of selection are flawed (and ultimately interviewing everyone would be the ideal, if impractical, solution) however the cognitive tests are perhaps a slightly better indicator than academic grades.
    • TSR Support Team
    Offline

    21
    ReputationRep:
    TSR Support Team
    There is apparently a significant link between UKCAT score and performance on our degree programme (not sure how the latter is measured), more-so than any other factor used in the application process. I suppose this varies on the format of a degree, and might explain why different medical schools have different selection policies (or at least, I really really hope it does explain it and they don't just make it up as they go along...)
    Offline

    13
    ReputationRep:
    A pubmed search for papers with 'UKCAT' in the title yields 11 results. None of them are particularly enlightening on its predictive capacity for medical school performance.

    So unless people are privy to knowledge hitherto undiscovered/unpublished, it's all just conjecture. And whilst it remains conjecture, I'll remain extremely skeptical about it being of any great use.

    EDIT:
    If you search for UK clinical aptitude test instead of UKCAT, you get five results, of which two are studies done at Nottingham that demonstrate no independent predictive power. In isolation it doesn't strictly mean a great deal, but it's a good place to start building a case.
    Offline

    20
    ReputationRep:
    (Original post by Kinkerz)
    A pubmed search for papers with 'UKCAT' in the title yields 11 results. None of them are particularly enlightening on its predictive capacity for medical school performance.

    So unless people are privy to knowledge hitherto undiscovered/unpublished, it's all just conjecture. And whilst it remains conjecture, I'll remain extremely skeptical about it being of any great use.

    EDIT:
    If you search for UK clinical aptitude test instead of UKCAT, you get five results, of which two are studies done at Nottingham that demonstrate no independent predictive power. In isolation it doesn't strictly mean a great deal, but it's a good place to start building a case.
    I imagine if medical schools were researching it, it would just be internal research that wouldn't get published? I'm not really sure though.
    Offline

    13
    ReputationRep:
    (Original post by Chief Wiggum)
    I imagine if medical schools were researching it, it would just be internal research that wouldn't get published? I'm not really sure though.
    No you're quite right. It's certainly possible.

    But if a medical school was conducting such research and had made the decision to use the UKCAT as a substantial factor for selection, surely they'd publish said results and say 'look how progressive we are' or some other such sentiment.

    I don't know. It's been in use for many years now and there are fewer than 20 published articles about it, very few of which are discussing its value as a predictive tool. I regard the proprietary nature of the test and the lack of investigation to be fishy.
    Offline

    10
    ReputationRep:
    (Original post by Kinkerz)
    A pubmed search for papers with 'UKCAT' in the title yields 11 results. None of them are particularly enlightening on its predictive capacity for medical school performance.

    So unless people are privy to knowledge hitherto undiscovered/unpublished, it's all just conjecture. And whilst it remains conjecture, I'll remain extremely skeptical about it being of any great use.

    EDIT:
    If you search for UK clinical aptitude test instead of UKCAT, you get five results, of which two are studies done at Nottingham that demonstrate no independent predictive power. In isolation it doesn't strictly mean a great deal, but it's a good place to start building a case.
    I think its dangerous to make the assumption that all information about its use will/should be published, not least because that raises further questions about what exactly good performance is. The explanation of one set of data I had from the academic responsible was that a score over 750 was a good predictor of getting through the first two years without academic trouble relative to those scoring less. However it appeared to be of no value for the clinical years. Should medical school admissions focus on selecting those who can jump through the academic hoops with minimal fuss to ensure most efficacious use of resources, or look to produce the best clinicians?

    Clearly a balance has to be struck, I feel the UKCAT has a significant role to play here, but we have to be clear about what its purpose is and what we are measuring or trying to predict.
    Offline

    13
    ReputationRep:
    I did really badly in my UKCAT so I don't rate it very highly due to the unrealistic expectations of the paper and its cost. I found with the UKCAT, the only way I could possibly get enough resources and advice, I'd have to invest a lot of money into it.
    Offline

    13
    ReputationRep:
    (Original post by 345rty)
    I think its dangerous to make the assumption that all information about its use will/should be published, not least because that raises further questions about what exactly good performance is.
    I'm not disputing that, but until it is in the public domain we can't know how valid and predictive it is. Only those in medical school admissions departments with spreadsheets of this could.

    And as for a good performance, it seems pretty clear how to measure performance. We have exams in medical school that lead to a ranking and decile.

    The explanation of one set of data I had from the academic responsible was that a score over 750 was a good predictor of getting through the first two years without academic trouble relative to those scoring less. However it appeared to be of no value for the clinical years. Should medical school admissions focus on selecting those who can jump through the academic hoops with minimal fuss to ensure most efficacious use of resources, or look to produce the best clinicians?
    Well it'd be interesting to see whether a score over 750 (unusually good) is also coupled with good A-levels, GCSEs, etc., which could also demonstrate predictive capacity for getting through/scoring well.
    Offline

    19
    ReputationRep:
    (Original post by Kinkerz)
    No you're quite right. It's certainly possible.

    But if a medical school was conducting such research and had made the decision to use the UKCAT as a substantial factor for selection, surely they'd publish said results and say 'look how progressive we are' or some other such sentiment.

    I don't know. It's been in use for many years now and there are fewer than 20 published articles about it, very few of which are discussing its value as a predictive tool. I regard the proprietary nature of the test and the lack of investigation to be fishy.
    I think the first cohort to take it would be FY2 now. Maybe there is research but it's focusing on achievement overall at medical school, in which case they haven't got much data to go on yet.
    Offline

    13
    ReputationRep:
    (Original post by Becca-Sarah)
    I think the first cohort to take it would be FY2 now. Maybe there is research but it's focusing on achievement overall at medical school, in which case they haven't got much data to go on yet.
    Maybe.

    But that's two cohorts who've gone through the entirety of medical school having sat the UKCAT plus another five ongoing cohorts. Seems like a reasonably good sample to me. At least for an initial examination.
    Offline

    19
    ReputationRep:
    (Original post by Kinkerz)
    Maybe.

    But that's two cohorts who've gone through the entirety of medical school having sat the UKCAT plus another five ongoing cohorts. Seems like a reasonably good sample to me. At least for an initial examination.
    True. Aberdeen did a study on the 2007 cohort, might ask if they've done anything with it since.

    Though if we take the purpose of the UKCAT not as to select for the best students, but to level the playing field from background, it seems to be pretty good.

    Will be intriguing in five years to see if the entry SJT scores match up to the exit SJT scores - ie does medical school teach you anything about common sense?
 
 
 
Reply
Submit reply
Turn on thread page Beta
Updated: October 13, 2013

University open days

  • University of Roehampton
    All departments Undergraduate
    Sat, 17 Nov '18
  • Edge Hill University
    Faculty of Health and Social Care Undergraduate
    Sat, 17 Nov '18
  • Bournemouth University
    Undergraduate Open Day Undergraduate
    Sat, 17 Nov '18
Poll
Black Friday: Yay or Nay?

The Student Room, Get Revising and Marked by Teachers are trading names of The Student Room Group Ltd.

Register Number: 04666380 (England and Wales), VAT No. 806 8067 22 Registered Office: International House, Queens Road, Brighton, BN1 3XE

Write a reply...
Reply
Hide
Reputation gems: You get these gems as you gain rep from other members for making good contributions and giving helpful advice.