Hey there! Sign in to join this conversationNew here? Join for free

Is there any evidence of correlation between UKCAT scores and BMAT scores? Watch

    • Thread Starter
    Offline

    8
    ReputationRep:
    (Original post by nexttime)
    There's something wrong with how you calculated the BMAT scores. They're lower than they should be if you're wanting to use the Oxford system.

    If its a consistent error it might not matter, but if its not or it fails to weigh the sections correctly it might make a difference.

    Edit: This formula (for row 35) gives the right answer: =((E35+F35-2)*5+(G35*2+H35)*4/3)
    Thank you. You are absolutely right. Thank you for checking.
    • Thread Starter
    Offline

    8
    ReputationRep:
    Corrected spreadsheet.

    I hope this is correct

    BMAT scores v UKCAT scores for medicine candidates who submitted their scores on the profiles thread 2014-2016.
    Attached Files
  1. File Type: xlsx ukcat v bmat redacted v2.xlsx (41.1 KB, 47 views)
    • TSR Support Team
    Offline

    19
    ReputationRep:
    TSR Support Team
    Quick and dirty analysis

    I reckon the lack of adjusted significance is type 2 error (n=40).
    Attached Images
     
    • Thread Starter
    Offline

    8
    ReputationRep:
    (Original post by Beska)
    Quick and dirty analysis

    I reckon the lack of adjusted significance is type 2 error (n=40).
    Thanks for this. Had to look up what this meant! Essentially there is something there but the data is failing to prove it. Correct?
    • TSR Support Team
    Offline

    19
    ReputationRep:
    TSR Support Team
    (Original post by rugose)
    Thanks for this. Had to look up what this meant! Essentially there is something there but the data is failing to prove it. Correct?
    Yeh. It's significant in the unadjusted analysis, but when controlled for year of entry it becomes insignificant. When further controlled for A* % it becomes even more insignificant. Although those two things are likely true confounders, the sample size is quite small and the more things you adjust for the more likely you're are to get an insignificant p value just due to the underlying stats.
    Offline

    2
    ReputationRep:
    There is actually lots of research to whether the UKCAT leads to better medical school applicant selection. If you go on the UKCAT website, you can find more than 20 research papers from many different institutions.
    • TSR Support Team
    Offline

    19
    ReputationRep:
    TSR Support Team
    (Original post by MoshBosh)
    There is actually lots of research to whether the UKCAT leads to better medical school applicant selection. If you go on the UKCAT website, you can find more than 20 research papers from many different institutions.
    Yup - our questions is different though. We're wondering if UKCAT correlates with BMAT.
    Offline

    20
    ReputationRep:
    (Original post by rugose)
    Thanks for this. Had to look up what this meant! Essentially there is something there but the data is failing to prove it. Correct?
    I don't think the adjustments detract from the overall conclusion for our purposes: the UKCAT certainly does correlate with the BMAT and this data does "prove" it (or at least, is unlikely to have occurred randomly).
    • TSR Support Team
    Offline

    19
    ReputationRep:
    TSR Support Team
    (Original post by nexttime)
    I don't think the adjustments detract from the overall conclusion for our purposes: the UKCAT certainly does correlate with the BMAT.
    Yeh but I wouldn't be confident to conclude that, especially since there's lots of different years (therefore different exams - e.g. 2013 UKCAT correlating with 2016 BMAT) being correlated. That's my main worry. The % A* is more interesting as adjusting for general previous performance - might be more interesting as a sub-group analysis but not enough numbers (i.e. it may only correlate in those with high prior performance, whereas low prior performance it may not as UKCAT scores would be higher than BMAT scores... which is what I suspect)
    Offline

    20
    ReputationRep:
    (Original post by Beska)
    Yeh but I wouldn't be confident to conclude that, especially since there's lots of different years (therefore different exams - e.g. 2013 UKCAT correlating with 2016 BMAT) being correlated. That's my main worry.
    Its been a lot time since I did stats but... That's not what you compared though. There are no data points with a 2013 UKCAT and 2016 BMAT (I presume), which would result in a systematic error - you've just got data points that cover a period of time but are still inherently part of what you are comparing, your key outcome.

    If my understanding is correct, what we're doing is trying to predict the BMAT result. By adjusting for the year what you've done is show that on average some years had a better BMAT than others, and that if you eliminate that variation the predictive value is a little less. You could do the same for month taken, or cell number, or a random number generator.

    For the purposes of a university looking to do a study on what different factors predict BMAT result I think you'd need a larger sample size and a multiple regression analysis etc.

    For our simple question: do those that have a high UKCAT also tend to have a higher BMAT, you've got your answer. Whether that's because of their higher GCSEs or the year they took it or whether they have a pet cat or not is not part of the remit!

    The % A* is more interesting as adjusting for general previous performance - might be more interesting as a sub-group analysis but not enough numbers (i.e. it may only correlate in those with high prior performance, whereas low prior performance it may not as UKCAT scores would be higher than BMAT scores... which is what I suspect)
    Interesting theory.
    • TSR Support Team
    Offline

    19
    ReputationRep:
    TSR Support Team
    (Original post by nexttime)
    Its been a lot time since I did stats but... That's not what you compared though. There are no data points with a 2013 UKCAT and 2016 BMAT (I presume), which would result in a systematic error - you've just got data points that cover a period of time but are still inherently part of what you are comparing, your key outcome.

    If my understanding is correct, what we're doing is trying to predict the BMAT result. By adjusting for the year what you've done is show that on average some years had a better BMAT than others, and that if you eliminate that variation the predictive value is a little less. You could do the same for month taken, or cell number, or a random number generator.

    For the purposes of a university looking to do a study on what different factors predict BMAT result I think you'd need a larger sample size and a multiple regression analysis etc.

    For our simple question: do those that have a high UKCAT also tend to have a higher BMAT, you've got your answer. Whether that's because of their higher GCSEs or the year they took it or whether they have a pet cat or not is not part of the remit!



    Interesting theory.

    Hmmm. Yes. Fair enough (the adjustment for year and % A* were multiple regression though - but my concern about sample size still there).
    • Thread Starter
    Offline

    8
    ReputationRep:
    Is it worth putting in the other 80 data points? ( the image I posted earlier was largelly those plots.) or have we got as much as we are likely to get from this exercise.
 
 
 
  • See more of what you like on The Student Room

    You can personalise what you see on TSR. Tell us a little about yourself to get started.

  • Poll
    Brussels sprouts
  • See more of what you like on The Student Room

    You can personalise what you see on TSR. Tell us a little about yourself to get started.

  • The Student Room, Get Revising and Marked by Teachers are trading names of The Student Room Group Ltd.

    Register Number: 04666380 (England and Wales), VAT No. 806 8067 22 Registered Office: International House, Queens Road, Brighton, BN1 3XE

    Quick reply
    Reputation gems: You get these gems as you gain rep from other members for making good contributions and giving helpful advice.