Join in on our 'Guess the Rep' Competition! >>

This discussion is now closed.

Check out other Related discussions

UMS "Scandal"

Ticki

? Do tell.

You'd think that a UMS mark had a very close correspondence with the raw mark, wouldn't you? Well, you'd be wrong. UMS mark have been inflating for many years and are now miles away from the raw mark in most exams. have a look at the attached graph which shows the divergence of the two for General Studies. You can get 100% UMS with how many raw marks?

image001.gif3.5KB

Scroll to see replies

Reply 1

Anne Elizabeth

grumballcake

But does this go for all subjects or only for GS?

Reply 2

grumballcake

It's similar for other subjects that I've looked at (Business Studies & Psychology). I suspect it's true across the board, although I think Art may be an exception.

Reply 3

grumballcake

ummm - isn't that the whole point of UMS marks - in a difficult exam you need fewer raw marks to get an A than in an easier exam.

Have you looked at the actual figures, or are you just guessing? You see, I've looked at the figures for multiple years in multiple subjects. Still, let's reach across to the pile of examiners' reports I have here...

OCR Business Studies (2872) Jun 2003:

A = 31/45
100% = 37/45

OCR Business Studies (2872) Jun 2004:

A = 31/45
100% = 37/45

OCR Business Studies (2872) Jan 2005:

A = 28/45
100% = 34/45

So, in 2003/4 you needed 82% of the raw marks to get 100% UMS. In 2005 you needed just 76%. Harder exam? Maybe, but still, isn't it a scandal that you can be awarded 100% of the marks for an exam where you got a quarter of it wrong?

In no series (other than Art) that I've looked at have I seen an exam where you needed more than 85% of the raw marks to get 100% UMS. That doesn't mean that there isn't one, merely that I haven't seen one.

you haven't labelled your two data series' with the years

That's because they aren't two years. The straight line is the correlation you'd expect between raw marks and UMS. The other line is the actual results. I can plot similar shaped graphs for the Business Studies data above.

Reply 4

grumballcake

I thought I'd better point out that Art is an exception to this. In AQA Art it's:

Grade___A_ B_ C_ D_ E
Raw (60) 51 44 38 32 26
UMS (90) 72 63 54 45 36

Sot it's not possible to get 100% in Art unless you get 100% raw marks and you need around 85% raw marks to get 80% UMS. Of course, Art is also one of the easier subjects to get an A in (around 27% of people get an A and over half get a B or better). It may be the effect of being a coursework only subject.

Reply 5

minimo

In Edexcel Maths, you often need far, far more than 85% raw marks to get 100 UMS.

Reply 6

grumballcake

Your statsistics seems to simply show that since the introduction of Currisulum 2000 exam papers have become progressively more difficult.

Or the more obvious interpretation (now publicly agreed by exam boards) that there has been steady grade inflation. The exams are the same difficulty, but each year it's got easier to get high UMS marks. The reason given is that examiners always worried about the students who'd miss their offers, so the effective grade boundaries crept down by around 1% per year. Officially that's now stopped, but we'll see.

I'm puzzled why you don't think it's a joke that you could get 100% on a paper where you missed out whole questions.

potentially with UMS an exam could be so easy that a student would need to achive MORE than 100% raw marks to get 100% UMS.

If you look at the AQA Art scores I published, that's actually the case. It's why there's a specific rule that says 100% raw always scores 100% UMS.

Reply 7

Camford

minimo

In Edexcel Maths, you often need far, far more than 85% raw marks to get 100 UMS.

This is maths we are talking about. Maths is the "easiest" subject to get full raw marks in. Not that I got any. But then again, I don't care about marks anymore.

Reply 8

minimo

You didn't get 100 UMS...but got a first at Cambs...gives little me some hope... :smile:

Reply 9

groovy_moose

Sot it's not possible to get 100% in Art unless you get 100% raw marks and you need around 85% raw marks to get 80% UMS. Of course, Art is also one of the easier subjects to get an A in (around 27% of people get an A and over half get a B or better). It may be the effect of being a coursework only subject.

i really couldn't disagree more about art. It's easy not to fail art, but it's not so easy to get a good mark, purely because there's no right answer. Maths is relatively easy to get good marks in because you know how to get them, art is soo much more ambiguous. You say that it's easier because more people get As and Bs, but that is surely because it's the people who are good at art that take it on in the first place. It's the same with further maths. The vast majority of people get As and Bs, not because it's easy, but because only the people that are good at it do it in the first place.

and as for exams getting easier to pass - there's really nothing we can do about it, so if it makes getting 3 As any easier, so be it.

Reply 10

Steveee

I retook Pure 2 maths this year: I was genuinely shocked that I'd got 89/100 UMS (81 first time around), as I'd thought that I'd got full marks or damn near it. I retook it again (playing that system, eh Grumballcake?) and again I'm hoping for 100% UMS (if not 100% raw).

Interestingly: 50/60 = 86% UMS, and 53/60 = 89% UMS in Pure 2, OCR, MEI, 2005 January.

Reply 11

grumballcake

minimo

In Edexcel Maths, you often need far, far more than 85% raw marks to get 100 UMS.

Well, I'd rather hoped you'd need 100% raw to get 100% UMS, but I'm old-fashioned like that. :smile:

Edexcel Maths certainly is better - you generally need 90+% to get 100% UMS. the lowest (6689) is just 83% though.

Reply 12

grumballcake

BTW Just to cover the website in the event of a libel claim - could you please direct me to where the exam boards have publically admitted to grade inflation in A levels.

They'd better start with the Guardian, then...

http://education.guardian.co.uk/ale...,635128,00.html

The government's exam watchdog, the Qualifications and Curriculum Authority, asked Professor Eva Baker, of the University of California, Dr Barry McGaw, deputy director for education at the Organisation for Economic Co-operation and Development, and Edinburgh University vice-chancellor Lord Sutherland to investigate whether standards were being maintained.

In their report, they said there would "always be an error of measurement" for examiners to confront when setting grade boundaries. Because changing the cut-off points by even a mark or two caused large numbers of students to get a better or worse grade than they would have done the previous year, the proportion of candidates getting As could "inch up".

Asked if that showed grade inflation was a reality, as some have claimed, Professor Baker replied: "There's growth from year to year. Sometimes half a percent, sometimes a percent, more students will get an A than would have the previous year. You can add up the influence of that over time and see it would make a difference in the proportion of students attaining As."

Dr McGaw said he had watched examiners debate these issues as part of the panel's year-long standards probe.

"There's a tendency to be generous. People's lives depend on the outcomes. They are faced with the choice of: 'will we make it this percentage or that percentage, will this group get over the boundary or under the boundary'."

QCA chairman Sir William Stubbs said that, in the past, examiners only had to refer changes for consideration by the heads of their boards if a grade boundary moved up or down by 2%. Two months ago, the watchdog decided on a fluctuation rate of zero, he disclosed.

Reply 13

Camford

minimo

You didn't get 100 UMS...but got a first at Cambs...gives little me some hope... :smile:

I have 100 UMS's. Just not raw marks.

Reply 14

grumballcake

I'm puzzled why you consider it a scandal that someone can get 100% UMS while missing whole questions.

I'm puzzled that you don't think that 100% = 100%. When I did A-levels in the 70s, no-one got 100%.

they simply demonstrated knowlegde and application of a subject at the same level as someone with 100% UMS from a previous year.

Except they didn't. Our exams were marked on a curve too, you know. In any two years the ability will not change, so you simply say thata the top 8% get an A etc. That automatically deals with year-to-year variation in exam difficulty, since you're being judged against your peers in each case.

It's not as if anyone expects a student presenting with 100% UMS to have absolute and full knowledge of their entire subject available for instant recall

Non sequitur. No-one expects an exam to test that, nor would it be possible to devise one. However, it is expected that it should be exceptionally hard to get 100% in a well-devised exam and that only the top 1 or 2 people in a year might manage it.

I got 100% in my Cambridge I Latin exam in 1993

Well done, but someone could have achieved last year, that with only 85% of your marks. (Go see what 1% compound growth does over 12 years)

For the sake of all "high achievers" it makes more sense to make the raw mark boundary for full UMS marks comfortably below the 100% line.

As a long term high achiever I want the bar high enough so that only I make it above.

The better you are, the harder you want exams to be, so that it both stretches you intellectually and shows that you're the best.

Are you suggesting that the Olympics should award gold medals to everyone who enters?

Reply 15

grumballcake

groovy_moose

i really couldn't disagree more about art. It's easy not to fail art, but it's not so easy to get a good mark, purely because there's no right answer.

Could I gently explain statistics to you? In Art, 27% of the candidates do get an A. The right answer is to do work that meets the requirements of the specification and around a quarter of people do.

Maths is relatively easy to get good marks in because you know how to get them,

Yet some maths modules have a lower percentage of As than Art. (Note: That's not true for AS modules where sometimes 60% of candidates get an A, which says to me that the exams are too easy).

if it makes getting 3 As any easier, so be it.

"Everyone shall win and all shall have prizes."

University entry becomes a lottery in this scenario. No longer can Cambridge sort out the best by A-level grades, since the top 20% of candidates might have AAA, but they can only take a small proportion of them. What do they do? Well, they either accept that they'll have to lower their standards, or they'll set their own exams (viz. BMAT and LNAT) or demand stretching exams like AEAs. Guess which is more likely.

Reply 16

groovy_moose

what i meant is that if you had a class as mixed as the class that picks up, say, maths, at A Level in art, then it would be harder for a lot of them to make the grade. As it is in my experience it is really only those people who are really good at art that take it in the first place, so it would make sense that there would be a higher proportion of As than in other subjects. Taking, as before, the example of Further maths - 49.5% got an A in the AS level last year, and 58.9% got an A in the A Level - now whilst i certainly don't think it's as hard as it could be, i don't think it canm be dismissed as riduculously easy either. The reason there's so many As, compared to other subjects, is purely the ability level of those taking it.

or demand stretching exams like AEAs. Guess which is more likely.

personally, i don't think that's a bad idea. i took an AEA this year and thought it was far more interesting than the normal A Level exam (although it remain to be seen whether i actually passed it or not :rolleyes:

), infact i remarked that that is how i felt the A Level synoptic paper should have been, but it isn't and i've accepted that and moved on. People can complain about the A Level system all they like, but there's nothing we (those currently taking our A Levels ) can do about it. If exams are slipping that's surely the governments fault, so for now, we'll just have to be happy with what we get from the current system - if i get three As this summer, i'll be extremely pleased, myself (though, even moreso if i get the AEA :biggrin:

)

Reply 17

minimo

Camford

I have 100 UMS's. Just not raw marks.

Reply 18

grumballcake

Of course I don't think 100% UMS = 100% raw marks.
If you assume that you've misunderstood the point of UMS

Well, I'm going from the officialy published literature from the QCA, I'm not just making it up.

Which I'm afraid DOESN'T provide year on year comparability. Some cohorts of students will be statistically brighter than others.

I'm guessing that you haven't studied statistics, to know how ridiculous that statement is. If 20,000 students sit A-levels every year, then that's an enormous sample of people in the population. The chance that the performance will differ significantly between years is minuscule.

You might get variation in a school from year to year because of the small sample size, but across the whole population? Dream on.

UMS conversions are done with reference not ONLY to the bell curve but to comparisons on the ease or difficulty of the exams year on year.

I understand that, but the UMS <-> Raw lines should be extremely close (as indeed they are in the published literature). In fact the calculations of whether it was a 'hard 'exam defend on the assumption that each cohort will be equal in ability.

Peer comparisons are USELESS in the real world

Well, I suspect I've lived in the real world rather longer than you have, and I've also run a quoted plc. I'm talking from my 25+ years of experience as an employer, plus some minor qualifications in management.

The most pernicious effect is that an A from the 70s is not the same as an A from the 90s. We've completely lost year to year comparison. Note that it's happening at degree level too - the proportion of firsts and 2.1s has risen.

And why does awarding full UMS marks to students with less than 100% make any difference to that?

Because it's misleading in the real world. If a surgeon said he'd taken 100% of the instruments out of my abdomen after an operation, I wouldn't expect to find a couple of pairs of forceps left behind. If an aeroplane claims to land with 100% of its passengers safely inside, I wouldn't expect to see a quarter of them spread out dead on the runway.

The introduction of 100% modular A levels has allowed people to measure themselves against the qualifying standard.

The problem is that it used to say who was best, now it doesn't. It's all so that the government and teachers can say how much better things are, despite all the real world evidence to the contrary.

If you honestly feel that this reducing, self selecting pool of bright students should be penalised with lower grades

Lower than what? What we want is consistent standards. Personally, I want to be able to hire the best people and that's easier to establish if we have a framework that's consistent from year to year. I don't want to have to have some sort of discount factor to apply to each year's grades - and nor do Cambridge tutors.

personally I'd rather get on with life than trying to put other people down

However, that hasn't stop your making snide comments, has it? I haven't tried to put anyone down - I'm simply living in a real multi-national world, where Britian can't just tell itself that its students are the best.

Reply 19

kellywood_5

grumballcake

Note that it's happening at degree level too - the proportion of firsts and 2.1s has risen.

Sorry to butt in on your debate here :redface:

but can I just ask, why is it that whenever any form of exam results, be they SATs, GCSEs, A-levels, or degrees, are published, we always hear comments like 'more people are getting level 7s/A*s/As/1sts, so that must mean exams are getting easier?' Why can't it just be that teaching standards are improving, students are getting better at knowing what the examiners are looking for through examiners reports from previous years, and students are working harder due to the increased importance of qualifications? After all, if exam results didn't improve every year, people would complain that standards were slipping. You can't win :p:

Related discussions

Latest

Trending

Try out the app

Continue on web