Measuring Rationality

I was wondering, if you had to measure someone's rationality, how would you go about it?

Firstly I thought maybe having a collection of previous experiments, studies, games, puzzles etc that have been carried out and basing a score around this might be a good idea... but this would just seem like a simple quiz.

Is there a method in measuring rationality that would have concrete evidence backing it without it being a simple quiz?

Reply 1

psychtutormentor

How are you operationalising "rationality"...?

Reply 2

cambio wechsel

20

A good place to start would be in reading this terrific book, likely available from your university library: http://www.amazon.co.uk/Irrationality-Stuart-Sutherland/dp/1905177070

The book details >100 academic studies designed to test subjects' rationality.

Reply 3

Daisuke

OP

Original post by psychtutormentor

How are you operationalising "rationality"...?

Well, I'm planning on telling the user to "get the highest possible score"... So their actions should be based around "maximising benefits, minimising cost". For "sealed bidding auction" games and things like that, there's obviously a direct relation in which the user can be "rational"... if that makes sense.

Original post by cambio wechsel

A good place to start would be in reading this terrific book, likely available from your university library: http://www.amazon.co.uk/Irrationality-Stuart-Sutherland/dp/1905177070

The book details >100 academic studies designed to test subjects' rationality.

Have you a copy of this book? Can you give me a bit more detail, maybe an example? Much appreciated!
It is not available at my library so I may have to purchase it.

more context: I'm doing Computer Science at my university and my project is to create an application that measures rationality... it's very difficult however as trying to create a complex computer science project on something I'm not too sure about is proving... difficult.

(edited 11 years ago)

Reply 4

Psych!

Perhaps use a measure of delay discounting. As you're coming from the CS angle, the math aspect and associated coding would be relatively impressive?

Reply 5

cambio wechsel

20

Original post by Daisuke

Have you a copy of this book? Can you give me a bit more detail, maybe an example? Much appreciated!
It is not available at my library so I may have to purchase it.

I haven't. I got if from the library. Even if its' not available at your university's library (and I'm amazed that it is not, that's a famous text) they will be able to get it on inter-library loan.

The book details a great many famous experiments done on this theme. To give the first example which pops into my noggin:

Person A and Person B are given £10 to share.
The deal is that Person A proposes the split.
If person B agrees to the proposed division, it is made.
There is no room for negotiation: this is a one shot deal.
If B does not agree, neither gets anything.

Now, what's interesting in the conducting of the experiment is that if B does not consider that the split proposed is pretty fair, he will often refuse it, even though this is done at cost to himself.

Example:
Person A proposes that he should get £9 and B £1.
Person B refuses, thereby preferring to have nothing than £1.
Experimenter keeps the money.

I think that often in this case, person A is a stooge.

Reply 6

Daisuke

OP

Sorry I forgot to include, one of the objectives of this application is to educate the user in rationality... important factor to leave out.

Original post by Psych!

Perhaps use a measure of delay discounting. As you're coming from the CS angle, the math aspect and associated coding would be relatively impressive?

By delay discounting do you mean using their response time in scoring?
But that's exactly what I'm especially struggling with, trying to use aspects of CS in a relatively simple application that simply asks questions... I was going to make some kind of "Rational AI", however I was told this would have been too subjective as it is MY interpretation of how "AI" would act Rational... so that went out the window.

I was also considering making some kind of classification of the questions. Since one of my objectives is to educate the user in rationality, asking them questions they are likely to get wrong would be best, rather than asking them questions that they are likely to get correct... (somehow different aspects of rationality).

Original post by cambio wechsel

I haven't. I got if from the library. Even if its' not available at your university's library (and I'm amazed that it is not, that's a famous text) they will be able to get it on inter-library loan.

The book details a great many famous experiments done on this theme. To give the first example which pops into my noggin:

Person A and Person B are given £10 to share.
The deal is that Person A proposes the split.
If person B agrees to the proposed division, it is made.
There is no room for negotiation: this is a one shot deal.
If B does not agree, neither gets anything.

Now, what's interesting in the conducting of the experiment is that if B does not consider that the split proposed is pretty fair, he will often refuse it, even though this is done at cost to himself.

Example:
Person A proposes that he should get £9 and B £1.
Person B refuses, thereby preferring to have nothing than £1.
Experimenter keeps the money.

I think that often in this case, person A is a stooge.

Ah that's great, another challenge is trying to find ENOUGH examples that I can base my game off, as I am NOT allowed to make up my own - it'd become subjective as I am not capable of running enough tests etc to support my assumptions.

Really appreciate the suggestions - I've asked numerous other places with no avail so glad I'm finally getting some input!

Reply 7

cambio wechsel

20

Original post by Daisuke

[
Ah that's great, another challenge is trying to find ENOUGH examples that I can base my game off, as I am NOT allowed to make up my own - it'd become subjective as I am not capable of running enough tests etc to support my assumptions.

The big names here are Tversky and Kahneman, who good as invented behavioural economics as a field of experimental enquiry.

http://en.wikipedia.org/wiki/Daniel_Kahneman

Reply 8

Joinedup

20

Kahneman's got a book 'Thinking fast and slow' out that goes over a lot of the research. tons of them in the bookshops and libraries at the moment.