I don't really agree with your criticisms. It sounds to me like you only read the abstract! The authors clearly state:
Items 1-8 of the checklist are as follows:
1. Being overpowered or forced by a man to surrender sexually against my will.
2. Being overpowered or forced by a woman to surrender sexually against my will.
3. Being overpowered or forced into a sexual act against my will because I was incapacitated due to drugs, alcohol, or sleep.
4. Being overpowered or forced by a man to give him oral sex.
5. Being overpowered or forced by a woman to give her oral sex.
6. Being overpowered or forced to have anal sex.
7. Being raped by a man.
8. Being raped by a woman.
I don't think those definitions are at all arbitrary, and I think they're a very comprehensive view of how someone might view their fantasy. Subjects were required to answer all 8 questions on a scale from 0 (never) to 6 (several times a day). For the other methods of measure used in the study (the sexual fantasy log and Kanin's question), the definition was equally clear. Either using the word "rape" directly, or using the legal definition of rape as suggests above.
The authors don't extrapolate their data outside of the population sample (US Southwestern female undergraduates), so I don't think your 2nd criticism is really a valid appraisal of the study. This sort of limitation is the case with almost
all human studies and is not a flaw in the study design - subsequent extrapolation from the population sample is a flaw of the data interpretor. In fact, the authors address this themselves:
3) I totally agree! I don't doubt that self-selection bias has systematically raised the reported prevalence (this is why repetition and consistency are so important in epidemiological studies!), but researchers incentivised participation with extra credit so as to attempt to lessen this effect - although it still remains significant.
4) Not at all. When someone wishes to measure prevalence and frequency, I don't think such methods are limiting at all! The authors want to investigate the
existence of rape fantasies - and with their comprehensive definitions - I think they do this well. Epidemiological studies do similar when trying to measure the prevalence of disease: defining a case, then simply looking at databases (or even sometimes surveying subjects) to see if they meet the criteria. This is what's been done here.
I think the most interesting finding are here:
However, the 62% figure is conflated by the combination of different items in the checklist. To give raw results:
Item 1: 62% reported having this fantasy
Item 2: 18% reported having this fantasy
Item 3: 25% reported having this fantasy
Item 4: 28% reported having this fantasy
Item 5: 10% reported having this fantasy
Item 6: 17% reported having this fantasy
Item 7: 33% reported having this fantasy
Item 8: 10% reported having this fantasy
The raw results give an interesting perspective on how the subjects view what "rape" actually is, imo. Given the disparity between item 7 directly asking "how often do you fantasise about being raped by a man?" and item 1 that asks "how often do you fantasise asking about being forced to surrender sexually to a man against your will?"
However, when subjects were asked to discuss a sexual fantasy that involved the use of coercion or aggression. Only a quarter of women wrote about, what qualified in the case definition, as a rape fantasy.
Interesting study, imo. I think the study is totally valid, but the measurements of prevalence and frequency have been artificially increased by the biases present and the statistical analysis used. Would like to see other authors replicate the same study.
All definitions used by the authors explicitly note "lack of consent" as a part of the criteria, with the exception of items 4 - 7, which introduces a potential source of ambiguity for the study subjects.