What is a representative sample?

a sample with the same characteristics as the target population. It is a small copy of the whole group.

something that distorts findings in one direction. Leading questions are one source of bias.

What is the observer effect?

people act differently because they know they are being watched. It is another name for the Hawthorne effect.

What does generalisability mean?

the findings can be applied to the whole target population. It depends on the sample being representative.

A study gives the same results every time it is repeated, but it measures the wrong thing. How should it be described?

reliable but not valid. Repeatable results can still be untrue to life.

Judging research: validity, reliability and bias

In plain words

Before you believe a study, ask four questions. Is it true to life? Would it come out the same again? Do the people studied look like everyone else? And did anything push the answers in one direction?

7 things to know

Validity: the research gives a true picture of what it set out to study. Reliability: if the research is repeated in the same way, it gives the same results.
Representativeness: the sample has the same characteristics as the target population, in the same proportions, such as the same mix of ages and genders.
Generalisability: the findings from the sample can be applied to the whole target population. Only a representative sample allows this.
Bias is anything that distorts the findings in one direction. It can come from the researcher's own opinions, from the wording of questions, or from the way the sample was chosen.
The interviewer effect: the presence of the interviewer changes the answers given. A person's answers may be shaped by the interviewer's age, gender, ethnicity or manner, or by a wish to look good.
The Hawthorne effect, or observer effect: people change their behaviour because they know they are being watched or studied.
These problems lower validity, because the researcher is no longer recording what people really think or do.

Worked example

A male researcher in a suit interviews teenage girls at school about whether they have ever shoplifted. Evaluate the research.

Interviewer effect: the girls may not admit shoplifting to an adult man who looks like a figure of authority.
So the answers may be untrue, and validity is low.
If only girls from one school are asked, the sample is not representative, so the findings cannot be generalised.
Improvement: an anonymous questionnaire, given to a sample from several schools.

Tips and tricks

To evaluate any method, run through the checklist: validity, reliability, representativeness, ethics, practical issues. Something will apply.
Representative describes the sample. Generalisable describes the findings. You need the first to claim the second.

5 questions, about 2 minutes.

It lands in your notebook with its questions as flashcards.

Judging research: validity, reliability and bias: 5 questions and answers

These are the quiz’s questions. Do the quiz first, then come back here for the ones that got you.

What is a representative sample?
- a sample of volunteers
- a sample with the same characteristics as the target population (the answer)
- a very small sample
- a sample chosen by the people in it
It is a small copy of the whole group.
What is bias?
- a kind of sample
- a list of the population
- something that distorts findings in one direction (the answer)
- a type of interview
Leading questions are one source of bias.
What is the observer effect?
- people act differently because they know they are being watched (the answer)
- the observer forgets what was seen
- the observer joins the group
- people refuse to be observed
It is another name for the Hawthorne effect.
What does generalisability mean?
- the findings are detailed
- the research was cheap
- the research can be repeated
- the findings can be applied to the whole target population (the answer)
It depends on the sample being representative.
A study gives the same results every time it is repeated, but it measures the wrong thing. How should it be described?Stretch
- valid but not reliable
- reliable but not valid (the answer)
- neither valid nor reliable
- both valid and reliable
Repeatable results can still be untrue to life.

5 questions

Tap an answer and you’ll see straight away whether it’s right, and why.

Worksheet

3 questions, 10 marks. Write your answers on paper, then check them.

Judging research: validity, reliability and bias

Cambridge IGCSE Sociology 0495 · 10 marks · papermunch.org

Name ______________________________ Date ______________

What is meant by the term "Hawthorne effect"?[2]
Show answerHide answer
People changing their behaviour because they know they are being studied or observed.
I got this one
Explain the difference between validity and reliability.[4]
Show answerHide answer
Validity means the research gives a true and accurate picture of what is being studied. Reliability means that the research could be repeated by another researcher in the same way and would give the same results.
I got this one
Explain how the interviewer effect can reduce the validity of research.[4]
Show answerHide answer
The respondent's answers are influenced by the interviewer, for example by their age, gender, ethnicity or manner. The respondent may give answers they think the interviewer wants, or that make them look good. The answers are then not a true picture, so validity is reduced.
I got this one

Answers: Judging research: validity, reliability and bias

1. People changing their behaviour because they know they are being studied or observed.
2. Validity means the research gives a true and accurate picture of what is being studied. Reliability means that the research could be repeated by another researcher in the same way and would give the same results.
3. The respondent's answers are influenced by the interviewer, for example by their age, gender, ethnicity or manner. The respondent may give answers they think the interviewer wants, or that make them look good. The answers are then not a true picture, so validity is reduced.