Pilot studies | AQA A-Level Psychology Revision
- Revision Notes
- Aug 4
- 12 min read
Updated: 6 days ago
For 7182 specification, first teach in September 2025
AQA A-Level Psychology | Free Revision Notes
Estimated study time: 35 minutes
Pilot studies are small-scale trial runs completed before a full psychological investigation. This Pilot studies A-Level Psychology revision page explains why researchers pilot their procedures, the practical and methodological problems a pilot may reveal, and how findings can be used to improve the main study. Pilot studies form part of the scientific processes section of AQA A-Level Psychology research methods, where students must understand both pilot studies and the aims of piloting.
Learning Objectives 🎯
By the end of this revision page, you should be able to:
Define a pilot study.
Explain the main aims of piloting a psychological investigation.
Identify practical, methodological and ethical problems revealed by a pilot study.
Apply knowledge of piloting to experiments, observations, questionnaires and interviews.
Suggest specific improvements based on pilot findings.
Explain how piloting may help researchers improve the quality of a full investigation.
Revision Notes 📚
What is a pilot study?
A pilot study is a small-scale trial run of an investigation that takes place before the full study is conducted.
The researcher follows the proposed procedure, uses the planned materials and collects trial data. This allows the researcher to check whether the study works as intended before committing time and resources to the full investigation.
A pilot study is not normally conducted to provide the final answer to the research question. Its main purpose is to identify problems and refine the research procedure.
Piloting should take place after the researcher has developed a clear research aim. Understanding how an investigation’s purpose is stated will therefore help you see where piloting fits within the research process.
The main aims of piloting
The overall aim of piloting is to test and improve the design of an investigation before the main study begins.
A pilot study may be used to check:
Whether the instructions are clear.
Whether participants understand what they are required to do.
Whether questions or tasks are suitable.
Whether the procedure takes an appropriate amount of time.
Whether equipment and research materials work correctly.
Whether variables have been operationalised clearly.
Whether extraneous variables have been controlled.
Whether the procedure creates demand characteristics or investigator effects.
Whether an observational coding system can be used consistently.
Whether the proposed procedure raises practical or ethical problems.
Piloting does not guarantee that the final study will be free from problems. However, it gives the researcher an opportunity to detect weaknesses and make evidence-based improvements.
Checking instructions and standardisation
Participants should receive instructions that are clear enough for them to understand but do not reveal unnecessary information about the investigation.
A pilot study may reveal that:
Some instructions are ambiguous.
Participants repeatedly ask the same questions.
Different participants interpret the task differently.
Different researchers explain the procedure in different ways.
Important stages of the procedure have not been described.
These problems may reduce standardisation, meaning that participants do not all experience the same procedure.
The researcher could improve the study by:
Rewriting unclear instructions.
Adding necessary details.
Removing confusing wording.
Producing a standardised script.
Training investigators to follow the same procedure.
These changes are closely connected to the use of standardisation and randomisation.
Checking the operationalisation of variables
Operationalisation means defining a variable in a precise and measurable way.
For example, a researcher studying concentration must decide exactly how concentration will be measured. It might be measured through the number of correct responses on a particular task completed within a stated time.
A pilot study may reveal that:
The dependent variable is difficult to measure.
Participants achieve almost identical scores.
The task is too easy or too difficult.
The measure does not record the intended behaviour clearly.
Researchers disagree about how a response should be scored.
The researcher may respond by:
Changing the task.
Revising the scoring system.
Providing clearer scoring criteria.
Making the measure more sensitive to differences between participants.
Defining the variable more precisely.
This applies knowledge from identifying and defining research variables.
Checking timing and practicality
A procedure that appears reasonable in a research plan may not work smoothly in practice.
A pilot study can show whether:
The study takes longer than expected.
Participants become tired or lose concentration.
There is not enough time to complete all parts of the procedure.
Equipment takes too long to prepare.
The testing location is unsuitable.
Instructions or materials are difficult to distribute.
The researcher could then:
Shorten or simplify the task.
Reduce unnecessary stages.
Allow more time for completion.
Rearrange the order of the procedure.
Change or test the equipment.
Use a more suitable testing environment.
When suggesting an improvement in an examination, make it specific. Saying “improve the procedure” is too vague. Explain exactly what should be changed and why.
Piloting questionnaires
A pilot questionnaire allows researchers to check whether each question produces useful and understandable responses.
This is particularly important when researchers are deciding how to use open and closed questions. You can revise these distinctions in designing effective self-report questions.
A pilot may reveal that:
A question is unclear or ambiguous.
Participants interpret a word in different ways.
Two questions appear to ask the same thing.
A closed question does not include an appropriate response option.
Participants leave a question unanswered.
The questionnaire is too long.
An open question produces responses that are difficult to analyse.
The order of questions influences later responses.
Possible improvements include:
Rewording an ambiguous question.
Replacing technical language with clearer wording.
Adding an appropriate response category.
Removing repeated or unnecessary questions.
Changing the order of questions.
Giving clearer instructions about how to respond.
Reducing the length of the questionnaire.
Example
A questionnaire asks:
“Do you regularly experience excessive academic pressure?”
During the pilot, several students ask what “regularly” and “excessive” mean.
This reveals that the terms have not been operationalised clearly. The researcher could replace the question with one that specifies a time period and measurable frequency, such as asking how many days during the previous school week the participant felt under academic pressure.
Piloting interviews
Piloting an interview can help a researcher determine whether the questions and interview procedure work as intended.
Understanding how structured and unstructured interviews differ will help you suggest appropriate improvements.
A pilot interview may show that:
Questions are difficult to understand.
Questions encourage very brief or irrelevant answers.
The order of questions disrupts the conversation.
The interviewer unintentionally leads participants.
Different interviewers use different prompts.
The interview takes too long.
Recording equipment does not capture responses clearly.
Possible improvements include:
Rewording questions.
Changing the order of questions.
Removing leading language.
Preparing standardised prompts for a structured interview.
Training the interviewer.
Testing or replacing recording equipment.
Reducing the number of questions.
Piloting an observation
During a pilot observation, researchers can test whether their proposed behavioural categories and sampling procedures allow behaviour to be recorded accurately.
Behavioural categories should be:
Clear.
Observable.
Appropriate to the research aim.
Distinct from one another.
A pilot may reveal that:
Two categories overlap.
A common behaviour has no category.
A category depends on subjective interpretation.
Too many behaviours occur at once to record them accurately.
The time-sampling interval is unsuitable.
The observers disagree about how to classify behaviour.
Possible improvements include:
Redefining vague categories.
Combining overlapping categories.
Adding a category for an important behaviour.
Removing categories that cannot be observed reliably.
Revising the time-sampling interval.
Training observers using examples of each category.
These decisions build directly on constructing behavioural categories and selecting sampling procedures.
Example
Researchers plan to record the following categories during a group task:
“Talking”
“Helping”
“Giving advice”
During the pilot, observers disagree about whether giving advice should be recorded as talking, helping or both.
The categories overlap. The researchers should provide more precise definitions or redesign the categories so that each observed behaviour is classified consistently.
Identifying extraneous variables
An extraneous variable is any variable other than the independent variable that could affect the dependent variable.
A pilot study may reveal unexpected influences such as:
Noise outside the testing room.
Differences in lighting.
Interruptions.
Participants using different equipment.
Researchers giving different amounts of help.
Differences in the time available to complete the task.
The researcher could:
Use the same room for all participants.
Remove interruptions.
Use identical equipment.
Standardise the time allowed.
Provide the same instructions and assistance.
Keep environmental conditions as similar as possible.
The improvement should address the exact extraneous variable identified during the pilot.
Identifying demand characteristics and investigator effects
Demand characteristics occur when participants notice features of an investigation that suggest its purpose. Participants may then change their behaviour because they think they know what the researcher expects.
Investigator effects occur when a researcher’s behaviour or characteristics influence participants or the recording of results.
A pilot study might reveal that:
Participants correctly guess the aim.
The title of a task reveals the expected outcome.
The researcher gives additional encouragement in one condition.
Tone of voice differs between participants.
An observer’s expectations influence how behaviour is recorded.
Possible improvements include:
Removing obvious clues about the aim.
Using neutral instructions.
Standardising interactions between the researcher and participants.
Using clearer behavioural categories.
Training investigators.
Using procedures that reduce opportunities for researchers to influence responses.
These problems are explored further in recognising participant and researcher influences.
Identifying ethical concerns
A pilot study may reveal ethical issues that were not obvious during the planning stage.
For example:
Participants may find a question more upsetting than expected.
Instructions about the right to withdraw may be unclear.
Personal information may be visible to other participants.
A task may create greater discomfort than anticipated.
Debriefing information may not explain the investigation sufficiently.
The researcher should revise the procedure before the full study. Improvements could include:
Removing or rewording an upsetting question.
Making the right to withdraw clearer.
Protecting confidential information.
Reducing unnecessary discomfort.
Improving the debrief.
Changing the procedure so that ethical issues are dealt with appropriately.
Using pilot findings to improve a study
An effective improvement must follow logically from the problem identified.
Pilot finding | Problem revealed | Suitable improvement |
Participants repeatedly ask what an instruction means | Instructions are unclear | Rewrite the instruction using precise wording |
Most participants score full marks | The task may be too easy to detect differences | Increase the difficulty of the task |
Two observers classify the same behaviour differently | Behavioural categories are unclear or overlapping | Redefine the categories and train observers |
One condition takes longer than the other | Time may act as an extraneous variable | Standardise the time allowed in each condition |
Participants guess the purpose of the study | Demand characteristics may affect behaviour | Remove clues and use more neutral instructions |
Several questionnaire items are left blank | Items may be confusing or unsuitable | Reword, replace or remove the unanswered items |
A recording device misses some responses | Data collection may be incomplete | Test, reposition or replace the equipment |
Participants report discomfort during part of the procedure | The procedure may create an ethical concern | Modify or remove the distressing part of the procedure |
A simple piloting process
A researcher could use the following process:
Prepare the proposed procedure and materials.
Conduct a small-scale trial using the planned method.
Observe how the procedure works.
Examine the trial data and any problems that occurred.
Identify unclear, unreliable, impractical or unethical elements.
Make specific changes.
Repeat the pilot if necessary.
Conduct the full investigation using the revised procedure.
Piloting is therefore an important part of planning a complete psychological study.
How pilot studies may improve research quality
Piloting may help improve the quality of research by allowing weaknesses to be corrected before the full investigation.
It may contribute to improved reliability where it leads to:
Clearer instructions.
More standardised procedures.
Better-defined behavioural categories.
More consistent scoring systems.
Improved equipment or recording methods.
It may contribute to improved validity where it leads to:
Better operationalisation of variables.
Fewer demand characteristics.
Better control of extraneous variables.
Questions that measure what the researcher intends to investigate.
A procedure that more accurately tests the research aim.
However, a pilot study does not automatically make an investigation reliable or valid. Its value depends on whether the researcher correctly identifies problems and makes suitable improvements.
Applying pilot studies in examination questions
Pilot studies are often assessed through an unfamiliar research scenario. You may be given a description of a proposed investigation or findings from a pilot and asked to identify problems or recommend changes.
Pay close attention to the command word:
Identify means select or name the problem.
Explain means make the nature or consequence of the problem clear.
Suggest means propose a suitable and practical improvement.
Justify means explain why the suggested improvement would help.
A strong applied response normally contains three connected elements:
The specific pilot finding.
The research problem it reveals.
A precise improvement that addresses that problem.
For example:
Several participants asked the researcher to explain the meaning of Question 4. This suggests that the wording was unclear. The researcher should rewrite Question 4 using more precise and accessible language so that all participants interpret it in the same way.
Key Words 🔑
Key word | Student-friendly definition | How it may be used in an exam |
Pilot study | A small-scale trial run conducted before the full investigation. | Define the term or explain why a researcher should conduct one. |
Piloting | Testing a proposed research procedure to identify problems and make improvements. | Explain the aims of piloting or apply it to a scenario. |
Standardisation | Keeping instructions, materials and procedures consistent for participants. | Suggest how inconsistency revealed by a pilot could be reduced. |
Operationalisation | Defining a variable so that it can be manipulated or measured precisely. | Explain why a task or measure identified during a pilot is unclear. |
Extraneous variable | A variable other than the independent variable that could affect the dependent variable. | Identify an uncontrolled factor revealed by a pilot study. |
Demand characteristics | Features of a study that allow participants to guess its purpose and alter their behaviour. | Explain why participants behaved differently during a pilot. |
Investigator effects | Ways in which a researcher may unintentionally influence participants or results. | Suggest greater standardisation or investigator training. |
Behavioural category | A clearly defined group of observable behaviours used to record an observation. | Improve categories that overlap or are interpreted inconsistently. |
Reliability | The consistency of a measurement or research procedure. | Explain how clearer procedures or categories may improve consistency. |
Validity | The extent to which an investigation measures what it intends to measure. | Explain how improved operationalisation or control may improve the study. |
Common Mistakes ⚠️
Mistake: Saying that the purpose of a pilot study is to produce the final research findings.
Why this is incorrect:A pilot study is a trial run used to check and improve the proposed investigation. It is not primarily intended to provide the final answer to the research question.
How to improve:Refer to identifying problems, testing procedures and making changes before the full study.
Mistake: Describing a pilot study as a sampling method.
Why this is incorrect:Sampling methods determine how participants are selected. A pilot study tests whether a research procedure works as intended.
How to improve:Keep the concepts separate. A pilot study may use participants, but its defining feature is that it is a trial of the investigation.
Mistake: Giving a vague improvement such as “make the study better”.
Why this is incorrect:The response does not identify what should change or how the change addresses the problem.
How to improve:Name a precise modification, such as rewriting an ambiguous question, standardising the instructions or redefining an overlapping behavioural category.
Mistake: Identifying a problem without explaining its effect.
Why this is incorrect:For an explanation question, naming a problem is not enough. The answer should show why it matters.
How to improve:Develop the point. For example, explain that unclear instructions may cause participants to complete different versions of the task, reducing standardisation.
Mistake: Suggesting an improvement that does not address the pilot finding.
Why this is incorrect:The improvement must logically follow from the identified problem.
How to improve:Use a clear chain: pilot finding → problem → targeted improvement.
Mistake: Claiming that conducting a pilot automatically produces valid and reliable results.
Why this is incorrect:A pilot only reveals potential weaknesses. Research quality improves only if suitable changes are made.
How to improve:Use cautious language, such as “may improve reliability by allowing the researcher to standardise the instructions”.
Mistake: Confusing unclear operationalisation with an extraneous variable.
Why this is incorrect:Poor operationalisation concerns how a variable has been defined or measured. An extraneous variable is a separate factor that may affect the dependent variable.
How to improve:Ask whether the problem concerns the definition of the measure or an additional uncontrolled influence.
Exam-Style Questions ✍️
Question 1
Define a pilot study. (1 mark)
Question 2
State two aims of piloting a psychological investigation. (2 marks)
Question 3
Explain one reason why a researcher might pilot a questionnaire before using it in a full investigation. (2 marks)
Question 4
A researcher is investigating whether background noise affects performance on a memory task. During a pilot study, several participants ask whether they should answer the questions as quickly or as accurately as possible.
Identify one problem revealed by the pilot study and suggest one improvement. (3 marks)
Question 5
A psychologist pilots a questionnaire about students’ use of social media. The following findings are recorded:
Pilot finding | Number of pilot participants affected |
Asked what “frequent use” meant | 6 |
Selected two answers for a question requiring one answer | 4 |
Left the final three questions unanswered | 5 |
Completed the questionnaire without reporting a problem | 2 |
Using the information in the table, suggest two changes that the psychologist should make before conducting the full study. Explain why each change would be useful. (4 marks)
Question 6
Two observers are piloting an observation of behaviour in a school common room. They use the following behavioural categories:
talking to another student
helping another student
giving advice to another student
The observers frequently place the same behaviour into different categories.
Explain one problem with the behavioural categories and suggest how the researchers could improve them. (4 marks)
Question 7
A researcher pilots a laboratory experiment. Participants in one condition receive more detailed instructions than participants in the other condition. Several participants also correctly guess the aim of the study.
Explain two problems revealed by the pilot study. For each problem, suggest an appropriate improvement. (6 marks)
Question 8
A psychologist plans to investigate whether the wording of questions affects eyewitness recall. During the pilot study:
Participants report that two questions are difficult to understand.
The researcher sometimes gives participants additional explanations.
One version of the procedure takes five minutes longer than the other.
Several participants say that they know what result the researcher expects.
Explain how the psychologist should use these pilot findings to improve the full investigation. (8 marks)
Comments