Inter-rater Reliability

Test-Retest Reliability

Internal Consistency

Match each type of measurement reliability with the aspect of consistency it evaluates.

A researcher is developing a new 15-item scale to measure 'subjective well-being.' To evaluate the measure, the researcher checks whether participants who agree with one item (e.g., 'I am happy with my life') also tend to agree with other items on the same scale (e.g., 'My life is close to my ideal'). Which type of reliability is the researcher primarily assessing?

A researcher is developing a new coding system to measure 'prosocial behavior' in toddlers by watching video recordings of their play. To ensure the measure is consistent, she has two different research assistants code the same set of videos. If their observations are highly similar, the researcher has established high test-retest reliability.

A psychologist developing a new behavioral observation tool for 'classroom aggression' finds that her three research assistants all report the same number of aggressive acts for each child. However, when the same children are observed again under identical conditions one week later, their aggression scores have changed dramatically. This pattern suggests the tool has high inter-rater reliability but low ________ reliability.

A psychologist is evaluating the overall reliability of a new behavioral observation scale designed to measure a stable personality trait. Rank the following reliability profiles from the one that provides the 'strongest' evidence of a scientifically sound measure to the one that provides the 'weakest' evidence.

You are tasked with creating a validation protocol for a new psychological instrument that measures 'Academic Persistence' using a combination of a $15$-item questionnaire and a timed behavioral task. To ensure your design accounts for consistency over time, consistency within the questionnaire items, and consistency between different researchers, which of the following sets of procedures must you synthesize into your research plan?

In psychological research, the consistency of a measure's results across different researchers or observers is referred to as test-retest reliability.