Learn Before
Based on Kraut and Johnston's methodology, explain how the research group should modify their process of identifying behaviors and assessing reliability. How does this compare to their current real-time unstructured descriptions, and why would this change improve the scientific rigor of their study?
Case context: A research group is planning an observational study on facial expressions during video gameplay. They watch players and write down descriptions of their faces in real-time. However, when comparing notes, they find their observations vary wildly. A student recommends adopting a methodology similar to Kraut and Johnston's bowling study to systematically record reactions and prove their observations are reliable.
Question: Based on Kraut and Johnston's methodology, explain how the research group should modify their process of identifying behaviors and assessing reliability. How does this compare to their current real-time unstructured descriptions, and why would this change improve the scientific rigor of their study?
Sample answer: The research group should first establish a predetermined list of target behaviors (such as specific smiles or face covers) that observers must memorize beforehand. They should also video-record a subset of the gameplay sessions so that two independent observers can code them separately. Finally, they must calculate their inter-rater reliability by finding the percentage of agreement (aiming for a high rate like Kraut and Johnston's ). This structured approach reduces observer bias and subjective interpretation compared to unstructured real-time writing.
Key points:
- Explains the need for a predetermined list of target behaviors that observers memorize.
- Describes using video recordings to allow independent observers to code the same subset of reactions.
- Explains how calculating the percentage of agreement assesses inter-rater reliability.
- Contrasts the structured coding system with the unstructured, real-time descriptions to show how it reduces subjectivity and bias.
Rubric: The answer should describe establishing a predetermined list of target behaviors, memorizing them, recording a subset of behaviors on video, having independent observers code them, and calculating a percentage of agreement. It must explain that this reduces subjective interpretation and measures inter-rater reliability.
0
1
Tags
KPU
Research Methods in Psychology - 4th American Edition @ KPU
Related
In Kraut and Johnston's behavioral coding study of bowlers, what did their findings suggest about the behavior of smiling?
Robert Kraut and Robert Johnston used behavioral coding to systematically study bowlers' reactions. Arrange the following steps of their methodology in the correct chronological order to show how they ensured their data was reliable.
Match each methodological goal of Kraut and Johnston’s study on bowlers with the specific action they took to achieve it.
In the study of bowlers' reactions, if the data had shown that bowlers smiled just as frequently when facing the pins alone as they did when facing their companions, the researchers' conclusion that smiling is a 'social signal' would still be supported.
In the study of bowlers' reactions, the observation that participants frequently smiled when turning toward their companions was used by researchers to support the idea that smiling is an automatic, internal reflex to success.
If a critic argued that the behavioral coding used in the study of bowlers was too subjective to be scientifically valid, the researchers could _____ this claim by citing the fact that independent observers achieved a agreement rate.
In Kraut and Johnston's behavioral coding study, two independent observers watched video recordings of bowlers and categorized each reaction using a predetermined list of behaviors such as 'open smile' and 'face cover.' The observers agreed on the coded reaction of the time. What does this agreement rate primarily establish about the study's methodology?
Match each design decision from Kraut and Johnston's bowling study to the research quality criterion it primarily addresses.
Kraut and Johnston's finding that bowlers rarely smiled while facing the pins but frequently smiled when turning toward their companions supports the analysis that smiling in this context functions primarily as _____ rather than as an automatic emotional response triggered by the outcome of the bowl.
A student is critically evaluating the scientific merit of Kraut and Johnston's behavioral coding study of bowlers. Arrange the following evaluative steps in the order that reflects sound critical analysis, moving from identifying the research claim to rendering a final overall judgment.
In the study conducted by Kraut and Johnston on bowlers' reactions, what were the specific target behaviors established in their predetermined list for behavioral coding, what method did they use to demonstrate the reliability of their observations, and what was the specific numerical agreement rate they achieved?
Based on Kraut and Johnston's methodology, explain how the research group should modify their process of identifying behaviors and assessing reliability. How does this compare to their current real-time unstructured descriptions, and why would this change improve the scientific rigor of their study?
Suppose you are designing a naturalistic observation study to test whether laughter is primarily a form of social communication rather than an automatic response to humor. Applying the design decisions and findings from Kraut and Johnston's bowling study, describe what specific situational comparison you would make to test this hypothesis.