Knowing there are pretty dangerous pitfalls in statistical analysis and that statistics is a challenging area to master: so, it should follow keep it simple stupid (KISS). Simpler experiments are actually more severe tests because there is nowhere to hide: something either clearly passes the test or it does not.

Safe Analysis

Having devised an experiment that is simple to analyse, it is important also not to fall at the final hurdle of interpreting the results of the statistical tests.

Interpreting Analysis

What statistics can indicate is some results are perhaps more unlikely than might be expected and therefore worthy of further investigation.
Experiments can be exploring the possible influences of many factors on particular interactions or experiential outcomes. Statistics cannot possibly provide evidence in support of any particular factor influencing the outcome of the experiment. 


Why Statistics Alone Do Not Provide Good Evidence

It is essential for an experiment to succeed. Statistics are needed to help identify differences outcome between testing conditions are systematic and meaningfully related to the independent variable.

School of Visual Arts

Arizona State University

1. Experimental design
2. Statistical analysis
3. Experimental write-up

Experiments have always played an important role in Human-Computer Interaction. 

Three Pillars of Good Experiments:

Cairns, Paul, https://www.thereformation.com/categories/sale

Experimental Methods in Human-Computer Interaction

Experimental designs are written up in papers as objective descriptions of a state of affairs and the complex processes that led to that particular experiment are glossed over. In particular, the fact that an experimental design may have been iterated on paper, tested in a pilot or even failed entirely and iterated further may only merit the briefest of mentions, if any.


Experimental design

It is essential for an experiment to succeed. 
Statistics are needed to help identify differences outcome between testing conditions are systematic and meaningfully related to the independent variable.

Statistical analysis

An experimental write-up is an important pillar in the development of experiments because it enables the reader to follow the chain of evidence, validity, and analysis.

Experimental Write-up

An experimental write-up is an important pillar in the development of experiments because it enables the reader to follow the chain of evidence, validity, analysis.

Learn Before

Related

Learn After