Introduction
to Accounting Research Seminar
Guidelines for Critiquing an Empirical
Research Paper
1.
What
are the core (bottom line) results of the paper? What key question do these results attempt to
answer?
2.
Why is
this question important (or why is it unimportant)?
3.
Identify
and describe the study’s null hypotheses.
Does the study provide sufficient front-end rationale or theory to
motivate these hypotheses? Explain.
4.
Describe
the logic of the hypothesis tests. Are
the hypotheses and tests structured in a way that could potentially provide
insight into the primary research question identified in (1) above? What were your prior beliefs with respect to
this question? Did the research design
provide the potential for the study to produce evidence that might have
influenced your prior beliefs? Why or
why not?
5.
What
are the study's dependent and independent variables? Describe any construct validity issues with
respect to these variables.
6.
Identify
any internal validity issues. In
particular, what are the potentially correlated omitted variables and how do
they threaten the validity of any inferences that the authors draw from their
results?
7.
Identify
any external validity issues. In
particular, to what population can the authors appropriately attribute
inferences drawn from this study? Did
the sampling strategy chosen by the authors limit the contribution of their
study? Explain.
8.
Identify
any statistical validity issues. In particular,
are the tests well-specified or are there problems with estimates of the
probability of type I errors?
9.
Could
the study have devised more powerful tests (reducing the probability of type II
error)? Explain.
10. Describe the results. What do you learn from each table and figure
in the paper? Describe the evidence with
respect to each hypothesis identified in (4) above. What is the paper's core result? If possible describe this result in both
statistical and economic terms.
11. What inferences do the authors draw from the
results of the study? Are these
inferences appropriate? Why or why not?
12. Have your prior beliefs, referred to in (4)
above, changed as a result of studying this paper? Explain.
In relation to other papers you have read, describe how this study
contributes to the literature.
13. What additional empirical work might you
suggest to the author(s) to make the paper more interesting and to improve the
paper’s contribution to the literature?
14. What important questions does this study
raise or leave unanswered? Do you see
any fertile areas for further research?
Explain.
Also see http://www.socialresearchmethods.net/kb/
for an excellent reference regarding research design issues (the notes above
are my own and did not come from the referenced website).