How to: Critically Review Technical Reports

Analytical studies play a crucial role in the energy policy world. They help decipher complex energy markets and technologies and often affect policy decisions. This makes it important for decision makers and other policy stakeholders to critically analyze these studies.

The following steps will help you critically analyze data whether you’re a policy stakeholder or an interested third party:

• Find the Technical Details – Whenever you encounter any new results or conclusions in a study, the first thing to do is look for the technical details of the analysis. These could be part of the main report in the form of appendices or a separate document. Within the technical details it is important to focus on the following components: scope, assumptions or data variables, data sources, methodology, and results and conclusions.
• Understand the Assumptions or Data Variables, Scope and Caveats – Assumptions or data variables, analysis scope and caveats set up the context in which the study was done and also decide its applicability as they represent subjective aspects of the analysis. It is important to go over them to understand the potential and limitations of the analysis’ findings.
• Investigate the Reliability of Data Sources – The data used in a study significantly affects the quality of its results and conclusions, making it important to note the reliability of data sources used in the study. If data used in a study is not properly sourced or referenced or the source is not credible, it may require further investigation.
• Review the Methodology – The methodology section of a study outlays the mathematical models used to analyze the respective problem. Reviewing the methodology helps us understand the sophistication and detail with which the problem has been analyzed.
• Apply What you Learned to the Results and Conclusions – The results and conclusions of a study generally contain the information relevant to the policy stakeholders’ decision-making process. A better understanding of the data used in a study can help gauge the limitation or applicability of the results and make sure the conclusions made are realistic.

These steps can help anyone look at analytical studies with a critical mind and will lead to a deeper understanding of the results, which is especially important when those results influence policy decisions.

Varun Kumar is Policy Data Analyst in Acadia’s Climate and Energy Analysis (CLEAN) Center. He assists in the analysis of energy, climate and economic information as part of Acadia Center’s advocacy and research efforts.