Imagine hearing this: “Climate change is a hoax! How can climate change be happening when this graph shows that the temperature of Earth hasn’t increased in over a hundred years?!”
This graph shows the average global temperature anomalies from 1880 to 2021. The anomalies refer to the difference from the long-term average global temperature that occurred from 1951 to 1980. By manipulated the vertical axis (y-axis) of the graph, data can be misrepresented to bias readers. We made this graph using real-world data from NASA and "stretched" the y-axis to show an example of a misleading graph.
Does this claim sound familiar? This is a classic real-world example of a misleading graph. In 2015, the National Review tweeted a similar graph to “back-up” their claim that climate change doesn’t exist. While this graph shows real data from NASA, the data is misrepresented to bias readers into believing a false claim.
As savvy critical thinkers, we know this claim is not true. But at first glance, the accompanying graph appears to back it up. It can be easy to believe claims if it also includes statistics or graphs. Thus, it’s always important to take a minute to add some context to the data. This is because humans collect data, analyze the results, and represent the data visually, usually in the form of infographics. This means data is prone to error, bias or, worse yet, purposeful manipulation to push the reader to a specific conclusion.
So, how can we confidently read and interpret graphs to accurately make conclusions? Here are 3 tips to help you confidently evaluate graphical data.
Tip #1: Ask yourself “who made the graph?”
Don’t fall into the trap of believing that data = truth. Humans collect and analyze data, which means we shouldn’t accept it no-questions asked. It’s important to always consider the context.
What is the source of the graph? To begin to evaluate the credibility of data, first look into who made the graph. Does the author have relevant expertise or credentials? If an organization created the graph, be sure to check out the organization’s mission statement and funding sources. Do you feel they could be motivated or biased to misrepresent data to show a specific conclusion?.
Do you trust the source? As you investigate where the data came from, keep an eye out for any red flags. If the author is anonymous, if you notice any potential conflicts of interest, or you can’t find any information about the source, these are signs that it may not be trustworthy.
Tip #2: Orient yourself to the graph
Once you trust the source, the next step is to get oriented to the graph.
What type of graph is it? There are all sorts of ways to visualize data, including scatter plots, bar graphs and pie charts. Depending on the data collected, certain graph types might be better suited than others. Here, we will focus on general tips that can be applied to most graphs. But recognizing the type of graph will be helpful as a first step to getting oriented to the data.
Does the data represent individuals or the mean? If data points are shown, is each data point from an individual or the mean of many individuals? If it’s a bar graph, it is likely showing the mean, but does it also show how variable the data was? Generally, variation is represented as either standard deviation or standard error of the mean. How variable the data points are is also an important factor in the confidence of the conclusions.
What are the axes? Be sure to familiarize yourself with the axes, including the axis labels and units. This will give you a sense of what data was collected and how it was measured. This is an especially important step because, as we can see from the example above, it is very easy to skew what the graph looks like simply by adjusting the maximum and minimums of an axis. In fact, as the Washington Post pointed out, graphically representing the same data from NASA using appropriate axes leads to a much different conclusion about the Earth’s temperature.
This graph shows the average global temperature anomalies from 1880 to 2021. Importantly, it is visualizing the same data as the misleading graph. By using an appropriate and meaningful vertical axis (y-axis) range, the data trend looks much different. So how do we know what is "meaningful?" We keep the variable in mind. In this case, it is global temperature changes, which has meaningful effects with small changes. Thus, a smaller range of values can be used as the y-axis. We made this graph using the same real-world data from NASA.
Tip #3: Ask yourself “how was the data collected?”
Consider this example: in the 1936 presidential election, incumbent President Franklin Roosevelt was running against Alf Landon. The magazine Literary Digest surveyed 2.4 million voters about who they would vote for in the upcoming election. Based on the survey results, Literary Digest confidently, but erroneously, predicted a landslide victory for Alf Landon. Instead, President Roosevelt won the election with an overwhelming margin. How did the Literary Digest poll get it so wrong? It comes down to who was sampled - those that received (and returned) the poll were not representative of the general population. This example tells us that it is always important to know how the data was collected to help interpret the conclusions.
Read the figure caption. This will give you a sense of how the data was collected. Here, look for any red flags, like a small sample size, or a non-randomized design (if Literary Digest surveyed a random group of people, which would then be more representative of the general population, do you think they would have found different results?)
If the authors didn’t collect the data themselves, do they cite their source? This is especially important! If the authors didn’t collect the data and didn’t cite where the data came from, how can you be confident that the data is real or trustworthy?
Was a control group included? A control group acts as a baseline or benchmark to compare with the experimental results. Without a control group, we can’t be confident that the results are due to what was manipulated in the experiment or if the results occurred because of other factors or just due to chance.
How many times was the experiment repeated? We want to be confident that the results of an experiment aren’t due to random chance, errors, or other factors. Thus, being able to reproduce the same results over multiple experiments helps increase the confidence of the conclusion. Look for how many experimental repetitions, denoted as N, were conducted. Don’t get N confused with the sample size, which is usually represented as n, and denotes how many samples were measured.
Great! You have evaluated the graphical data by asking yourself some important questions about the graph and how the data was collected. Following these tips can help you confidently sleuth through the graphs you’ll find on social media and the internet. By being savvy and critically evaluating graphs, you can make your own conclusions about the data. Does your conclusion match what the author claimed?
Data visualizations like graphs and charts are great ways to simplify data and increase accessibility. However, as we’ve seen, they are easy to manipulate. So when you look at data that is represented visually, or when you’re making your own graph, be sure to see if the data is presented fairly.