GOP debate finalists chosen, with questions over methodology

Fox News announced its much-anticipated lineup for the first GOP presidential debate, scheduled for Thursday in Cleveland.

The 10 candidates who qualified to be onstage in prime time, as determined by Fox’s criteria of how well they performed in five recent national polls, are business mogul Donald Trump, former Florida Gov. Jeb Bush, Wisconsin Gov. Scott Walker, former Arkansas Gov. Mike Huckabee, Texas Sen. Ted Cruz, neurosurgeon Ben Carson, Florida Sen. Marco Rubio, Kentucky Sen. Rand Paul, Ohio Gov. John Kasich and New Jersey Gov. Chris Christie.

The debate, which will air live at 9 p.m. Eastern time, is slated to run two hours and will be co-sponsored by Facebook. 

Kasich, who likely came in 10th, edged out former Texas Gov. Rick Perry for a spot in prime time. Besides Perry, the six candidates who did not make the cut but will be featured in an earlier debate, to be aired at 5 p.m. Eastern, are former Pennsylvania Sen. Rick Santorum, Louisiana Gov. Bobby Jindal, former Hewlett-Packard CEO Carly Fiorina, South Carolina Sen. Lindsey Graham, former New York Gov. George Pataki and former Virginia Gov. Jim Gilmore.

The 10 candidates were invited on the basis of their average standing in five recent national polls. Among polling experts, there has been some dissatisfaction with this method, since poll results aren’t always able to differentiate front-runners from the rest of the pack. One polling center suspended its presidential preference question this week to avoid being part of the debate’s candidate selection process.

The main objection is that poll results have a margin of error that could shift candidates hovering near the cutoff. The reported numbers for each poll are an estimate of the country’s preferences at the moment, but there are other statistical factors that are important, particularly when there are so many candidates.

Since it is not feasible to survey every eligible voter, polls derive their figures from a representative sample, with a narrow range of probable results. In general, the smaller the sample or the smaller the subset of that sample, the greater the margin of error. For example, if a candidate on the fringes of the preference polls comes in at 3.0 percent with a 2.2 percent margin of sampling error, then 95 percent of the time, the candidate will poll from 0.8 to 5.2 percent. Among the GOP presidential candidates, such a swing could move someone to the bottom of the list or into the top 10.

Another, arguably larger factor is the set of assumptions each pollster makes about likely voters and those the pollster was able to reach during polling. These assumptions can affect how a pollster weighs each respondent’s answers, which could change final standings by several percentage points.

Some polls target likely voters — determined by preliminary questions about past electoral behavior — while others sample from a list of all registered voters. The variations in selection can produce different rankings.

Al Jazeera calculated top 10 lists for every combination of five poll result subsets drawn from nonpartisan phone surveys reported on Pollster.com from June and July 2015 and calculated the number of times each candidate made the top 10, on the basis of each combination of polls. While a majority of the combinations garnered the same top 10, there are scenarios in which the ninth through 11th candidates play musical chairs.

Margins of error and assumptions notwithstanding, using only five polls can generate different rankings, depending on the selected surveys and method used to average them.

Fox News may not average results using either of these methods. But the differences between them indicate how the method used can change rankings.