The final debate of the 2016 Presidential Election is over, and the tone of the debate felt—at least to me as a viewer—much different than the previous two. There were non-answers, interruptions, and pivots, but it didn’t have that free-for-all quality that inspired these analyses. Part of that was the influence of moderator Chris Wallace who asked very focused questions.
This is crucial to our analysis efforts, because the questions serve as our “baseline” for the results. It’s the content of the questions that allows us to determine if the candidates were on topic that night. It seemed that Wallace’s goal was to get the candidates to address each other without fighting with each other. In some cases, in order to get the candidates to compare themselves to each other, Wallace would ask essentially a different question to each candidate.
“Wallace’s technique to engender discussion was often simple. ‘Why are you right, and he’s wrong,’ he asked on several occasions, reversing the question for each candidate,” according to The Washington Post.
Take the opening question Wallace asked about the U.S. Constitution and the Supreme Court. The way Chris Wallace framed it—what the candidates’ thought the role of the court was and whether the law is set in stone—and others allowed the candidates to filibuster on their answer, throwing out “red meat” to their base.
Some accused Wallace of carrying the water for conservative interests with how he posed both the first and second question. “To a casual observer, this seems like a fair question, but Wallace uses loaded language that makes originalism seem very reasonable and living constitutionalism to seem highly susceptible to mischief,” according to The Post’s annotated transcript.
Now, this doesn’t affect our analysis as Salience can parse that language to get at the important themes in the question. However, it can affect the answers and, thus, how they are scored.
In the above chart, we can see that Clinton’s answer was more “on-topic” than Trump’s answer. Partly, this is because one of the top themes of her response—“I see the court”—was one of the top themes in Wallace’s question. The rest of the relevant part of Clinton’s answer directly addressed how she saw the court going forward, arguing that the best Supreme Court of the United States is one that preserves the rights of individual Americans.
Donald Trump, however, did not have the same kind of focus in his response, which is why he scored so low. Theme extraction focused on his emphasis on comments made by Justice Ruth Bader Ginsburg about his candidacy, and an emphasis on the Second Amendment. These topics, while tangentially related to the topic, did not score as highly relevant to the direction that the SCOTUS should take.
The Second Amendment bit especially was a topic aimed at his voters, because that is an issue most relevant to them when they think about the SCOTUS. Regardless, it isn’t exactly germane to the question the moderator asked.
The second question about the economy drew ire from some because Wallace seemed to imply that the Obama stimulus “caused” slow growth rather than the financial crisis it was meant to combat. This possibly affected the way Clinton answered. It’s also possible she was simply taking her opportunity to throw some political red meat to her supporters.
In my opinion, Clinton took a human-interest angle in her response. That may endear her to the electorate, but it did not relate as closely to the subject of the question. The top themes in her response focus on segments of the population she’s looking to help, the themes related to her actual approach scored weaker and thus were not key sentences used to relate to the question.
Trump’s answer scored much higher, in fact the highest-scoring answer of the questions we’ve analyzed. His answer dealt with what some consider his “strongest” issues, specifically avoiding tax increases for any income level and focusing on “trade deals,” while the current economic situation “is a disaster.”
In the question about the effort to retake Mosul from the so-called Islamic State, Trump scored much lower than Clinton. While Clinton’s extracted themes were focused on the specific conflict, Trump pivoted from talking about that war effort into criticizing Clinton about the Iran Nuclear Deal and the possible threat that country might pose.
Again, this could be political posturing; Since the commencement of the battle for Mosul it’s increasingly difficult for Trump to align himself with his past narrative that the US-led coalition is being bested by ISIL. It’s also possible that it was a flailing attempt to draw the debate towards another topic he was more practiced in discussing.
On the topic of accepting the results of the election, Hillary Clinton scored surprisingly low considering how highly her answer to this question in the first debate was scored. Yet, this question wasn’t really for her, since Clinton has left no question about whether or not she’d accept the results of the election. The question posed by Chris Matthews was specifically about Trump’s recent statements about the election being “rigged.” Clinton’s response was more of an attack on Trump and his personality than at all related to the moderator’s question about accepting the outcome.
Finally, on the question about Donald Trump, his treatment of women, and his assertion that what the Clintons have done in the past is “worse” both candidates were remarkably on-topic. However, since the question was framed mostly towards the allegations against Trump, Clinton was able to score higher than him because she didn’t talk much about his allegations against her.
All of Trump’s top extracted themes were about the accusers and alleged Democratic operatives, including his belief that “they want fame.” Clinton’s top themes weren’t about the accusers or the women who’ve levied allegations against her husband, but rather about Donald Trump.
Overall I think the analysis over each of these debates speaks to the method of communication that each candidate uses. Trump often does not speak in complete sentences, and has tangents that at times diverge for the core of even his own response. Clinton often seems closer to the mark, but at times tries to make the voters the core of her response rather than really driving the point home with policy.
It’s worth reiterating that, as impressive and remarkable as Salience is, it’s not a fact-checker. We are not suggesting that any of these answers are “right” or “wrong.” Rather, this project began as a matter of curiosity on my part about whether the answers to the debate questions were actually answers to the question that was asked. Could we use linguistic tools to cut through the political posturing and redirection to extract the elements of their responses that held the most weight, whether or not the candidate realized what they were emphasizing? In that effort, I think I succeeded and now we have a useful new tool in the toolbox that allows voters to push past partisan rhetoric and campaign spin for an unbiased and qualitative analysis of this crazy political process.
In other election/data science news, you might wanna head over to this neural network trained on Donald Trump transcripts, it’s pretty great.