Earlier this month, The Asia Foundation (TAF) released a large survey on voter behavior in Afghanistan’s last parliamentary election. With two years until the next major Afghan election (excluding for a moment the important but widely-ignored Provincial Council elections, due in mid-2013 but unlikely to be on-schedule), it might seem like an odd time to be releasing this kind of work. However, with talk of pushing the presidential poll forward by a year and—if the Kabul rumor mill is to be believed—what seems like a genuine appetite among Afghan actors for reform of the country’s electoral system, there has perhaps never been a better time to talk about how the country votes, with an eye to managing the process better the next time round. Conducted country-wide TAF’s study is packed with authoritative-sounding, extensively disaggregated numbers on everything from how and why people voted, to perceptions on how free and fair the polls were. But reading through it, there’s an awful lot that isn’t covered and—more problematic, in many ways—no acknowledgement of its absence.
It is, of course, all too easy to criticize from the sidelines. In Afghanistan’s desperately data-poor environment, research with this level of coverage is gold dust, not to mention being a major logistical and financial undertaking. However, TAF’s institutional credibility combined with the fact that they’re one of the very few shops out there conducting country-level surveys also means that their work is ascribed a huge amount of weight—sometimes blindly so. Figures from its annual “Survey of the Afghan People” can be found quoted as fact again and again and again in anything from op-eds to donor project documents, usually presented with little in the way of background or analysis of its sometimes serious limitations. Given their very real potential to drive debate, it’s therefore critical to work out just what studies like this one can—and can’t—tell us. What follows isn’t a play-by-play analysis of the entire survey; there are many areas that are pretty straightforward, while others I’m simply not qualified enough to comment on. Instead, I’ll just try to highlight a few examples of how numbers taken in isolation—authoritative though the sample size might seem—can obscure or miss the wider story.
To begin with, TAF’s respondents did an awful lot of…well, voting:
Sixty-five percent of respondents report voting in the elections, while 34% say they did not.
Great. The only problem is that (admittedly notoriously unreliable) estimates for 2010 put turnout somewhere in the 40% range, with less votes being cast than in any other previous election. Having voters disproportionately represented in the data might make sense given that it’s a voter behavior survey, but is likely to have a significant impact on results obtained, especially when it comes people’s perceptions of the legitimacy of elections or electoral institutions. This wouldn’t be so much of a problem if this discrepancy was referenced in the study (it’s not). As it stands anyone not doing their homework (i.e. most people) is more likely than not refer to this data as representative of “Afghans,” rather than “mostly voters.”
Then there’s this:
Despite a decade of support dedicated to awareness raising on elections, a large proportion of our survey respondents report not knowing enough about the elections and electoral principles. (p. 1)
The survey attempted to measure how much information respondents had about the last parliamentary elections in Afghanistan, which were the elections for the Wolesi Jirga (lower house) held in 2010. Around one third (35%) of respondents say they had a lot of information, half (50%) say they had some information, and 8% say they had no information at all. (p. 11)
What does “information” mean here? Knowing when the poll was? Knowing how to vote? Knowing what to expect of an MP? The ins and outs of single non-transferrable vote (SNTV)? The presence of a women’s quota…? In studies we’ve conducted, people have generally had much better information about the first couple of questions than the remainder. Indeed, people’s expectations of their MPs (along with MP’s perceptions of their own roles) have been shown to differ dramatically from their statutory responsibilities, with major consequences for how representative governance in general is both practiced and perceived.
Again, extra context is needed. As it happens, the “decade of support dedicated to awareness raising” has been overwhelmingly and, given limited resources, deliberately skewed in favor of teaching people how to vote, rather than the political system that vote is helping to uphold. In fact, there hasn’t even really been a “decade” of support in this regard. Civic education has tended to take place in bursts of intense activity in the immediate run-up to elections, with few programs sustained in the interim. Meanwhile, the country’s school curriculum is also, two electoral cycles in, still devoid of civics classes (why this is the case is a mystery. It’s possible that teaching civics is viewed as politically thorny and hence best avoided; see the Ministry of Education’s “don’t mention the war” approach to history teaching).
But if some of the responses are vague without context, there are others that are perhaps misleadingly clear. This is especially true when it comes to people’s motivation to vote. Going by TAF’s data, the Afghan electorate are paragons of civic virtue, a plurality voting to “serve the country” (43%, p. 23) and a majority doing so to elect “educated, experienced…servant[s] of the people” (68%, p. 28). These are sentiments which have also cropped up repeatedly in many of our election-related interviews and are in many cases heartfelt—indeed, it seems that despite the ugliness of the last round of polls a reasonable number of Afghans have remained strikingly positive about the essential idea of selecting their leaders by popular vote.
But it’s also highly likely that there’s an element of social desirability bias in play—i.e. people telling interviewers what they think they want to hear. At their root, elections—in Afghanistan as with anywhere else—are about distributing power and resources (hence all that fraud). The “good citizen” picture TAF’s numbers presents may have an important grain of truth to it, but without context it also masks the essential messiness of what’s going on: the posturing, the wheeling and dealing, the bloc voting, the mobilization along ethnic or solidarity group lines… The large, multi-member, provincial-level constituencies that Afghanistan’s SNTV system has created means that in many instances, “representation” de facto boils down to MPs serving the interests of whichever communities or groups they happen to be personally linked to—and not any broader ideological or geographical area. Significantly, the seemingly widespread importance of “familiarity”—the importance of a community electing a candidate to which it has strong links to a community so that they can bring in resources and be held accountable—as a driver of voter behavior is notably absent from TAF’s numbers.
I’d go on, but you get the general picture. The real problem is perhaps not so much the survey’s lack of depth—it’s a survey, after all—but with the way the data is presented. As with other TAF outputs, the report seems to exist in its own hermetically sealed universe. Along with fairly minimal background information, there are only three references in all of its 133 pages—all to other TAF surveys. While still a drop in the ocean, a lot of other research has been done on Afghanistan’s elections, all of which could usefully have been linked to here. There’s the Afghanistan Research and Evaluation Unit (AREU)’s eight or so years of ethnographic work on the subject; the Afghanistan Analysts Network’s extensive examinations of local electoral politics; political analysis from the International Crisis Group; observation reports from Democracy International, the country’s own monitoring body the Free and Fair Elections Foundation of Afghanistan (with by far the most observers on the ground in 2010), and the National Democratic Institute (the latter particularly exhaustive); studies specifically documenting women’s experiences at the polls…the list goes on. While there could be an argument for letting the numbers speak for themselves, unfettered by nitpicking academic baggage, this is undermined when what they actually say becomes deeply misleading without the necessary context.
Anyway. If you have even a passing interest in elections in Afghanistan, TAF’s study definitely makes for important and timely reading. Just make sure to keep in mind, as you leaf through all of those pleasing percentages, of the seething, multi-faceted mess that lies beneath.
The author was a researcher on AREU’s women and elections project; his views are his own and not necessarily representative of those of AREU.