A comment on `Common ecology quantifies human insurgency’

by Joshua Foust on 1/27/2010 · 6 comments

A joint response by:
Drew Conway, Department of Politics New York University;
Thomas Zeitzoff, Department of Politics New York University; and
Joshua Foust, http://www.registan.net/

In a recent article in the science journal Nature, J. C. Bohorquez, S. Gourley, A. R. Dixon, M. Spagat, N. F. Johnson (BGDSJ from now on) presented a unique data set of “54,679 violent events reported within nine diverse insurgent conflicts,” and found that insurgencies exhibit a common power law distribution of α≈2.5. It’s been presented as a big advancement for the study of warfare, netting Gourley a well-regarded TED talk, tons of blogosphere and academic chatter. Even Nature’seditors found the rather short article—a little under two pages of text—important enough to give it the cover over articles discussing a cure for cancer and the possible existence of an extra-solar Earth-like planet.

Power law distributions are a common feature of complex systems, whether stock prices, the structure of the World Wide Web, or even networks of sexual partners1—which makes us wonder what is so novel about seeing it in a complex system like an insurgency. What, exactly, do we learn by observing that insurgencies behave complexly?

To be clear: We commend BGDSJ for exploring this question, as the underlying fundamentals of warfare are important. But their methodology contains serious flaws. The assumptions of their model do not reflect empirical observations of insurgent organizations, their research relies on confirmation bias, and their dataset, while extensive, is neither comprehensive nor representative of the insurgencies they highlight.

BGDSJ present a simplified, bi-polar model of insurgency: assuming there is one group of insurgents that reacts to previous (and publicized) successes and one group of government forces. Their model rests on the key assumption that insurgents update their organizational and tactical decisions based on a universally observed and interpreted global signal independent of the signal’s content. Why limit discussion to the mere existence of a global signal? “Actual content is unimportant provided it becomes the primary input for the group’s decision-making process,” they argue. This poses several problems:

  1. Selection Bias. The “global signal” BGDSJ posit is a function of the frequency and mortality of insurgent attacks, but both of those factors rely on reporting. In other words, they limited themselves to published reports in the media. But, at best, the global media publicize only a tiny fraction of the actual violent event in a given insurgency—and that’s assuming their reports are even remotely accurate.
  2. Most insurgents groups cannot operate freely throughout a country, which makes private information a critical factor in modeling their behavior. How does private information replace, supplement, or contradict the “global signal,” and to what effect?
  3. Finally, joining and leaving an insurgent groups is not a costless process, as insurgent groups have recruitment procedures and often violently prevent exit. What effect does group pressure play in the way insurgents respond to a “global signal?”

In a way, BGDSJ’s arguments are rooted in the somewhat muddy waters of various reflexivity theories, though that is beyond what we’re discussing directly here. What they’ve really done is reverse-engineered a model to fit their data—something that, of course, confirms the model’s usefulness. Since that data is bad—they use ex-post facto reports of violence, not the behavior of either side in the conflict to prove their point—they wind up assuming individual behavior from group processes. The Ecology of War has an ecological problem.

In reality, this ecological model can only be considered one of several competing theories to describe the dataset. BGDSJ try to preempt such criticis by saying, “any competing theory would also need to replicate the results.” But creating a model to fit one’s data is an inversion of the scientific process, reducing the study to mere deduction. When respected newspapers write stories claiming ten percent of all Chechens live in South Waziristan, we must seriously question just how one goes about creating a useful model of behavior based on media accounts.

Given a limited and biased dataset and the previously mentioned ecological issues, the authors have become mired in confirmation bias—a poor way to conduct social science research. A model’s usefulness is not in how well it fits a given set of observations, but how well it serves as a useful approximation of reality—as such, BGDSJ’s model fails.

While BGDSJ find that attacks in nine different insurgencies follow a specific type of distribution, that’s not terribly interesting. As we’ve discussed before, the most interesting question is not what makes two wars similar, but what makes them different? Looking only at similarities between wars ends up obscuring vital differences—which leads to people arguing that methods in Iraq should apply directly to Afghanistan. This blog has discussed at length the uniqueness of Afghanistan’s history and conflicts; while it would be possible to draw some lessons from similar or nearby crises, when we only consider those similarities we open ourselves to quite literally fatal errors of conception.

Moving beyond Central Asia, we should ponder why Peru’s insurgency died but Colombia’s is still going on. The answer isn’t in the numbers. Without considering the human element, without contextualizing the numbers in politics, culture, society, even personality, any study of insurgencies will, ultimately, come up short.

Non-web accessible works cited:
1 Liljeros, Fredrik, Christofer R Edling, and Luis A Nunes Amaral. “Sexual networks: implications for the transmission of sexually transmitted infections.” Microbes and Infection 5, no. 2 (February 2003): 189-196.

Subscribe to receive updates from Registan

This post was written by...

– author of 1848 posts on 17_PersonNotFound.

Joshua Foust is a Fellow at the American Security Project and the author of Afghanistan Journal: Selections from Registan.net. His research focuses primarily on Central and South Asia. Joshua is a correspondent for The Atlantic and a columnist for PBS Need to Know. Joshua appears regularly on the BBC World News, Aljazeera, and international public radio. Joshua's writing has appeared in the Columbia Journalism Review, Foreign Policy’s AfPak Channel, the New York Times, Reuters, and the Christian Science Monitor. Follow him on twitter: @joshuafoust

For information on reproducing this article, see our Terms of Use


Larry Dunbar January 28, 2010 at 11:48 am

I keep hearing the words of a song that goes something like this: “it doesn’t take a Weatherman to tell you which way the wind blows.” Now, “Weatherman” could mean a person gathering hard facts through scientific methods to determine where the high and low potentials are that determine which way the winds will displace air to and from. Another meaning could be an insurgent group located between political fronts, or potentials, that are deciding which way their political “winds” are displacing to or from. Either way, you have 3 realities, the weatherman, the Weatherman, and you, who don’t need either to tell you the direction of the wind.

However, reality is not real; death is real, and you are trying to assign something real to something that is not real. My guess is the data BGDSJ is using is as close to the reality of everyone, the insurgent, the incumbent, and the media to make it real. While the story about 10 percent of Chechens live in South Waziristan may not be real, it may have seemed real to someone. It maybe reality base, but just not real, or correct. Throw it all in the same pot, and call it reality of the insurgency you’re fighting. Subtract what you can, but remember it is all reality based.

While the insurgent may have real information that is private, his reality is not. His reality is one shared by the incumbent, the insurgency, and the media. If the data used in plotting the power-law is not just real, but reality based, then perhaps the kind of structure the power-law tells of, will be reality based. A sociologist then may be able to give you a better image of what that structure consists of, but the fact you have the slope is a start. If this slope flattens-out, gets steeper, or goes positive, the information this provides is useful, in itself.

As an example, If the slope flattens out, you may want to switch sides; if it gets steeper, you may want to slack-off just a bit, or you will be out of a job; or if it goes positive, you may want to let the guys on the ground know (if they don’t know already) and be on the look-out for the cause of the structure change.

While the guy or gal on the ground fighting the insurgency knows what’s real better than those at the battalion level, neither may have all the information necessary to understand the reality of the situation. The effort of the BGDSJ could go a long way in connecting what is real with reality.

Realist Writer January 28, 2010 at 6:18 pm

Am I the only person who believe the Peru’s Insurgency did NOT die? It only was dormant, trying to recover, but it is slowly growing.

Still, the article’s good.

Nathan Carter January 30, 2010 at 11:03 am

This post was…unfortunate. It seems you based your criticism of the study on Gourley’s TED talk—which was admittedly full of hype—rather than the study itself.

First of all, I fail to see how the article suffered from the ecological problem. The author’s never claimed to be saying anything about individual behavior. They were talking about what tended to be true across individual cases.

Second, the reliance on media reports is not that problematic for a project of this scope. Everyone knows that the media record of things like war deaths is often inaccurate and/or incomplete. That leaves two options:

1. The media reports were consistently flawed. So nearly all of every 100 reports overstated/understated the number of deaths by a certain amount.

2. The media reports were randomly flawed. So out of every, say, 100 possible reports of war deaths, a random number of them were inaccurate or missing, and of those inaccurate reports, each over-reported or under-reported the number of deaths by a random amount.

If option number 1 is correct, and the amount of distortion was consistent across the board, then the distortion doesn’t matter because the overall power-law distribution would be the same. And, besides that, we would have to believe that those flaws were consistent not only across individual deaths but also across all of the different insurgencies for which the author’s found similar results.

That means option 2 is the more likely one. Random noise certainly affects the soundness of the study, but not enough to discount the findings. It just means there was a bunch of random errors embedded in the reports. That is why they used statistics. Statistical procedures are designed specifically for the purpose of making sense of data that have been distorted by random errors in reporting. Accounting for those errors is literally built into the programming.

They also accounted for option 2 by replicating their study for a variety of different insurgency situations. They found the same patterns for every country they looked at. That means that if there was gross flaws in the reporting, those flaws were at least consistent in every situation, which would mean that option 1 was true, which would mean that the flaws weren’t a problem.

Media-reporting flaws is certainly a concern when you want to say something about how many people died in a specific instance. But it you’re talking about overall patterns, as these authors were, it’s simply not a problem.

Third, you failed to note that, on a number of points, the authors interpreted their results in a way that was not necessarily supported by their equations. They placed a lot of emphasis on things like insurgents’ desire to have news of their attacks reported by the media, which they modeled as a “global signal.”. The equation they used to model that signal only dealt with the availability of information about the relative vulnerability of various targets. It can very reasonably be interpreted as representing that “private information” that you said the study failed to take into account. The basic model was sound, their interpretation of it was not. It’s important to distinguish between those two things when forming an opinion of their work.

Fourth, a study doesn’t have to say everything about insurgencies for it to say something useful. The study showed that a couple basic patterns prevailed across insurgencies, and that those patterns could be explained by a couple simple mechanisms, mostly dealing with the availability of information and freedom of movement. It showed how the shape and scope of insurgencies are limited by a few basic factors, regardless of more context-specific factors.

And last, you said, “Without considering the human element, without contextualizing the numbers in politics, culture, society, even personality, any study of insurgencies will, ultimately, come up short.” At least the Nature article authors presented evidence and reasoning for their claims. All you presented was a straw man.

In the end, you displayed a disappointing lack of familiarity with what the article actually said. You picked out a few surface issues, hyped them up as much as Gourley hyped up his article in his TED talk, and then made your argument for an alternative based on assumption rather than evidence.

That’s too bad. I had come to expect better from this blog.

Joshua Foust January 31, 2010 at 3:25 pm

Actually, this comment was… unfortunate. If the media accounts contain errors that are systematically correlated, they are done so differently by insurgency: for example, in Iraq it may be Coalition bias, while in Colombia it might be a lack of reporting. The authors don’t say so definitively, which is why we question their methods.

And this statement:

“Statistical procedures are designed specifically for the purpose of making sense of data that have been distorted by random errors”

Doesn’t make any sense. The vast majority of statistical methods (OLS, etc) assume errors are random, otherwise their findings are errant. The more difficult task occurs when errors are not random, requiring various corrective measures. The point about different types of errors implies that they are not random, which is why we find the dataset suspect.

Outlaw January 31, 2010 at 2:51 pm

Note: there is a single comment in your article that clearly indicates your total lack of understanding of say the Iraqi Sunni insurgency—-there was strong cross movement between members of the various groups—based on tribe, personal relations, marriage, prayer groups, previous military/police service.

Not as you stated: “Finally, joining and leaving an insurgent groups is not a costless process, as insurgent groups have recruitment procedures and often violently prevent exit. What effect does group pressure play in the way insurgents respond to a “global signal?”

Joshua Foust January 31, 2010 at 3:21 pm

I’m missing the part where we limited the discussion of insurgent group cohesion to Iraq. Help me out?

Previous post:

Next post: