The Art of Self-Evaluation

by Joshua Foust on 4/26/2011 · 11 comments

I’ve never been a big proponent of attempts to “sell” the success of the Afghan National Security Forces, especially the National Police. That is because, while we can all point to individual examples of certain units succeeding, by and large we have no sense of how well the police are doing—and, worryingly, we have lots of stories about how badly they’re doing in terms of operational readiness and retention (pdf).

This came up in February when General Caldwell, who runs the training mission in Kabul, wrote a long op-ed about how well his training mission is going. He did so by pointing, exclusively, to recruitment numbers. My sense, at the time, was that recruitment numbers don’t really tell you much about your training effectiveness, operational capacity, and overall effect on the war or on certain operations. Recruitment numbers are part of it, but by no means all.

It is an important concept to keep in mind: as the commander of the effort, Caldwell has every incentive to portray his efforts as successful, just as he is under no obligation to highlight his failures for public consumption. This is true at all levels of command, and it is why I get so frustrated at the obviously lopsided portrayals of the war by senior officers.

Anyway, now SIGAR has audited the ANP. And the story they tell is not very pretty:

U.S. auditors have found that the Afghan government cannot determine how many people work for its national police force, whose payroll is primarily funded by the United States and other international donors, making it difficult to determine whether the money is being properly spent.

The audit found a 10 percent difference in records and databases maintained by the Ministry of Interior, which runs the police. The various record-keeping systems showed that as of Sept. 30, 2010, the size of the force ranged from 111,774 to 125,218 personnel.

“The Ministry of Interior cannot accurately determine the actual number of personnel that work for ANP because it has been unable to reconcile its personnel records with ANP personnel available for work,” the report said.

This is, to put it bluntly, basic record keeping that shouldn’t be a challenge in the year 2011. Not even in Afghanistan, which has computers and people who know how to use them (and is supported, very generously, by international donors including the U.S.). In an appendix to the report (pdf), one of Caldwell’s subordinates urges SIGAR to stop holding the Afghan Ministry of the Interior to western accounting standards. This is an understandable concern, however it’s not really relevant: we have had ten years for training, record-keeping, and so on. That the Afghan Police still don’t operate in a minimally effective way is a stinging indictment not of them, but of the people training them—which is, by sheer weight of cash, the United States and the UNDP. The focus on computers and automation strikes me as deeply misguided: the Afghans trying to administer the program need to start with the basics of record-keeping and administration, not complicated electronic systems that might not even be relevant to the effort.

As we ponder General Petraeus’ strategy of building Afghan security forces as an exit strategy, concerns about the Afghan security forces will mount. However, this audit—and these concerns about administrative failures—still doesn’t speak to the ANP’s effectiveness. And it is here that the house of cards starts to teeter a bit. Despite, or perhaps because of, our inability to track who is serving and who is drawing a paycheck, we cannot guarantee that the ANP will perform effectively when required to. That remains the real scandal at the heart of the training mission—and the heart of why our chosen path toward success is more fraught than we seem willing to admit.


Subscribe to receive updates from Registan

This post was written by...

– author of 1849 posts on Registan.net.

Joshua Foust is a Fellow at the American Security Project and the author of Afghanistan Journal: Selections from Registan.net. His research focuses primarily on Central and South Asia. Joshua is a correspondent for The Atlantic and a columnist for PBS Need to Know. Joshua appears regularly on the BBC World News, Aljazeera, and international public radio. Joshua's writing has appeared in the Columbia Journalism Review, Foreign Policy’s AfPak Channel, the New York Times, Reuters, and the Christian Science Monitor. Follow him on twitter: @joshuafoust

For information on reproducing this article, see our Terms of Use

{ 11 comments }

Burk April 26, 2011 at 11:58 am

Could I offer that retention may also not be a terribly important metric? Since the quality of the force is so low, a great deal of selective attrition/firing might be the best possible policy to raise quality and effectiveness.

Boris Sizemore April 26, 2011 at 1:02 pm

You are correct, most of these metrics are just to make Caldwell look good.

The situation for the ANP on the ground ranges from full up capacity, with rapid reaction and a full range of skills in Kabul to fully non existent in many parts of the country.

It is really a sliding scale, as you drive out of the main cities the ANP begins to vanish until you get back to another main town.

I have spoken to countless ANP members. They are clear that when push comes to shove they need to hightail out of the zone when things become too hot to handle. It does make sense. They are lightly armed gendarmie and have no place in a combat zone. One certainly cannot blame them.

It is up to the Afghans the ANSF to decide the final outcome of this war. These metrics will never be able to measure fighting spirit. Until there is a resurgence in the Afghan leadership at all levels this will just not happen. And, it cannot happen ever under the auspices of Caldwell and ISAF. So it does not matter what we measure, the future holds all the cards in this one.

Dishonesty? April 26, 2011 at 2:11 pm

Retention problem?Nonsense!
although….

May 2010
Elite Afghan units trained by U.S. Special Forces. The 7,000-strong Afghan Commando Brigade – the country’s premier infantry force
http://www.armytimes.com/news/2010/05/army_afghan_special_forces_051810w/

March 2011
More than 200 Afghan National Army soldiers completed their transition from the regular ANA Army to the elite commandos
——-
With the newest class of graduates, there are currently more than 5,200 members of the commando brotherhood poised to respond wherever there country needs them, whether that calls for the capture of key al-Qaida leaders, improving the quality of life in villages, bringing the fight to insurgent forces or any of the other missions the incorruptible force has accomplished in the past.
http://www.dvidshub.net/news/68138/commando-brotherhood-increases-more-than-200

anan April 27, 2011 at 3:49 am

The ANA Commandos and Special Forces have low attrition.

Many press reports and ISAF press releases on them are inaccurate. Including regarding their numbers. The ANA commando brigade has been split into two commando brigades and 1 forming Special Forces brigade group with a planned:
-72 A-teams
-4 battalion HQs

The 9 commando combat battalions and 1 special force battalion equivalent are the real deal and cause a lot of concern over the Durand.

M Shannon April 26, 2011 at 3:55 pm

In a really bad outfit attrition can lower the standard. As sensible recruits learn just how bad things are they leave while the dregs hang in for a pay check. See the US Army post 1975 for an example.

Don Bacon April 27, 2011 at 1:02 am

General Caldwell is a student of General Petraeus.

General Petraeus’s ability to forecast the success of foreign army training is not as great as his ability to pander to presidents.

Petraeus headed the Iraq Security Transition Command Jun 2004–Sep 2005. After he had been in charge of training the Iraqi Army for three months, he famously wrote an op-ed for the Washington Post which was published in September 2004.

During a crucial time in the presidential campaign, when Kerry said Iraq was headed south, Petraeus’s piece was a rosy prediction of things to come. “Six battalions of the Iraqi regular army and the Iraqi Intervention Force are now conducting operations. . .Within the next 60 days, six more regular army and six additional Intervention Force battalions will become operational. . . Nine more regular army battalions will complete training in January”

Nope.

Sep 2005–Gen. George W. Casey Jr., who oversees U.S. forces in Iraq, said there are fewer Iraqi battalions at “Level 1″ readiness than there were a few months ago. . . The number of Iraqi army battalions that can fight insurgents without U.S. and coalition help has dropped from three to one, top U.S. generals told Congress yesterday.

From 21 battalions to one. Quite a drop.

General Caldwell prefers the Chicago Tribune.

Chicago Tribune
The unnoticed surge in Afghan security
By William B. Caldwell IV
February 15, 2011
A little known but potentially decisive story developing in Afghanistan is the “surge of Afghans,” that is, how Afghan men and women have swelled the ranks of the Afghan National Security Forces to levels more than double the U.S. surge.

The surge of Afghans is the remarkable story of the tremendous growth of the Afghan National Security Force, a story will only continue as the army and police grow by an additional 35,000 by the end of October.
http://www.chicagotribune.com/news/opinion/ct-oped-0215-afghan-20110215,0,4075776.story

anan April 27, 2011 at 3:51 am

Don Bacon, Petraeus never over stated the success of the ISF. Rather he understated ISF success to “lower expectations.”

Caldwell has also not overstated ANSF success.

anan April 27, 2011 at 3:27 am

“I’ve never been a big proponent of attempts to “sell” the success of the Afghan National Security Forces, especially the National Police.”
Has anyone sold the ANSF? If so, who? I haven’t seen it. There is however a PR campaign against the ANSF, much of it directed by the Pakistani army and gullibly repeated by uninformed western press.

“while we can all point to individual examples of certain units succeeding, by and large we have no sense of how well the police are doing” I don’t follow. There are many data points regarding specific units and specific areas. At the aggregate level as well, there is substantial information available.

“and, worryingly, we have lots of stories about how badly they’re doing in terms of operational readiness and retention”
The AUP has high retention. Is there any reason to believe that AUP retention won’t stay high?

The unit with high turnover is ANCOP. ABP is in between.

“General Caldwell, who runs the training mission in Kabul, wrote a long op-ed about how well his training mission is going. He did so by pointing, exclusively, to recruitment numbers.”

The OPED was very short and Caldwell avoided mentioning many statistics that NTM-A freely gives out to avoid confusing people. Recruitment numbers are not over emphasized by NTM-A and the ANSF training commands in briefings.

“recruitment numbers don’t really tell you much about your training effectiveness, operational capacity, and overall effect on the war or on certain operations.”

Far more important than recruitment numbers is how many ANSF are training at any given time. Flows matter more than stocks. This is the most important flow number.

When Obama was elected President only 1 thousand ANP were trained at any given time. Today it is 12 thousand on the way to 24 thousand. [By contrast the geographically and population wise smaller Iraq has trained near 40 thousand policepeople at any given time since the mid 2000s.]

“as the commander of the effort, Caldwell has every incentive to portray his efforts as successful, just as he is under no obligation to highlight his failures for public consumption. This is true at all levels of command”
Joshua, Caldwell hasn’t promoted the ANSF. Nor has Petraeus. Nor has ISAF. They have done the “OPPOSITE” of promoting the ANSF in an attempt to resource ANSF combat enablers [they continually imply that the ANSF isn't getting enough resources in private and to a lesser degree in public.] No one is promoting the ANSF other than to some extent the ANSF themselves. That is a problem.

“and it is why I get so frustrated at the obviously lopsided portrayals of the war by senior officers.” Josh, they are saying what they believe. Would you rather they didn’t?

To summarize the most important events taking place in Afghanistan today . . . There is a major surge in ANSF capacity and quality and a corresponding surge in Taliban capacity [assisted by the Taliban's international backers.] The primary short term aim of the Taliban alliance and its international supporters is to persuade the international community to reduce support for the ANSF.

“SIGAR has audited the ANP. And the story they tell is not very pretty”

SIGAR has said little that isn’t widely known by ANSF watchers, other than providing some statistics on specific Afghan regions. Generally the quality of SIGAR reports is pretty low.

“U.S. auditors have found that the Afghan government cannot determine how many people work for its national police force”
Duh! The MoI doesn’t have workable computer systems. Never has. This is partly by design to assuage the deep state.

“making it difficult to determine whether the money is being properly spent.” Duh. No computer system means SIGAR doesn’t know how to measure effectiveness. Are we 5 years old?

“10 percent difference in records and databases maintained by the Ministry of Interior, which runs the police. The various record-keeping systems showed that as of Sept. 30, 2010, the size of the force ranged from 111,774 to 125,218 personnel.” Has been widely known since last year. Why reports like this are called news escapes me.

“The Ministry of Interior cannot accurately determine the actual number of personnel that work for ANP because it has been unable to reconcile its personnel records with ANP personnel available for work,” the report said.
More obvious blah blah blah. Saying nothing of any use. The MoI has multiple overlapping record systems. What else is new.

“This is, to put it bluntly, basic record keeping that shouldn’t be a challenge in the year 2011.”
Completely lost me here. The failure to provide MoI ministry enablers before November, 2009, was intentional. This was by design. ISAF, NATO, EU and America didn’t see this as their responsibility.

“Not even in Afghanistan, which has computers and people who know how to use them”

Completely incorrect. The MoI still refuses to provide training that lasts more than 6 months for any ANP officer or NCO. To imply that MoI has computer savvy Afghans is inaccurate. The MoI has never been given the funding to educate any ANP. Again, intentionally.

“supported, very generously, by international donors including the U.S.” Simply untrue. The ANSF were not supported by the international community before November, 2009. Especially the ANP. The reason for this was because of fear that Pakistan might support the Taliban and Al Qaeda in retaliation.

“In an appendix to the report (pdf), one of Caldwell’s subordinates urges SIGAR to stop holding the Afghan Ministry of the Interior to western accounting standards.” Couldn’t agree more. Many of the SIGAR writers are incompetent and out of touch to put it delicately.

“This is an understandable concern, however it’s not really relevant: we have had ten years for training, record-keeping, and so on.” What are you talking about? Which countries have been helping the MoI with training, record-keeping and so on?

“That the Afghan Police still don’t operate in a minimally effective way is a stinging indictment not of them, but of the people training them—which is, by sheer weight of cash, the United States and the UNDP.”

No one . . . to repeat . . . “NO ONE” . . . was training the ANP before Obama was elected in any material way.

“The focus on computers and automation strikes me as deeply misguided” Agreed.

“the Afghans trying to administer the program need to start with the basics of record-keeping and administration, not complicated electronic systems that might not even be relevant to the effort.” Agreed. More importantly, the number of ANP training at any given time needs to be increased to 40 thousand ASAP. This would enable longer training cycles, especially for NCOs and officers. Better educated ANP officers and NCOs would do more to improve the MoI record keeping system tan anything else.

“Despite, or perhaps because of, our inability to track who is serving and who is drawing a paycheck, we cannot guarantee that the ANP will perform effectively when required to. That remains the real scandal at the heart of the training mission—and the heart of why our chosen path toward success is more fraught than we seem willing to admit.”

All war is uncertain. The Taliban are not that formidable of a foe. And their financing, officers and logistics are limited. Why might supporting the ANSF not work?

anan April 27, 2011 at 3:44 am

“Could I offer that retention may also not be a terribly important metric?” Or rather, retention of NCOs and officers is a much more important metric than aggregate retention.

“since the quality of the force is so low” which unit are you refering to?

“a great deal of selective attrition/firing might be the best possible policy to raise quality and effectiveness.” Agreed on this. If Karzai did this, many in the ANSF would cheer.

“You are correct, most of these metrics are just to make Caldwell look good.” Here I don’t agree. Caldwell has not over promoted the ANSF. Far from it. He is still fighting for more resources for NTM-A and the ANSF and is therefore arguing that internationals haven’t done enough to assist the ANSF.

“I have spoken to countless ANP members. They are clear that when push comes to shove they need to hightail out of the zone when things become too hot to handle. It does make sense. They are lightly armed gendarmie and have no place in a combat zone. One certainly cannot blame them.”

For most AUP this is true. Many AUP want to be armed with mortars, light artillery and other combat enablers. This is opposed by some regional powers and has therefore not happened.

I think there is a strong case for more heavily arming ABP and ANCOP.

“It is up to the Afghans the ANSF to decide the final outcome of this war.” True.

“These metrics will never be able to measure fighting spirit.” Metrics can say a lot. Many ANSF have a lot of fighting spirit but lack the education, training, teamwork, planning and coordination to channel their fighting spirit effectively. The Taliban invest more in their officers and embedded combat advisors than the MoI invests in ANP officers and NCOs. Even today.

“Until there is a resurgence in the Afghan leadership at all levels this will just not happen.” Assume you are only talking about MoI. If so, I believe that MoI minister Mohammedi is a good leader and has a lot of fighting spirit. Not as sure about ANP training command commanding MG Pattang. What are your thoughts on him.

“And, it cannot happen ever under the auspices of Caldwell and ISAF.” To be blunt, fighting spirit never was and never will be ISAF’s responsibility. That responsibility belongs to MoI and Karzai. Similarly to blame Caldwell for ANP fighting spirit is absurd.

However Caldwell should be judged on how much he helps MG Pattang, MG Karim, as well as how well he does at soliciting international aid for the ANSF and how well he coordinates international aid for the ANSF.

Caldwell is not a field commander or an operational advisor. He is an enabler of ANSF force generators.

Caldwell has his faults. But why is there so much antogonism towards him? He has done more to resource the ANSF than almost anyone else.

Don Anderson April 28, 2011 at 2:24 pm

Anan- “Hobby Man”,

Let me point out some basic stuff you should know as you go forward in life and with your “Afghan Hobby Blogging” which you are doing since it is obvious that you have never ever been to Afghanistan and know very little of the real world out here. You seem to be doing this for some strange reason of your own.

I guess it is better than you being a stalker or child predator, but still pretty strange. You are out there. You should have an excuse for all of this, I imagine?

A. For some of us this is not a hobby. We live here. People are dying and we often knew them before their death. This is a serious war that we have put many years into. It is not a hobby with information gleaned from a PAO or newspaper article.

B. Reality and what the Generals and other self serving careerists out here preach is completely different. When they present statistics it more than often is completely not true. Not to break your bubble or hobby facts, but this is sadly true.

You can “believe” that they “believe” what they are saying it is true. You can also believe in Santa Claus. It is up to you.

What Mr. Foust is pointing out are the obvious discepancies from what the self serving careerists are saying and the small glimpse of reality that appears from the small glimpse of information that you get from the data that is realeased -especially when it is usually just force fed propaganda from ISAF itself.

C. Many Afghan units look great on paper but in reality barely exist. Many are staffed by “ghost soldiers” who appear on the unit list but are actually not there. The American “inspectors” who come in and run out for fear of dying via IED or most recently blue on Afghan blue incidents have no way of knowing what is going on. Sorry this is another fact.

MoI and the units exist within different universes. MoI often has no clue what is going on in the provinces. No one in their right mind is going to leave the comfort of Kabul to go down to Uruzgan or Khost or Kunar to find out what is really going on. Same goes for Corps Commander in general.

Most are doing as little as they need to. Why bother when ISAF is really in charge and taking credit for each and everything? Why take charge of a war being executed to a great extent to make P4 look good?

Because of how we have “prosecuted the War” the Afghans themselves have been pushed to the side in their own country and their own struggle.

The rating of individual units up to Corps is so superficial as to be ridiculous. I estimate it is always off by at least thirty percent and more in all categories. Thus every other day when you laundry list the strength of different units in the ANA, ANAN you are really lost in space. Things are not as reported at all.

D. Yes, I am implying there is a great deal of cover up going on. This ranges from the top to the bottom of the war effort. The “feel good” story that was presented this last year by P4, soon to leave, has been manufactured with “personal interest” in mind and not facts. He is not the first to be “kicked upstairs” after failing miserably. Still it is very very notable what has happened here.

E. The ANSF are from top to bottom not ready to start fighting by themselves. Caldwell has not done a good job. I would suggest that by presenting such a “pretty picture” to make himself look good he has done a disservice to both Afghanistan and the United States.

Some of us here, not “Afghan Blogging Hobby” people like you are very upset that ten plus years and closing on one trillion dollars and several thousand dead and wounded have been wasted and misused. This has been done by a series of vain and incapable managers who have been more interested in making themselves look good than getting positive results. The results are not there and the pressure these Takfiri criminals are putting on the country is increasing daily.

F. I expect and hope that some time in the future the fraud and incompetence of all of these “managers” will be exposed because they have done a job which borders on criminal and definitely unpatriotic. They are directly responsible for the growth of the enemy and Afghanistan will suffer many years because of their failures. The United States will be less safe as a result of this failure.

This is why many of us are upset and hate to see “Afghan Blogging Hobby” people like you get so excited by what are false facts and incorrect data presented about the ANSF by those careerists
“here today and gone tomorrow” working at ISAF now.

Caldwell is just one example of this incompetence. You, Anan, support him by brainlessly spouting off incorrect facts which you get spoon fed by PAOs and do nothing to intelligently analyse.

That is why we enjoy Mr. Foust’s blog and would never get near yours. He knows and analyzes things, you never do. Have a good day in “Hobby Land.” Greetings from Ghazni, “Hobby Man.”

Don Bacon April 27, 2011 at 11:09 am

AP, Jul 2010: “The Afghan soldiers look like they belong. They wear beards, carry Soviet-era rifles and stride confidently through fields of wheat, melons and okra. . .But in southern Afghanistan, the focus of the U.S. war effort, nearly all the Afghan soldiers are foreigners too. Most don’t even speak the local language. They have to communicate through interpreters hired for the Americans. Despite ethnic quotas and recruiting drives, the Afghan army is still dominated by northern minorities who were oppressed by the Taliban. Nearly all Taliban are ethnic Pashtuns. Although many Pashtuns, the country’s biggest ethnic community, are not connected to the Taliban, the rift between northerners and the southern Pashtuns runs deep.”
http://www.msnbc.msn.com/id/38432732/ns/world_news-south_and_central_asia/

Brookings Afghanistan Index, Mar 29 2011:
ANA Kandaks (battalions) capable of independent action — 0
ANCOP Kandaks ” ” ” — 0
http://www.brookings.edu/~/media/Files/Programs/FP/afghanistan%20index/index.pdf

Feb 23, 2011
NATO: Afghan attrition remains stubbornly high
U.S. Lt. Gen. William Caldwell, the commander of NATO’s training mission in Afghanistan, said the Afghan army loses about 32 percent of its personnel each year. In the police, that number is nearly 23 percent.
http://www.google.com/hostednews/ap/article/ALeqM5hVHvRCD5Eb73ggDtxHiAQ_5owqiQ?docId=7b9ca6ed6f8b4521ae0ac8175e3ae77b

Feb 3, 2011
Caldwell told conference attendees the three “A’s” that he foresees as being the main focus this year.

They are attrition, accountability and Afghans out in front.

The attrition rate amongst the Afghan National Security Force is too high, which won’t allow for planned growth. He added that 98 percent of the attrition rate now comes after the Afghan recruits have arrived at their assigned unit not during their training.

Regarding accountability, Caldwell said this falls under the umbrella of stewardship. Caldwell said he sees teaching and training the Afghans to account for equipment such as vehicles, radios and weapons, as being crucial to their ability to become an enduring and self- sustaining force.

On the third “A,” Caldwell said 2011 is pivotal year. “This is the year we want to ensure the Afghans are out in front.” He reinforced the importance of helping the Afghan leaders be more open to speaking with the press corps, especially the Afghan press. “We must help them understand the importance of sharing their story with their media to their people.”
http://www.ntm-a.com/news/1-categorynews/2035-petraeus-lauds-transition-efforts-in-afghanistan-at-conference?lang=

Caldwell, Feb 15, Chicago Tribune: The surge of Afghans is the remarkable story of the tremendous growth of the Afghan National Security Force, a story will only continue as the army and police grow by an additional 35,000 by the end of October.
http://www.chicagotribune.com/news/opinion/ct-oped-0215-afghan-20110215,0,4075776.story
NOTE: NO MENTION OF ANA UNITS IN THE FIELD — OR ATTRITION,ACCOUNTABILITY or Afghans are out in front — THE “THREE A’S”

Previous post:

Next post: