I love stuff like this. Wether or not it delivers the goods, who knows? But I really think a device like this could make the job of the troops out in the field a little bit easier. I do have some reservations about totally depending on such a device, but if used in conjunction with other evidence collected, these kinds of technologies can help in separating the bad guys from the good guys.
This would have been great at the various ECP’s I have had to run. But contractors being authorized to use such a tool might be a little sticky. There are certain situations though, where something like this could be nice. Like screening your guard force for one. I have been on sites where new guards are streaming into the base on a daily basis. Things like biometrics and this device could help to keep tabs on screening your guard force. But yet again, it always boils down to cost, and the various companies out there can be pretty stingy.
At this time, I have not heard of any companies using the PCASS. Although it would not surprise me that there are a few playing around with such a device. –Head Jundi
The PCASS in action.
New U.S. weapon: Hand-held lie detector
U.S. troops in Afghanistan first to get new device; ‘red’ means you’re lying
By Bill Dedman
Investigative reporter
updated 3:00 a.m. PT, Wed., April. 9, 2008
FORT JACKSON, S.C. – The Pentagon will issue hand-held lie detectors this month to U.S. Army soldiers in Afghanistan, pushing to the battlefront a century-old debate over the accuracy of the polygraph.
The Defense Department says the portable device isn’t perfect, but is accurate enough to save American lives by screening local police officers, interpreters and allied forces for access to U.S. military bases, and by helping narrow the list of suspects after a roadside bombing. The device has already been tried in Iraq and is expected to be deployed there as well. “We’re not promising perfection — we’ve been very careful in that,” said Donald Krapohl, special assistant to the director at the Defense Academy for Credibility Assessment, the midwife for the new device. “What we are promising is that, if it’s properly used, it will improve over what they are currently doing.”
But the lead author of a national study of the polygraph says that American military men and women will be put at risk by an untested technology. “I don’t understand how anybody could think that this is ready for deployment,” said statistics professor Stephen E. Fienberg, who headed a 2003 study by the National Academy of Sciences that found insufficient scientific evidence to support using polygraphs for national security. “Sending these instruments into the field in Iraq and Afghanistan without serious scientific assessment, and for use by untrained personnel, is a mockery of what we advocated in our report.”
The new device, known by the acronym PCASS, for Preliminary Credibility Assessment Screening System, uses a commercial TDS Ranger hand-held personal digital assistant with three wires connected to sensors attached to the hand. An interpreter will ask a series of 20 or so questions in Persian, Arabic or Pashto: “Do you intend to answer my questions truthfully?” “Are the lights on in this room” “Are you a member of the Taliban?” The operator will punch in each answer and, after a delay of a minute or so for processing, the screen will display the results: “Green,” if it thinks the person has told the truth, “Red” for deception, and “Yellow” if it can’t decide.
The PCASS cannot be used on U.S. personnel, according to a memo authorizing its use, signed in October by the undersecretary of defense for intelligence, James R. Clapper Jr.
The Army has bought 94 of the $7,500 PCASS machines, which are sold by Lafayette Instrument Co. of Lafayette, Ind. The algorithm, or computer program that makes the decisions, was written by the Advanced Physics Lab at Johns Hopkins University. Besides the Army, other branches of the U.S. military have seen the device and may order their own. The total cost of the project so far is about $2.5 million.
Congress has not held any hearings on the PCASS device. Myron Young, a spokesman for the Pentagon’s Counterintelligence Field Activity agency, which sponsored development of PCASS, said it informed congressional committees in a memo in November that the device had been approved for use. But five months later, no hearing has been scheduled. Congress has already scaled back its oversight of the polygraph. Five years ago it eliminated a requirement that the Defense Department produce an annual report on polygraph use.
Less accurate than a polygraph
Polygraphs have sparked a fierce debate for at least a century. While supporters claim the devices are reliable, the National Academy of Sciences allows only that they’re “well above chance, though well below perfection.” Polygraphs are not allowed as evidence in most U.S. courts, but they’re routinely used in police investigations, and the Defense Department relies heavily on them for security screening.
Both the proponents and critics agree on one thing: This new device is likely to be less accurate than a polygraph, because it gathers less physiological information.
Like a polygraph, the PCASS uses two electrodes to attempt to measure stress through changes in electrical conductivity of the skin. It also gauges cardiovascular activity, though with a pulse oximeter clipped to a fingertip, rather than a polygraph’s arm cuff, which has the advantage of measuring both pulse rate and blood pressure. Unlike the polygraph, the PCASS does not measure changes in the rate of breathing, and it has no way to detect countermeasures, or efforts to fool the machine, such as by making unusual movements.
The training is different, too. While polygraph examiners for the Defense Department must have four-year college degrees and experience in law enforcement, the PCASS operators are mostly mid-level enlisted personnel and warrant officers, some as young as 20 years old. While polygraph examiners take a 13-week course and a six-month internship, PCASS operators undergo only one week of training, though most have military training as interrogators. The Defense Department says PCASS is simple to operate, because judgment of truthfulness is left to the computer.
Discarding ‘inconclusives’
The debate over the device’s usefulness boils down to a disagreement over its accuracy.
The Pentagon, in a PowerPoint presentation released to msnbc.com through a Freedom of Information Act request, says the PCASS is 82 to 90 percent accurate. Those are the only accuracy numbers that were sent up the chain of command at the Pentagon before the device was approved.
But Pentagon studies obtained by msnbc.com show a more complicated picture: In calculating its accuracy, the scientists conducting the tests discarded the yellow screens, or inconclusive readings.
That practice was criticized in the 2003 National Academy study, which said the “inconclusives” have to be included to measure accuracy. If you take into account the yellow screens, the PCASS accuracy rate in the three Pentagon-funded tests drops to the level of 63 to 79 percent.
Even if you accept the lower accuracy rates, the Pentagon officials say, the device is still better than relying on human intuition.
“Let’s take a worst-case scenario here, and let’s say PCASS really is 60 percent accurate,” said Krapohl, who heads the project for the Defense Academy for Credibility Assessment at Fort Jackson, S.C. “So let’s get rid of the PCASS because it makes errors, and go back to the approach we’re currently using, which has less accuracy? As you can see, that’s really quite untenable if we’re interested in saving American lives and serving the interests of our commanders overseas.”
Msnbc.com asked Fienberg, a professor of statistics and social sciences at Carnegie Mellon University in Pittsburgh, to review the unclassified Pentagon studies of the PCASS. He said he was not impressed.
“I, like everyone else I know, want the troops in Iraq, in Afghanistan, elsewhere in the world, to be protected,” Fienberg said. “I want terrorists to be detected. I do not want to be living in a threatened world, and I want to give my government the best possible advice.
“They need devices that work. And if they rely on things that really don’t work, and act as if they do, we will have a greater disaster on our hands than we already do in the field in Iraq. We simply do not know what a device like this hand-held device will produce in that kind of setting, except for the fact that there’s scant evidence that it will produce anything of value.”
Only ‘a triage device’
Pentagon officials say that the new device will not be used to make final decisions, that the rules forbid it. They say it is intended only to pare down a large group of people to a smaller group that will receive further scrutiny through traditional means, such as interviews or a full polygraph exam.
“The PCASS is envisioned more or less like a triage device,” Krapohl said. “That is, it’s not used as a standalone technology to make final decisions regarding a person’s truthfulness. … There are locals, there are Iraqis and third-country nationals who apply for access to U.S. military bases to provide support functions. And as you might well imagine, there’s not really a good way to do a background investigation of these individuals. And so decisions whether to allow them to come on post and to take these jobs are being based on some pretty imprecise methods, primarily interviews and whatever record checks are available. … So the idea of adding the PCASS is to incrementally improve the decisions that are made so that we protect our forces.”
The term “triage” normally means deciding who gets attention first. But if PCASS is used to pare down a large group to a smaller one, wouldn’t a person who’s red-lighted be denied access to the military post? Krapohl acknowledges that possibility, saying it’s not dissimilar from the ways colleges choose which students to admit.
“Let’s say that they have 10 positions that are open, and they have 100 people who apply, which is very realistic,” Krapohl said. “Whatever tools you use to make that assessment, 90 people are not going to get that job anyway. So your role as a decision-maker is to help improve your decision process by making sure that those 10 are flawless, that those individuals have nothing in their background to raise your attention. Therefore a commander might be inclined if he tests or she tests 100 people, and you get 50 green lights, … to restrict the decisions to just those 50.”
No harm in that, Krapohl said. “That’s how we make decisions for hiring people everywhere, or making decisions on college applications or so forth. Most people are not going to get in. How do you improve the likelihood that those who do get in are going to be good candidates?”
One floor down from Krapohl’s office, the lead instructor for the PCASS project, team chief James Waller, said he’s confident that the science behind the device is solid.
“Spent 20 years in the military. If I didn’t think it would work, I wouldn’t put it in the hands of a soldier, ’cause I was a soldier,” Waller said. “So it’s not perfect, but they don’t have a device at all over there to do this kind of work.”
Testing far from a battlefield
Fueling the debate over the PCASS is the fact that the tests before deployment were all conducted on Americans, in English, far from a battlefield. The Pentagon tested the device on Army recruits at Fort Jackson, S.C., and on civilians in Columbus, Ohio, who answered classified ads. In all, only 226 people were tested in the scientific experiments before the machine was approved for use by soldiers in the field.
Although the device was also tried in the field last year in Iraq and Qatar on allied forces, Iraqi police officers, translators and prison guards, the Defense Department says these were only “field vetting,” not scientific tests. The goal was to make sure the device was easy to use and would hold up to battlefront conditions.
“It’s virtually impossible to do a validity study in a war zone,” Krapohl said.
The desire for a portable lie detector at the war front is described in a memo from Camp Cropper, a U.S. Army detention camp on the outskirts of Baghdad. The unclassified memo from April of last year was written by David Thompson, the team leader of the counterintelligence human intelligence support team at Cropper. In screening Iraqis to work as corrections officers and interpreters, Thompson said, the HUMINT collectors (or human intelligence interrogators) are trying to discover who might be involved in militia or insurgent activities, or who might break the rules by passing information from prisoners to people outside the base.
To make these decisions, Thompson wrote, the interrogators often have little more than gut instinct to go on.
“The ability of the collector to filter truth from fiction is highly dependent on the individual collectors training and real world experience,” Thompson wrote. “As the demand for HUMINT personnel has risen, the standards we use to select applicants to the HUMINT field has dropped, yielding a higher percentage of young and inexperienced personnel.”
So what’s needed, Thompson explains, is a machine.
“While not a perfect technological solution that would completely remove the ‘human factor,’ it appears to be an invaluable tool,” Thompson writes. “PCASS is not a system that will remove the need for experienced collectors but rather will enhance their capabilities. … If the PCASS system gives a ‘red’ reading to one of these questions, the collector can focus his efforts on exploitation, and use the traditional skills of a collector to uncover deception or information.”
‘What’s new here?’
Lie detectors sometimes work because people believe they work, deterring the wrong people from applying for jobs in the first place, or prompting admissions of guilt during interrogations. Thompson explains that may be the most important use of PCASS. “Once this capability is introduced and word spreads of its use, militia or insurgent related individuals will be less likely to attempt to gain admittance” as correctional officers.
Fienberg, the leader of the National Academy of Sciences study, said this memo demonstrates one of his concerns, because the interrogator shifts all of his attention to those who got a red light. The professor said the danger to soldiers doesn’t come from the false positive, the person who fails the lie detector test with a red light despite being truthful. Those people just get further scrutiny, or barred from base. The danger, he said, lies in the false negative, the liar who tests “green” and escapes further scrutiny.
“They’ve been testing polygraph-like devices for a long long time, and this one doesn’t look like it works as well as other ones that were discredited,” Fienberg said. “I believe we need a different way to think about interrogation and information gathering, instead of using the polygraph as a magic bullet.”
The designers of the PCASS said they attempted to make it lean toward detecting deceptive people, because of the disproportionate consequences of green-lighting a liar in a war zone. The Johns Hopkins researchers said they tweaked the algorithm so it takes only a little evidence of deception to turn the lights red. They also tried to minimize the yellow lights, at the Pentagon’s request.
But they acknowledged that this was no easy task. They use the word “non-trivial,” which in scientific lexicon means a problem is difficult, even unsolvable.
“Determining these decision rules,” the researchers wrote, “is both non-trivial and subjective.”
Fienberg, the professor of statistics, said the Pentagon’s pre-deployment studies have several weaknesses: They’re conducted and paid for by the same people promoting the device. They’re small studies, involving few people. The mock crime scenarios don’t match the stresses of a war. And no allowance was made for differences in language and culture. Finally, they assume that about half the population is telling the truth, and half are lying; in the real world, where the number of liars may be very small, the number of false positives (truthtellers who get a red light) can overwhelm the resources of investigators.
“Ask the question, what’s new here?” Fienberg said of the PCASS. “They went to a group that developed a computer-based algorithm that produces green, yellow and red lights. That’s the new technology. It’s still based on the same noisy psycho-physiological channels for detecting deception. So we have no new way to detect deception. What we’ve got is a streamlined version of the polygraph. Therefore you would expect this not to be as accurate as the polygraph.
“Almost everything that has gone on since our report was published is obfuscation,” Fienberg said, “including the use of the euphemism ‘credibility assessment’ to describe the DoD activities.”
Krapohl and the Defense Department say they hope to develop better tools for detecting deception. Meanwhile, by being portable and better than human intuition, PCASS fills a niche in a war zone.
“That’s the best that we can do right now,” Krapohl said. “But if you come back next year or in five years, I might have a different answer for you.”
The PCASS is being developed only for military use. Krapohl said he doesn’t foresee it being sold at the pharmacy for use by parents of teenagers, or by the local police.
“I don’t want to see a state trooper with one in the back seat asking you what you have in the trunk.”
————-
BACKGROUNDER
How accurate is it?
The Defense Department has paid for three studies of the handheld lie detector, PCASS. Here are brief descriptions. Follow the links to read the full study reports.
Test one
In the first test, Army basic trainees at Fort Jackson, S.C., were put through what’s called a mock-crime drill. Some recruits were told to place a fake bomb by a roadside, while others remained inside. After a detonation was heard, each was questioned.
They were given an incentive to tell the truth: If they were found to be deceptive, they would have to give a 10-minute speech to their unit on the subject of honesty. (That threat was itself a lie.) The examiners were all polygraph instructors, and the interviews were in English.
The test was performed in three rounds: first with a traditional polygraph, then twice with PCASS.
In the first round, with a polygraph, the examiner was correct 79.7 percent of the time, or 55 out of 69. The Pentagon prefers to cite an accuracy rate that sets aside the “yellow” results, or inconclusives; that yields a success rate of 85.9 percent, or 55 out of 64. (The polygraph was uncertain in 5 cases out of 69, or 7.2 percent.)
In the second round, with PCASS, the device was correct in 63.0 percent of the cases, or 46 out of 73. The Pentagon cites an accuracy rate, setting aside the inconclusives, of 86.8 percent, or 46 out of 53. (The PCASS was uncertain in 20 out of 73 cases, or 27 percent.)
In the third round, with PCASS, the device was correct in 62.2 percent of the cases, or 51 out of 82. The Pentagon cites an accuracy rate, setting aside the inconclusives, of 73.9 percent, or 51 out of 69. (The PCASS was uncertain in 13 out of 82 cases, or 16 percent.)
Test two
In the second test, in Columbus, Ohio, civilians who answered a classified ad for a scientific study on deception. Their incentive: Participants were paid a $50 bonus if the machine showed them to be truthful. Battelle Memorial Institute, a defense contractor, was paid $305,000 to perform this test. The instructors were experienced law-enforcement polygraphers, and the participants were all American, English-speaking and college-educated.
Again, this was a mock-crime test, with the fake theft of a ring from a secretary’s desk. Both those who did and didn’t steal the ring were then questioned.
The PCASS was correct in 78.9 percent of the cases, or 56 out of 71. Setting aside the inconclusive, the Pentagon cites an accuracy rate of 91.8 percent, or 56 out of 61. (The PCASS was uncertain in 10 out of 71 cases, or 14 percent.)
Test three
In the third test, the algorithm that makes the decisions was tested in the lab by its creators at the Johns Hopkins University Advanced Physics Lab. The university was paid $1.2 million for its work.
The PCASS was still being developed while they wrote the software, so they had no PCASS exam data to work with. They used a set of polygraph exam records, part of the same set of records that were used to develop the device. All of these records had been independently verified, such as using urinalysis to prove whether or not someone was lying about using drugs.
For every 100 deceptive people, the researchers reported, the device would detect 86 (red), with two false negatives (green) and 12 uncertain (yellow).
For every 100 truthful people, they said, it would detect 50 (green), with eight false positives (red) and 42 uncertain (yellow).