Enlarge/ The “barcode” pattern of light and dark points along the seam of a pair of jeans.
72 with 51 posters participating
Is every pair of jeans like no other? According to the testimony of FBI forensic analysts, the patterns seen on denim are reliably unique and can be used to identify a suspect in surveillance footage.
The problem is, this technique has never been subjected to thorough scrutiny, and evidence acquired through it may not be as strong as it has been claimed to be. A paper published in PNAS this week puts denim-pattern analysis through its paces, finding that it isnt particularly good at matching up identical pairs of jeansand may create a number of false alarm errors to boot.
Shoddy evidence
For some time, there have been rumblings about the reliability and quality of commonly used forensic techniques. In 2009, the National Academy of Sciences published a weighty report observing that, apart from nuclear DNA analysis, no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.
The problems with forensic evidenceincluding fingerprint, bloodstain, and ballistics analysishave terrible real-world consequences. According to the National Registry of Exonerations, nearly a quarter of wrongful convictions in the United States for the last 30 years can be attributed to flawed or misleading forensic evidence.
Computer scientists Sophie Nightingale and Hany Farid wanted to look at one technique in particular: photographic pattern analysis, which matches up the patterns of details on faces, hands, or clothing between suspects and crime-scene photographs. Jeans, for example, have a barcode pattern of dark and light splotches along their seams.
Denim barcodes
These patterns have been used as central evidence to convict people, but is this kind of analysis reliable? That hasnt been established. To test it out, Nightingale and Farid went out to buy 100 pairs of jeans from second-hand stores. They laid the jeans out flat on a hard surface, photographed the seams along the legs, and digitally traced the pattern of light and dark points along the seams. To bump up their sample, they had Amazon Turk workers supply images from another 111 pairs, photographed using careful instructions.
Then, the researchers set about quantifying how different the patterns were across different pairs of jeans. Obviously, theres a lot of randomness at play heretwo pairs could be quite similar, just by chance, while another two pairs could be entirely different, also by chance. And most pairs would fall somewhere in the middle, with some degree of similarity. Based on these measurements, Nightingale and Farid worked out the range of similarity between the “barcode” patterns on different pairs of jeans.
The important question, of course, is whether these patterns can be used to determine whether two images show the same pair of jeans. So the researchers selected 10 pairs of jeans and took 10 photos of each using different cameras, in different lighting, and with different draping. What they found was that any given pair of photos could come back with a lot of similarities but could also come back with very different readings on the pattern. The range was broadas Nightingale and Farid point out, soft fabric photographed in a bunch of different ways is going to have distortions that vary from one image to the next.
False alarms
So if one pair of jeans can look noticeably different in different photos, is denim-pattern analysis actually a useful forensic technique? The researchers used their measurements to estimate how often a true match would come up and how often their jeans would throw up a false alarma score that looked like a match even though the images actually came from two different pairs.
They found that the false alarm rate could be as high as one in a thousand. Given that the FBI has reported using photographic pattern analysis in hundreds of cases each year, thats a meaningful possibility. The true match rate was also not great, at around 40 to 50 percent, depending on factors like the length of the seam being analyzed.
This means the technique of matching up jeans is likely to be pretty hit and missnot catching actual similarities a lot of the time and possibly throwing up a high rate of false alarms. And thats under controlled experimental conditions using high-quality images and jeans laid out nice and flat, not grainy security footage showing jeans being worn. On the other hand, different features like damage, branding, and size could corroborate an analysis to improve the evidence one way or another.
Theres more work needed on whether jeans could be analyzed in a more reliable way using additional featuresand also whether other pattern analysislike freckles on a face or patterns on other types of clothingare similarly unreliable. But for now, write Nightingale and Farid, identification based on denim jeans should be used with extreme caution, if at all.
PNAS, 2020. DOI: 10.1073/pnas.1917222117  (About DOIs).