NIST Finds Face-Scanning Accuracy Declines on Masked People

Face recognition algorithm accuracy declines substantially with masked faces, the National Institute of Standards and Technology reported Monday. For the most accurate algorithms tested, authentication failure rates increased from 0.3% for unmasked individuals to 5% when scanning digitally masked faces, NIST said.

Less reliable but “competent” algorithms failed between 20% and 50% of the time when scanning masked faces, NIST said. Algorithms weren’t able to extract facial features “well enough to make” effective comparisons among photos, the report said. The agency altered the photos with nine different digital masks of varying size and color.

The COVID-19-driven requirement that “people wear protective face masks in public places has driven a need to understand how cooperative face recognition technology deals with occluded faces,” the agency said.

The dataset included 6.2 million images of 1 million people, and researchers analyzed 89 algorithms. Technologies from Samsung, Panasonic and Canon were among those tested. Researchers collaborated with the Department of Homeland Security’s Science and Technology Directorate, Office of Biometric Identity Management and Customs and Border Protection.

This was the first in a series from the agency’s Face Recognition Vendor Test (FRVT) program on face-covering performance. The agency plans to test accuracy of “algorithms that were intentionally developed with masked faces in mind” later this summer, said NIST computer scientist Mei Ngan.

Algorithms were less accurate the more the digital mask covered the individual’s nose, the report found. NIST explored low, medium and high levels of nose coverage. In general, false negatives, in which the technology failed to match two photos of the same person, increased. False positives, where photos of different people are incorrectly matched, “remained stable or modestly declined,” the report said: “The modest decline in false positive rates show that occlusion with masks does not undermine this aspect of security.”

The report found that error rates were generally lower with round masks and that black masks decreased accuracy more than surgical blue masks. “We expect the technology to continue to improve,” Ngan said. “But the data we’ve taken so far underscores one of the ideas common to previous FRVT tests: Individual algorithms perform differently. Users should get to know the algorithm they are using thoroughly and test its performance in their own work environment.”

Stakeholders and industry groups didn't immediately comment.