With governments globally looking to strengthen national security and businesses trying to improve security and reduce identity theft, interest in biometrics remains high. Forecasts suggest the industry will see annual growth exceeding 20% over the next five years and, in response, suppliers have been innovating to improve technical performance.
Biometric system suppliers will always want to present their capabilities in as favourable light as possible. However, capability testing based solely on in-house tests and results means that the stated performance levels may not always be realised in practice.
Before a system is purchased, real world testing of biometrics is essential to ensuring systems perform as required, in the environment where they are going to be used.
These tests should include consideration of:
Dataset sizes and populations: If the dataset size or volunteer populations used to test a biometric system are not representative of the buyer’s particular needs and user populations then the results will not reflect real world operations. For example, if the system needs to operate in an ethnically diverse society, a small volunteer test population of a single ethnicity will not provide results with statistically meaningful confidence levels. However, as PA’s work on the UK’s National Identity Scheme has showed, it is possible to develop detailed test specifications to address these issues.
The recording environment: Work undertaken by the US National Institute of Science and Technology (NIST) and the UK National Physical Laboratory (NPL) has shown that a biometric system’s performance can be vastly improved if particularly high quality biometric images are used. Test datasets are frequently recorded in laboratory-like environments, by highly skilled technicians who can repeat recordings. In the real world lower quality images are likely to be the norm, with a consequent drop in operational system performance.
Security and tolerances: Buyers need to consider whether the system will be focused on high security or high throughput. There is a trade-off between the two and settings, tolerances and thresholds for liveness detection, spoof resistance and image quality should be set appropriately for particular operational needs, and the tests conducted on this basis.
Other important factors include the need to protect the real identities of volunteers and of all biometric samples to ensure they are not mislaid or fall into the wrong hands; allowing sufficient time windows between enrolment and validation attempts; and the nature and extent of spoof resistance testing. PA’s work in biometric system testing has shown that biometric systems can easily be spoofed so buyers need to test this vulnerability.
Undertaking testing is rarely cheap but when set against the investment needed for biometric deployment and the desired business benefits it is a wise use of money.
PA’s experience across a wider range of sectors has consistently shown that independent tests give a clear understanding of real world performance, reduce the risks in the business case, and help to inform the selection of the most appropriate supplier and equipment.
To find out more about how we can help you secure the right biometric solution for your organisation, please contact us now.