Benchmarks
SourceAFIS is a free algorithm that is primarily designed for ease of use, but it is important to measure and publish performance data for the algorithm. Measurement is done via third party competitions and using associated SourceAFIS CLI tool. Benchmark results are summarized on this page.
Accuracy
SourceAFIS calculates similarity score of two fingerprints, which is then compared to some application-chosen threshold to arrive at match/non-match decision. This decision can suffer from two types of errors:
- False match (also called false positive or type I error) occurs when non-matching pair scores above threshold and it is thus erroneously considered matching.
- False non-match (also called false negative or type II error) occurs when matching pair scores below threshold and it is thus erroneously considered non-matching.
| Same person | Different person | |
|---|---|---|
| Matched | true match | false match |
| Not matched | false non-match | true non-match |
Accuracy is then a measure of how often these errors occur:
- FMR(t) or False Match Rate is the probability of false match at given threshold.
- FNMR(t) or False Non-Match Rate is the probability of false non-match at given threshold.
- EER or Equal Error Rate (also called Crossover Error Rate (CER)) is FMR or FNMR at threshold chosen so that FMR(t) = FNMR(t), i.e. where the two types of errors are balanced.
- FMR100, FMR1000 (FMR1K), or FMR10000 (FMR10K) is FNMR at threshold chosen so that FMR is 1%, 0.1%, or 0.01% respectively.
- Zero FMR (FMR0) is FNMR at the lowest threshold that yields zero FMR.
Accuracy is measured on a dataset of sample fingerprints. Since algorithm accuracy varies with test dataset and algorithms can be tuned for particular dataset, fair comparison between algorithms requires secret dataset and a trusted third party that evaluates algorithms on the dataset. SourceAFIS has been submitted to FVC-onGoing competition for this purpose. Results are below.
| Benchmark area | Benchmark | Algorithm | Version | EER | FMR100 | FMR1K | FMR10K | Zero FMR | Details |
|---|---|---|---|---|---|---|---|---|---|
| Fingerprint Verification | FV-STD | SourceAFIS | 3.14.0 | 3.87 % | 4.73 % | 6.55 % | 9.05 % | 10.6 % | submission 8426 |
| Fingerprint Verification | FV-HARD | SourceAFIS | 3.14.0-net | 8.19 % | 11.6 % | 15.1 % | 19.4 % | 21 % | submission 8427 |
These results show that SourceAFIS is on the lower end of the accuracy spectrum. Closer look reveals that SourceAFIS mostly makes false non-match errors. FNMR peaks around 10% at threshold that yields zero FMR. FMR is therefore a non-issue given sufficiently high threshold. FNMR on the other hand remains an issue even at very low thresholds. SourceAFIS is essentially having trouble matching the lowest quality 5% of fingerprints.
Applications that need to keep FNMR low should either ensure high fingerprint quality during enrollment or capture multiple fingers per subject.
Competition details: SourceAFIS in FVC-onGoing.
Speed
See CLI output.
Template size
See CLI output.