Face clustering benchmarks

Benchmark: ~22000 photos of the first ~2000 people from IMDb-Face dataset with 20-200 photos each.

v4.0.1

Results

{ totalPhotos: 22775 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6483863885839737 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 2622 }
{ clusteredFacesRate: 0.17755806866662152 }
{ clusteredTargetFaces: 959 }
{ clusteredTargetFacesRate: 0.1052111903455842 }
{ shitClusterRate: 0.10638297872340426 }
{ targettedShitClusterRate: 0.06593406593406594 }
{ averageClusterAccuracy: 0.8848324957536976 }
{ averageClusterTargetAccuracy: 0.9186972476685288 }
{ weightedAccuracy: 0.15710914903949313 }
{ weightedTargetAccuracy: 0.09665723099441789 }

v4.0.1 with voting and smaller minSampleSize

Results

{ totalPhotos: 22775 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6483863885839737 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 3145 }
{ clusteredFacesRate: 0.21297487641362498 }
{ clusteredTargetFaces: 1153 }
{ clusteredTargetFacesRate: 0.12649478880965442 }
{ shitClusterRate: 0.13392857142857142 }
{ targettedShitClusterRate: 0.046296296296296294 }
{ averageClusterAccuracy: 0.8452653408860407 }
{ averageClusterTargetAccuracy: 0.9183654501253514 }
{ weightedAccuracy: 0.1800202815119251 }
{ weightedTargetAccuracy: 0.11616844366368956 }

v4.0.1 with voting and sampled rejections

Results

{ totalPhotos: 22775 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6483863885839737 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 3421 }
{ clusteredFacesRate: 0.2316651994311641 }
{ clusteredTargetFaces: 1257 }
{ clusteredTargetFacesRate: 0.13790455293472298 }
{ shitClusterRate: 0.12698412698412698 }
{ targettedShitClusterRate: 0.05785123966942149 }
{ averageClusterAccuracy: 0.865709659123329 }
{ averageClusterTargetAccuracy: 0.9234653901213209 }
{ weightedAccuracy: 0.2005548008302911 }
{ weightedTargetAccuracy: 0.1273500817753703 }

v4.0.1 with voting, sampled rejections and original min sample size

Results

{ totalPhotos: 24225 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6095768833849329 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 2839 }
{ clusteredFacesRate: 0.19225299654635336 }
{ clusteredTargetFaces: 1024 }
{ clusteredTargetFacesRate: 0.11234229292375206 }
{ shitClusterRate: 0.10185185185185185 }
{ targettedShitClusterRate: 0.04854368932038835 }
{ averageClusterAccuracy: 0.894196891388111 }
{ averageClusterTargetAccuracy: 0.9392296444909238 }
{ weightedAccuracy: 0.1719120318717984 }
{ weightedTargetAccuracy: 0.10551521184407087 }

v4.0.1 with voting , sampled rejections, and proper sample size / cluster size ratio

Results

{ totalPhotos: 24225 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6095768833849329 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 3291 }
{ clusteredFacesRate: 0.22286178641565654 }
{ clusteredTargetFaces: 1178 }
{ clusteredTargetFacesRate: 0.1292375205704882 }
{ shitClusterRate: 0.07534246575342465 }
{ targettedShitClusterRate: 0.029411764705882353 }
{ averageClusterAccuracy: 0.9090423913173917 }
{ averageClusterTargetAccuracy: 0.9500084568014378 }
{ weightedAccuracy: 0.20259081125655423 }
{ weightedTargetAccuracy: 0.12277673747801357 }

v4.0.1 with voting , sampled rejections, proper sample size / cluster size ratio and pruning fix

Results

{ totalPhotos: 24225 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6095768833849329 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 3340 }
{ clusteredFacesRate: 0.2261799959368863 }
{ clusteredTargetFaces: 1201 }
{ clusteredTargetFacesRate: 0.1317608337904553 }
{ shitClusterRate: 0.06711409395973154 }
{ targettedShitClusterRate: 0.02877697841726619 }
{ averageClusterAccuracy: 0.9126709292573599 }
{ averageClusterTargetAccuracy: 0.9493285761299435 }
{ weightedAccuracy: 0.20642790707114392 }
{ weightedTargetAccuracy: 0.12508432473198708 }

v4.0.1 with voting , sampled rejections, reduced sample size / cluster size ratio and pruning fix

Results

{ totalPhotos: 24225 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6095768833849329 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 3412 }
{ clusteredFacesRate: 0.23105573237624433 }
{ clusteredTargetFaces: 1214 }
{ clusteredTargetFacesRate: 0.13318705430608888 }
{ shitClusterRate: 0.04790419161676647 }
{ targettedShitClusterRate: 0.019867549668874173 }
{ averageClusterAccuracy: 0.928979062345268 }
{ averageClusterTargetAccuracy: 0.9632300892368473 }
{ weightedAccuracy: 0.21464593761238263 }
{ weightedTargetAccuracy: 0.1282897782044468 }

v4.0.1 with voting , sampled rejections, proper sample size / cluster size ratio and alpha factor = 0.6

Results

{ totalPhotos: 24225 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6095768833849329 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 3718 }
{ clusteredFacesRate: 0.25177761224351597 }
{ clusteredTargetFaces: 1307 }
{ clusteredTargetFacesRate: 0.14339001645639057 }
{ shitClusterRate: 0.10596026490066225 }
{ targettedShitClusterRate: 0.034482758620689655 }
{ averageClusterAccuracy: 0.8612823800669005 }
{ averageClusterTargetAccuracy: 0.9113445503727365 }
{ weightedAccuracy: 0.21685162112065662 }
{ weightedTargetAccuracy: 0.13067771007538856 }

v4.0.1 with voting , sampled rejections, proper sample size / cluster size ratio and alpha factor = 0.8

Results

{ totalPhotos: 24225 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6095768833849329 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 3780 }
{ clusteredFacesRate: 0.25597616306629645 }
{ clusteredTargetFaces: 1335 }
{ clusteredTargetFacesRate: 0.14646187602852442 }
{ shitClusterRate: 0.09933774834437085 }
{ targettedShitClusterRate: 0.04895104895104895 }
{ averageClusterAccuracy: 0.8559242912656785 }
{ averageClusterTargetAccuracy: 0.8997387909814197 }
{ weightedAccuracy: 0.21909621595342754 }
{ weightedTargetAccuracy: 0.13177743126277514 }

v4.0.1 with voting , sampled rejections, increased min sample size and alpha factor = 0.8

Results

{ totalPhotos: 24225 }
{ detectedFaces: 14767 }
{ detectedFacesRate: 0.6095768833849329 }
{ detectedTargetFaces: 9115 }
{ clusteredFaces: 2492 }
{ clusteredFacesRate: 0.16875465565111397 }
{ clusteredTargetFaces: 866 }
{ clusteredTargetFacesRate: 0.0950082281952825 }
{ shitClusterRate: 0.09722222222222222 }
{ targettedShitClusterRate: 0.08571428571428572 }
{ averageClusterAccuracy: 0.8765403514566142 }
{ averageClusterTargetAccuracy: 0.9022357439479288 }
{ weightedAccuracy: 0.14792026517436735 }
{ weightedTargetAccuracy: 0.08571981944694529 }

v4.0.1 with voting , sampled rejections, reduced sample size / cluster size ratio, pruning fix and fixed dataset

Results

{ totalPhotos: 41119 }
{ detectedFaces: 27978 }
{ detectedFacesRate: 0.680415379751453 }
{ detectedTargetFaces: 7678 }
{ clusteredFaces: 7726 }
{ clusteredFacesRate: 0.27614554292658516 }
{ clusteredTargetFaces: 1920 }
{ clusteredTargetFacesRate: 0.25006512112529306 }
{ shitClusterRate: 0.08136482939632546 }
{ targettedShitClusterRate: 0.02109704641350211 }
{ averageClusterAccuracy: 0.8972526496222504 }
{ averageClusterTargetAccuracy: 0.9576253187048861 }
{ weightedAccuracy: 0.24777232007225342 }
{ weightedTargetAccuracy: 0.2394686913145847 }

v4.0.1 with voting , sampled rejections, reduced sample size / cluster size ratio, pruning fix, fixed dataset and unbatched clustering

Results

{ totalPhotos: 28751 }
{ detectedFaces: 26483 }
{ detectedFacesRate: 0.9211157872769643 }
{ detectedTargetFaces: 7267 }
{ clusteredFaces: 7625 }
{ clusteredFacesRate: 0.28792055280746137 }
{ clusteredTargetFaces: 1772 }
{ clusteredTargetFacesRate: 0.2438420255951562 }
{ averageTargetFacesPerIdentity: 5.196480938416422 }
{ averageClusteredTargetFacesByIdentity: 4.422287390029325 }
{ clusteredTargetFacesByIdentityRate: 0.6211279158486672 }
{ identitiesWithPhotos: 710 }
{ identitiesWithDetections: 341 }
{ identitiesWithEnoughDetections: 245 }
{ identitiesWithClusters: 227 }
{ identitiesWithClustersRate: 0.926530612244898 }
{ shitClusterRate: 0.06178489702517163 }
{ targettedShitClusterRate: 0.012145748987854251 }
{ averageClusterAccuracy: 0.9100092673206291 }
{ averageClusterTargetAccuracy: 0.9598004757938043 }
{ weightedAccuracy: 0.26201037130686844 }
{ weightedTargetAccuracy: 0.23403969218475593 }

v4.1.0

Results

{ totalPhotos: 30751 }
{ detectedFaces: 26484 }
{ detectedFacesRate: 0.8612402848687848 }
{ detectedTargetFaces: 7267 }
{ clusteredFaces: 8421 }
{ clusteredFacesRate: 0.31796556411418214 }
{ clusteredTargetFaces: 2059 }
{ clusteredTargetFacesRate: 0.2833356268061098 }
{ averageTargetFacesPerIdentity: 5.656593406593407 }
{ averageClusteredTargetFacesByIdentity: 4.664835164835165 }
{ clusteredTargetFacesByIdentityRate: 0.5979721111772682 }
{ identitiesWithPhotos: 2001 }
{ identitiesWithDetections: 364 }
{ identitiesWithEnoughDetections: 272 }
{ identitiesWithClusters: 238 }
{ identitiesWithClustersRate: 0.875 }
{ shitClusterRate: 0.0821917808219178 }
{ targettedShitClusterRate: 0.02631578947368421 }
{ averageClusterAccuracy: 0.881315888643071 }
{ averageClusterTargetAccuracy: 0.9255094181587252 }
{ weightedAccuracy: 0.2802281036951858 }
{ weightedTargetAccuracy: 0.2622297911089604 }
{ combinedScore: 0.6086151415765864, minCombinedScore: 0.6 }

v4.1.0 + increase cluster separation to 0.35

Results

{ totalPhotos: 30751 }
{ detectedFaces: 26484 }
{ detectedFacesRate: 0.8612402848687848 }
{ detectedTargetFaces: 7267 }
{ clusteredFaces: 8442 }
{ clusteredFacesRate: 0.3187584956955143 }
{ clusteredTargetFaces: 2079 }
{ clusteredTargetFacesRate: 0.2860877941378836 }
{ averageTargetFacesPerIdentity: 5.711538461538462 }
{ averageClusteredTargetFacesByIdentity: 4.815934065934066 }
{ clusteredTargetFacesByIdentityRate: 0.6181629470580804 }
{ identitiesWithPhotos: 2001 }
{ identitiesWithDetections: 364 }
{ identitiesWithEnoughDetections: 272 }
{ identitiesWithClusters: 243 }
{ identitiesWithClustersRate: 0.8933823529411765 }
{ shitClusterRate: 0.06818181818181818 }
{ targettedShitClusterRate: 0.022813688212927757 }
{ averageClusterAccuracy: 0.8997421100912804 }
{ averageClusterTargetAccuracy: 0.9466899220823666 }
{ weightedAccuracy: 0.28680044152660433 }
{ weightedTargetAccuracy: 0.27083643154110915 }
{ combinedScore: 0.6218875401497135, minCombinedScore: 0.6 }