docs: full data-grounded rewrite of the performance report
Replaces narrative claims with verified numbers across all report pages: - Cross-model held-out validation (LVFace/mbf/r18, all 5 held-out films): LVFace wins every film outright, not just "consistent with" the training-set pick. r50 dropped from the detailed comparison (gallery has ~30% fewer reference images per actor than the other three models on identical source photos). - Per-film training breakdown: LVFace does not win every training film (mbf beats it on Lord of War); the 75.3% macro figure hides a 10.7pp spread. - Gallery coverage computed per film (20.3%-78.6%) instead of one flat 67%-missing average. - Found and fixed a real scoring bug in optimize.py: a candidate whose hardest film's replay timed out was averaged over survivors instead of penalized, silently rewarding partial coverage. Affected 3 of 16 training combos; corrected throughout, and optimize.py now scores an incomplete evaluation f1=0.0 instead of averaging over whichever films happened to finish. - Every FPI frame in the deep dive now comes from the proper montage renderer (Onscreen/Offscreen panel, ghosts never drawn as boxes), never the bare-box debug overlay used earlier. - Every distinct out-of-cast name across all 9 films gets its own frame at its first appearance (9 names, 4 films), not a single-example spot check: 2 ground-truth gaps, 1 photograph misread as a person, 6 genuine lookalike confusions. - New methodology.md: the scene-level-vs-per-second scoring mismatch that the rest of the report assumes, written out once. - Cut the deadlock/gdb debugging narrative from the experiment log; kept the one fact that matters (KPN's node/network split lets the expensive GPU stage run once and the cheap stage replay against cached embeddings). - Plain declarative style throughout, no em dashes, no blog voice.
This commit is contained in:
@@ -0,0 +1,221 @@
|
||||
{
|
||||
"LVFace-B_Glint360K": {
|
||||
"config": {
|
||||
"prob_threshold": 0.7540024664611272,
|
||||
"anneal_sec": 35.53996030397922,
|
||||
"extinction_sec": 57.43359645269811
|
||||
},
|
||||
"films": {
|
||||
"Caf\u00e9_Society": {
|
||||
"name": "Caf\u00e9 Society",
|
||||
"TPI": 14499,
|
||||
"FPI": 1380,
|
||||
"FPI_misid": 0,
|
||||
"FPI_incast": 1380,
|
||||
"FN": 12231,
|
||||
"precision": 0.9130927640279615,
|
||||
"precision_raw": 0.9130927640279615,
|
||||
"recall": 0.5424242424242425,
|
||||
"f1": 0.6805604449764134,
|
||||
"agreement_rate": 0.57285804629501,
|
||||
"exact_match_rate": 0.18947003810183582,
|
||||
"n_seconds": 5774,
|
||||
"duration_sec": 5774.0
|
||||
},
|
||||
"Lord_of_War": {
|
||||
"name": "Lord of War",
|
||||
"TPI": 13893,
|
||||
"FPI": 1654,
|
||||
"FPI_misid": 174,
|
||||
"FPI_incast": 1480,
|
||||
"FN": 5737,
|
||||
"precision": 0.811838952842868,
|
||||
"precision_raw": 0.8936129156750499,
|
||||
"recall": 0.7077432501273561,
|
||||
"f1": 0.7562256756388972,
|
||||
"agreement_rate": 0.7005158404089996,
|
||||
"exact_match_rate": 0.38715420432758146,
|
||||
"n_seconds": 7302,
|
||||
"duration_sec": 7302.0
|
||||
},
|
||||
"Scarface": {
|
||||
"name": "Scarface",
|
||||
"TPI": 20518,
|
||||
"FPI": 1078,
|
||||
"FPI_misid": 58,
|
||||
"FPI_incast": 1020,
|
||||
"FN": 14722,
|
||||
"precision": 0.9276607288181572,
|
||||
"precision_raw": 0.9500833487682904,
|
||||
"recall": 0.5822360953461975,
|
||||
"f1": 0.7154363820216882,
|
||||
"agreement_rate": 0.6297735703976657,
|
||||
"exact_match_rate": 0.25910733470065433,
|
||||
"n_seconds": 10239,
|
||||
"duration_sec": 10239.0
|
||||
},
|
||||
"Sound_of_Metal": {
|
||||
"name": "Sound of Metal",
|
||||
"TPI": 13349,
|
||||
"FPI": 677,
|
||||
"FPI_misid": 0,
|
||||
"FPI_incast": 677,
|
||||
"FN": 6504,
|
||||
"precision": 0.9517324967916726,
|
||||
"precision_raw": 0.9517324967916726,
|
||||
"recall": 0.6723920818012391,
|
||||
"f1": 0.7880397886596416,
|
||||
"agreement_rate": 0.7112222835587533,
|
||||
"exact_match_rate": 0.40311896218603366,
|
||||
"n_seconds": 7246,
|
||||
"duration_sec": 7246.0
|
||||
}
|
||||
}
|
||||
},
|
||||
"arcface_w600k_mbf": {
|
||||
"config": {
|
||||
"prob_threshold": 0.8371114538930519,
|
||||
"anneal_sec": 48.80011450565114,
|
||||
"extinction_sec": 59.3110040220424
|
||||
},
|
||||
"films": {
|
||||
"Caf\u00e9_Society": {
|
||||
"name": "Caf\u00e9 Society",
|
||||
"TPI": 13456,
|
||||
"FPI": 1434,
|
||||
"FPI_misid": 180,
|
||||
"FPI_incast": 1254,
|
||||
"FN": 13274,
|
||||
"precision": 0.8150211992731677,
|
||||
"precision_raw": 0.9036937541974479,
|
||||
"recall": 0.5034044145155256,
|
||||
"f1": 0.622386679000925,
|
||||
"agreement_rate": 0.5463565775524695,
|
||||
"exact_match_rate": 0.19154832005542086,
|
||||
"n_seconds": 5774,
|
||||
"duration_sec": 5774.0
|
||||
},
|
||||
"Lord_of_War": {
|
||||
"name": "Lord of War",
|
||||
"TPI": 13778,
|
||||
"FPI": 1740,
|
||||
"FPI_misid": 60,
|
||||
"FPI_incast": 1680,
|
||||
"FN": 5852,
|
||||
"precision": 0.8580146967243741,
|
||||
"precision_raw": 0.8878721484727413,
|
||||
"recall": 0.7018848700967907,
|
||||
"f1": 0.7721362923111411,
|
||||
"agreement_rate": 0.6881858851455998,
|
||||
"exact_match_rate": 0.36469460421802247,
|
||||
"n_seconds": 7302,
|
||||
"duration_sec": 7302.0
|
||||
},
|
||||
"Scarface": {
|
||||
"name": "Scarface",
|
||||
"TPI": 18863,
|
||||
"FPI": 862,
|
||||
"FPI_misid": 0,
|
||||
"FPI_incast": 862,
|
||||
"FN": 16377,
|
||||
"precision": 0.956299112801014,
|
||||
"precision_raw": 0.956299112801014,
|
||||
"recall": 0.535272417707151,
|
||||
"f1": 0.6863640498499044,
|
||||
"agreement_rate": 0.594178111391709,
|
||||
"exact_match_rate": 0.2357652114464303,
|
||||
"n_seconds": 10239,
|
||||
"duration_sec": 10239.0
|
||||
},
|
||||
"Sound_of_Metal": {
|
||||
"name": "Sound of Metal",
|
||||
"TPI": 12642,
|
||||
"FPI": 554,
|
||||
"FPI_misid": 0,
|
||||
"FPI_incast": 554,
|
||||
"FN": 7211,
|
||||
"precision": 0.9580175810851773,
|
||||
"precision_raw": 0.9580175810851773,
|
||||
"recall": 0.6367803354656727,
|
||||
"f1": 0.7650458410239341,
|
||||
"agreement_rate": 0.687468948385309,
|
||||
"exact_match_rate": 0.3789677063207287,
|
||||
"n_seconds": 7246,
|
||||
"duration_sec": 7246.0
|
||||
}
|
||||
}
|
||||
},
|
||||
"arcface_r18": {
|
||||
"config": {
|
||||
"prob_threshold": 0.8955101189489445,
|
||||
"anneal_sec": 59.08214397442713,
|
||||
"extinction_sec": 59.29474134414983
|
||||
},
|
||||
"films": {
|
||||
"Caf\u00e9_Society": {
|
||||
"name": "Caf\u00e9 Society",
|
||||
"TPI": 12207,
|
||||
"FPI": 1119,
|
||||
"FPI_misid": 60,
|
||||
"FPI_incast": 1059,
|
||||
"FN": 14523,
|
||||
"precision": 0.88035482475119,
|
||||
"precision_raw": 0.9160288158487168,
|
||||
"recall": 0.45667789001122333,
|
||||
"f1": 0.6013892994383683,
|
||||
"agreement_rate": 0.5125626845924863,
|
||||
"exact_match_rate": 0.1674748874263942,
|
||||
"n_seconds": 5774,
|
||||
"duration_sec": 5774.0
|
||||
},
|
||||
"Lord_of_War": {
|
||||
"name": "Lord of War",
|
||||
"TPI": 13122,
|
||||
"FPI": 1409,
|
||||
"FPI_misid": 60,
|
||||
"FPI_incast": 1349,
|
||||
"FN": 6508,
|
||||
"precision": 0.870678787074514,
|
||||
"precision_raw": 0.9030348909228546,
|
||||
"recall": 0.6684666327050433,
|
||||
"f1": 0.7562894441082388,
|
||||
"agreement_rate": 0.6698963754222389,
|
||||
"exact_match_rate": 0.3389482333607231,
|
||||
"n_seconds": 7302,
|
||||
"duration_sec": 7302.0
|
||||
},
|
||||
"Scarface": {
|
||||
"name": "Scarface",
|
||||
"TPI": 16961,
|
||||
"FPI": 685,
|
||||
"FPI_misid": 0,
|
||||
"FPI_incast": 685,
|
||||
"FN": 18279,
|
||||
"precision": 0.961181004193585,
|
||||
"precision_raw": 0.961181004193585,
|
||||
"recall": 0.48129965947786607,
|
||||
"f1": 0.6414173883447416,
|
||||
"agreement_rate": 0.5440709115628456,
|
||||
"exact_match_rate": 0.2017775173356773,
|
||||
"n_seconds": 10239,
|
||||
"duration_sec": 10239.0
|
||||
},
|
||||
"Sound_of_Metal": {
|
||||
"name": "Sound of Metal",
|
||||
"TPI": 12017,
|
||||
"FPI": 591,
|
||||
"FPI_misid": 122,
|
||||
"FPI_incast": 469,
|
||||
"FN": 7836,
|
||||
"precision": 0.8767692981176127,
|
||||
"precision_raw": 0.953125,
|
||||
"recall": 0.6052989472623784,
|
||||
"f1": 0.7161715188176049,
|
||||
"agreement_rate": 0.6594534915815532,
|
||||
"exact_match_rate": 0.3573005796301408,
|
||||
"n_seconds": 7246,
|
||||
"duration_sec": 7246.0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user