MPI AGORA Evaluation

Welcome to AGORA Benchmark

Here we provide a detailed evaluation of 3D Human pose and shape estimation methods on AGORA test images. Please check the github repository for more details on evaluation metric and protocol. Please login to upload your predictions on test images and get the evaluation results.


Leaderboard

SMPL-X Algorithms
AlgorithmNMVENMJEMVEMPJPE
FBBFBBFBBFLH/RHFBBFLH/RH
SMPLify-X [smplx1] 333.1 263.3 326.5 256.5 236.5 187.0 48.9 48.3/51.4 231.8 182.1 52.9 46.5/49.6
ExPose [smplx2] 265.0 184.8 263.3 183.4 217.3 151.5 51.1 74.9/71.3 215.9 150.4 55.2 72.5/68.8
Frankmocap [smplx3] 207.8 204.0 168.3 54.7/55.7 165.2 52.3/53.1
BEDLAM [smplx4] 179.5 132.2 177.5 131.4 131.0 96.5 25.8 38.8/39.0 129.6 95.9 27.8 36.6/36.7
BEDLAM-finetuned [smplx5] 142.2 102.1 141.0 101.8 103.8 74.5 23.131.7/33.2 102.9 74.3 24.729.9/31.3
PIXIE [smplx6] 233.9 173.4 230.9 171.1 191.8 142.2 50.2 49.5/49.0 189.3 140.3 54.5 46.4/46.0
Hand4Whole-finetuned [smplx7] 144.1 96.0 141.1 92.7 135.5 90.2 41.6 46.3/48.1 132.6 87.1 46.1 44.3/46.2
PyMAF-X [smplx8] 141.2 94.4 140.0 93.5 125.7 84.0 35.0 44.6/45.6 124.6 83.2 37.9 42.5/43.7
HybrIK-X [smplx9] 120.5 73.7 115.7 72.3 112.1 68.5 37.0 46.7/47.0 107.6 67.2 38.5 41.2/41.4
OSX [smplx10] 130.6 85.3 127.6 83.3 122.8 80.2 36.2 45.4/46.1 119.9 78.3 37.9 43.0/43.9
HumanWild [smplx11] 125.4 81.9 123.8 81.5 105.3 68.8 31.2 38.5/39.7 104.0 68.5 33.1 36.1/37.3
SMPLer-X [smplx12] 107.2 68.3 104.1 66.3 99.7 63.5 29.9 39.1/39.5 96.8 61.7 31.4 36.7/37.2
AiOS (0.5 score) [smplx13] 97.861.396.060.791.957.6 24.6 38.7/39.6 90.257.1 25.7 36.4/37.3
AiOS (0.3 score) [smplx14] 103.0 63.5 100.8 62.6 98.9 61.0 27.7 42.5/43.4 96.8 60.1 29.2 40.1/40.9
SMPLer-X (AiOS) [smplx15] 102.4 63.8 99.5 62.1 98.3 61.2 30.3 40.4/40.7 95.5 59.6 31.7 37.9/38.2
Multi-HMR [smplx16] 102.0 63.4 101.8 64.1 95.9 59.6 27.7 40.2/40.9 95.7 60.3 29.2 38.1/39.0
NLF-L nonparam. [smplx17] 98.6 62.1 96.6 61.9 92.7 58.4 27.0 37.9/38.1 90.8 58.2 28.5 34.4/34.9
OSX (AiOS) [smplx18] 126.4 81.8 123.4 80.0 121.3 78.5 36.1 45.9/46.3 118.5 76.8 37.6 43.5/44.0
SMPL Algorithms
AlgorithmNMVENMJEMVEMPJPE
EFT [smpl1] 196.3 203.6 159.0 165.4
HMR [smpl2] 217.0 226.0 173.6 180.5
CenterHMR [smpl3] 233.9 242.3 161.4 168.1
SPIN [smpl4] 216.3 223.1 168.7 175.1
SPIN-finetuned [smpl5] 193.4 199.2 148.9 153.4
PARE [smpl6] 167.7 174.0 140.9 146.2
SPEC [smpl7] 126.8 133.7 106.5 112.3
ROMP-finetuned [smpl8] 130.8 134.0 113.8 116.6
BEV-finetuned [smpl9] 108.3 113.2 100.7 105.3
PyMAF [smpl10] 200.2 207.4 168.2 174.2
Hand4Whole-finetuned (body only) [smpl11] 90.2 95.5 84.8 89.8
ROMP2_finetuned [smpl12] 113.6 118.8 103.4 108.1
CLIFF-finetuned [smpl13] 83.5 89.0 76.0 81.0
CLIFF(PRoM)-finetuned [smpl14] 66.3 70.7 61.0 65.0
PLIKS-finetuned [smpl15] 71.6 76.1 67.3 71.5
HybrIK-finetuned [smpl16] 81.2 84.6 73.9 77.0
NIKI-finetuned [smpl17] 70.2 74.0 63.9 67.3
ProPose-finetuned [smpl18] 78.8 82.7 70.9 74.4
BoPR [smpl19] 148.2 154.7 128.9 134.6
BoPR_finetuned [smpl20] 84.7 90.8 74.5 79.9
PyMAF-finetuned [smpl21] 89.9 95.0 84.5 89.3
ReFit [smpl22] 69.6 73.2 62.6 65.9
W-HMR-finetuned [smpl23] 70.4 75.4 63.4 67.9
Multi-HMR (body only) [smpl24] 64.3 68.7 61.1 65.3
AiOS (body only) [smpl25] 61.268.057.563.9

References


Contact & citation

If you use this work, please cite:

@inproceedings{Patel:CVPR:2021,
    title = {{AGORA}: Avatars in Geography Optimized for Regression Analysis},
    author = {Patel, Priyanka and Huang, Chun-Hao P. and Tesch, Joachim and Hoffmann, David T. and Tripathi, Shashank and Black, Michael J.},
    booktitle = {Proceedings IEEE/CVF Conf.~on Computer Vision and Pattern Recognition ({CVPR})},
    month = jun,
    year = {2021},
    month_numeric = {6}
}
    

If you have any questions or problems regarding this dataset, please do not hesitate to contact us.


Acknowledgement

This website was built by the Software Workshop at Max-Planck Institute for Intelligent Systems