Here we provide a detailed evaluation of 3D Human pose and shape estimation methods on AGORA test images. Please check the github repository for more details on evaluation metric and protocol. Please login to upload your predictions on test images and get the evaluation results.
| SMPL-X Algorithms | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Algorithm | NMVE | NMJE | MVE | MPJPE | ||||||||
| FB | B | FB | B | FB | B | F | LH/RH | FB | B | F | LH/RH | |
| SMPLify-X [smplx1] | 333.1 | 263.3 | 326.5 | 256.5 | 236.5 | 187.0 | 48.9 | 48.3/51.4 | 231.8 | 182.1 | 52.9 | 46.5/49.6 |
| ExPose [smplx2] | 265.0 | 184.8 | 263.3 | 183.4 | 217.3 | 151.5 | 51.1 | 74.9/71.3 | 215.9 | 150.4 | 55.2 | 72.5/68.8 |
| Frankmocap [smplx3] | 207.8 | 204.0 | 168.3 | 54.7/55.7 | 165.2 | 52.3/53.1 | ||||||
| BEDLAM [smplx4] | 179.5 | 132.2 | 177.5 | 131.4 | 131.0 | 96.5 | 25.8 | 38.8/39.0 | 129.6 | 95.9 | 27.8 | 36.6/36.7 |
| BEDLAM-finetuned [smplx5] | 142.2 | 102.1 | 141.0 | 101.8 | 103.8 | 74.5 | 23.1 | 31.7/33.2 | 102.9 | 74.3 | 24.7 | 29.9/31.3 |
| PIXIE [smplx6] | 233.9 | 173.4 | 230.9 | 171.1 | 191.8 | 142.2 | 50.2 | 49.5/49.0 | 189.3 | 140.3 | 54.5 | 46.4/46.0 |
| Hand4Whole-finetuned [smplx7] | 144.1 | 96.0 | 141.1 | 92.7 | 135.5 | 90.2 | 41.6 | 46.3/48.1 | 132.6 | 87.1 | 46.1 | 44.3/46.2 |
| PyMAF-X [smplx8] | 141.2 | 94.4 | 140.0 | 93.5 | 125.7 | 84.0 | 35.0 | 44.6/45.6 | 124.6 | 83.2 | 37.9 | 42.5/43.7 |
| HybrIK-X [smplx9] | 120.5 | 73.7 | 115.7 | 72.3 | 112.1 | 68.5 | 37.0 | 46.7/47.0 | 107.6 | 67.2 | 38.5 | 41.2/41.4 |
| OSX [smplx10] | 130.6 | 85.3 | 127.6 | 83.3 | 122.8 | 80.2 | 36.2 | 45.4/46.1 | 119.9 | 78.3 | 37.9 | 43.0/43.9 |
| HumanWild [smplx11] | 125.4 | 81.9 | 123.8 | 81.5 | 105.3 | 68.8 | 31.2 | 38.5/39.7 | 104.0 | 68.5 | 33.1 | 36.1/37.3 |
| SMPLer-X [smplx12] | 107.2 | 68.3 | 104.1 | 66.3 | 99.7 | 63.5 | 29.9 | 39.1/39.5 | 96.8 | 61.7 | 31.4 | 36.7/37.2 |
| AiOS (0.5 score) [smplx13] | 97.8 | 61.3 | 96.0 | 60.7 | 91.9 | 57.6 | 24.6 | 38.7/39.6 | 90.2 | 57.1 | 25.7 | 36.4/37.3 |
| AiOS (0.3 score) [smplx14] | 103.0 | 63.5 | 100.8 | 62.6 | 98.9 | 61.0 | 27.7 | 42.5/43.4 | 96.8 | 60.1 | 29.2 | 40.1/40.9 |
| SMPLer-X (AiOS) [smplx15] | 102.4 | 63.8 | 99.5 | 62.1 | 98.3 | 61.2 | 30.3 | 40.4/40.7 | 95.5 | 59.6 | 31.7 | 37.9/38.2 |
| Multi-HMR [smplx16] | 104.0 | 68.1 | 103.6 | 68.8 | 93.6 | 61.3 | 25.1 | 35.7/36.1 | 93.2 | 61.9 | 26.0 | 33.7/34.2 |
| OSX (AiOS) [smplx17] | 126.4 | 81.8 | 123.4 | 80.0 | 121.3 | 78.5 | 36.1 | 45.9/46.3 | 118.5 | 76.8 | 37.6 | 43.5/44.0 |
| SMPL Algorithms | ||||
|---|---|---|---|---|
| Algorithm | NMVE | NMJE | MVE | MPJPE |
| EFT [smpl1] | 196.3 | 203.6 | 159.0 | 165.4 |
| HMR [smpl2] | 217.0 | 226.0 | 173.6 | 180.5 |
| CenterHMR [smpl3] | 233.9 | 242.3 | 161.4 | 168.1 |
| SPIN [smpl4] | 216.3 | 223.1 | 168.7 | 175.1 |
| SPIN-finetuned [smpl5] | 193.4 | 199.2 | 148.9 | 153.4 |
| PARE [smpl6] | 167.7 | 174.0 | 140.9 | 146.2 |
| SPEC [smpl7] | 126.8 | 133.7 | 106.5 | 112.3 |
| ROMP-finetuned [smpl8] | 130.8 | 134.0 | 113.8 | 116.6 |
| BEV-finetuned [smpl9] | 108.3 | 113.2 | 100.7 | 105.3 |
| PyMAF [smpl10] | 200.2 | 207.4 | 168.2 | 174.2 |
| Hand4Whole-finetuned (body only) [smpl11] | 90.2 | 95.5 | 84.8 | 89.8 |
| ROMP2_finetuned [smpl12] | 113.6 | 118.8 | 103.4 | 108.1 |
| CLIFF-finetuned [smpl13] | 83.5 | 89.0 | 76.0 | 81.0 |
| CLIFF(PRoM)-finetuned [smpl14] | 66.3 | 70.7 | 61.0 | 65.0 |
| PLIKS-finetuned [smpl15] | 71.6 | 76.1 | 67.3 | 71.5 |
| HybrIK-finetuned [smpl16] | 81.2 | 84.6 | 73.9 | 77.0 |
| NIKI-finetuned [smpl17] | 70.2 | 74.0 | 63.9 | 67.3 |
| ProPose-finetuned [smpl18] | 78.8 | 82.7 | 70.9 | 74.4 |
| BoPR [smpl19] | 148.2 | 154.7 | 128.9 | 134.6 |
| BoPR_finetuned [smpl20] | 84.7 | 90.8 | 74.5 | 79.9 |
| PyMAF-finetuned [smpl21] | 89.9 | 95.0 | 84.5 | 89.3 |
| W-HMR-finetuned [smpl22] | 70.4 | 75.4 | 63.4 | 67.9 |
| AiOS (body only) [smpl23] | 61.2 | 68.0 | 57.5 | 63.9 |
smplx1. Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3d hands, face, and body from a single image. IEEE Conference on Computer Vision and Pattern Recognition, 2019
smplx2. Vasileios Choutas, Georgios Pavlakos, Timo Bolkart, Dimitrios Tzionas, and Michael J. Black. Monocular expressive body regression through body-driven attention. European Conference on Computer Vision, 2020
smplx3. Yu Rong, Takaaki Shiratori, and Hanbyul Joo. Frankmocap: Fast monocular 3d hand and body motion capture by regression and integration. arXiv preprint, 2020
smplx4. Michael J. Black, Priyanka Patel, Joachim Tesch, Jinlong Yang. BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion. CVPR, 2023 link
smplx5. Michael J. Black, Priyanka Patel, Joachim Tesch, Jinlong Yang. BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion. CVPR, 2023 link
smplx6. Yao Feng, Vasileios Choutas, Timo Bolkart, Dimitrios Tzionas, Michael J. Black. Collaborative Regression of Expressive Bodies using Moderation. International Conference on 3D Vision, 2021 link
smplx7. Gyeongsik Moon, Hongsuk Choi, Kyoung Mu Lee. Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimation. CVPRW, 2022 link
smplx8. Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, Liang An, Zhenan Sun, Yebin Liu. PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images. IEEE TPAMI, 2023 link
smplx9. Jiefeng Li, Siyuan Bian, Chao Xu, Zhicun Chen, Lixin Yang, Cewu Lu. HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery. link
smplx10. Jing Lin, Ailing Zeng, Haoqian Wang, Lei Zhang, Yu Li. One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer. The IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023 link
smplx11. Anonymous submission
smplx12. Anonymous submission
smplx13. Qingping Sun, Yanjun Wang, Ailing Zeng, Wanqi Yin, Chen Wei, Wenjia Wang, Haiyi Mei, Chi Sing Leung, Ziwei Liu, Lei Yang, Zhongang Cai. AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation. CVPR, 2024 link
smplx14. Qingping Sun, Yanjun Wang, Ailing Zeng, Wanqi Yin, Chen Wei, Wenjia Wang, Haiyi Mei, Chi Sing Leung, Ziwei Liu, Lei Yang, Zhongang Cai. AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation. CVPR, 2024 link
smplx15. Qingping Sun, Yanjun Wang, Ailing Zeng, Wanqi Yin, Chen Wei, Wenjia Wang, Haiyi Mei, Chi Sing Leung, Ziwei Liu, Lei Yang, Zhongang Cai. AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation. CVPR, 2024 link
smplx16. Baradel*, Fabien; Armando, Matthieu; Galaaoui, Salma; Brégier, Romain; Weinzaepfel, Philippe; Rogez, Grégory; Lucas*, Thomas. Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot. arXiv, 2024 link
smplx17. Qingping Sun, Yanjun Wang, Ailing Zeng, Wanqi Yin, Chen Wei, Wenjia Wang, Haiyi Mei, Chi Sing Leung, Ziwei Liu, Lei Yang, Zhongang Cai. AiOS: All-in-One-Stage 3D Wholebody Mesh Recovery. AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation, 2024 link
smpl1. Hanbyul Joo, Natalia Neverova, and Andrea Vedaldi. Exemplar fine-tuning for 3d human pose fitting towards in-the-wild 3d human pose estimation. arXiv preprint, 2020
smpl2. Angjoo Kanazawa, Michael J. Black, David W Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. IEEE Conference on Computer Vision and Pattern Recognition, 2018
smpl3. Yu Sun, Qian Bao, Wu Liu, Yili Fu, and Tao Mei. CenterHMR: a bottom-up single-shot method for multi-person 3d mesh recovery from a single image. arXiv preprint, 2020
smpl4. Nikos Kolotouros, Georgios Pavlakos, Michael J. Black, and Kostas Daniilidis. Learning to reconstruct 3D human pose and shape via model-fitting in the loop. International Conference on Computer Vision, 2019
smpl5. Priyanka Patel, Chun-Hao Paul Huang, Joachim Tesch, David Hoffmann, Shashank Tripathi, and Michael J. Black. AGORA: Avatars in geography optimized for regression analysis. IEEE Conference on Computer Vision and Pattern Recognition, 2021
smpl6. Muhammed Kocabas, Chun-Hao P. Huang, Otmar Hilliges, and Michael J. Black. PARE: Part Attention Regressor for 3D Human Body Estimation. International Conference on Computer Vision (ICCV), 2021 link
smpl7. Muhammed Kocabas, Chun-Hao P. Huang, Joachim Tesch, Lea Müller, Otmar Hilliges, and Michael J. Black. SPEC: Seeing People in the Wild with an Estimated Camera. International Conference on Computer Vision (ICCV), 2021 link
smpl8. Yu Sun, Qian Bao, Wu Liu, Yili Fu, Michael J. Black, Tao Mei. Monocular, One-stage, Regression of Multiple 3D People. ICCV, 2021 link
smpl9. Yu, Sun; Qian, Bao; Wu, Liu; Yili, Fu; Tao, Mei; Michael J. Black. Putting People in their Place: Monocular Regression of 3D People in Depth. CVPR, 2022 link
smpl10. Hongwen Zhang, Yating Tian, Xinchi Zhou, Wanli Ouyang, Yebin Liu, Limin Wang, Zhenan Sun. PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop. ICCV, 2021 link
smpl11. Gyeongsik Moon, Hongsuk Choi, Kyoung Mu Lee. Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimation. CVPRW, 2022 link
smpl12. Sun, Yu; Bao, Qian; Liu, Wu; Fu, Yili; Mei, Tao; Michael J. Black. Modified version of ROMP, presented in BEV paper. link
smpl13. Zhihao Li, Jianzhuang Liu, Zhensong Zhang, Songcen Xu, and Youliang Yan. CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation. ECCV, 2022 link
smpl14. Kerui Gu, Zhihao Li, Shiyong Liu, Jianzhuang Liu, Songcen Xu, Youliang Yan, Michael Bi Mi, Kenji Kawaguchi, Angela Yao. Learning Unorthogonalized Matrices for Rotation Estimation. link
smpl15. Anonymous. PLIKS: A Pseudo-Linear Inverse Kinematic Solver for 3D Human Body Estimation.
smpl16. Jiefeng Li, Chao Xu, Zhicun Chen, Siyuan Bian, Lixin Yang, Cewu Lu. HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation.
smpl17. Jiefeng Li, Siyuan Bian, Qi Liu, Jiasheng Tang, Fan Wang, Cewu Lu. NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation. CVPR, 2023
smpl18. Qi Fang, Kang Chen, Yinghui Fan, Qing Shuai, Jiefeng Li, Weidong Zhang. Learning Analytical Posterior Probability for Human Mesh Recovery. CVPR, 2023
smpl19. Anonymous submission
smpl20. Anonymous submission
smpl21. Hongwen Zhang , Yating Tian , Xinchi Zhou , Wanli Ouyang , Yebin Liu , Limin Wang , Zhenan Sun. PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop. ICCV, 2021 link
smpl22. Wei Yao, Hongwen Zhang, Yunlian Sun, Jinhui Tang. W-HMR: Human Mesh Recovery in World Space with Weak-supervised Camera Calibration and Orientation Correction. link
smpl23. Qingping Sun, Yanjun Wang, Ailing Zeng, Wanqi Yin, Chen Wei, Wenjia Wang, Haiyi Mei, Chi Sing Leung, Ziwei Liu, Lei Yang, Zhongang Cai. AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation. CVPR, 2024 link
@inproceedings{Patel:CVPR:2021,
title = {{AGORA}: Avatars in Geography Optimized for Regression Analysis},
author = {Patel, Priyanka and Huang, Chun-Hao P. and Tesch, Joachim and Hoffmann, David T. and Tripathi, Shashank and Black, Michael J.},
booktitle = {Proceedings IEEE/CVF Conf.~on Computer Vision and Pattern Recognition ({CVPR})},
month = jun,
year = {2021},
month_numeric = {6}
}
If you have any questions or problems regarding this dataset, please do not hesitate to contact us.
This website was built by the Software Workshop at Max-Planck Institute for Intelligent Systems