KITRO: Refining Human Mesh by 2D Clues and Kinematic-tree Rotation
(CVPR 2024)

Fengyuan Yang, Kerui Gu, Angela Yao

[Paper], [Supp], [arXiv]

1. Requirements

• Python 3.6
• PyTorch 1.10.1

2. Datasets

• Download the preprocessed data from this link and put in './data' folder.
  • the data structure KITRO desires is as following:

    {
        'imgname'  # image name (list)
        'pred_theta'  # Predicted 3D rotation matrix (shape: [samples, 24, 3, 3])
        'pred_beta'    # Predicted body shape parameters (shape: [samples, 10])
        'pred_cam'      # Predicted camera translation (shape: [samples, 3])
        'intrinsics'  # Intrinsic camera parameters (shape: [samples, 3, 3])
        'keypoints_2d'  # Given 2D keypoints (shape: [samples, 24, 2])
        'GT_pose'        # Ground truth 3D rotation parameters (shape: [samples, 72])
        'GT_beta'        # Ground truth body shape parameters (shape: [samples, 10])
    }

3. Usage

• Test on 3DPW

  python eval_KITRO.py --data_path 'data/ProcessedData_CLIFFpred_w2DKP_3dpw.pt' >> logs/runkitro_3dpw.out 2>&1

  • Corresponding output logs can found at logs/runkitro_3dpw.out

• Test on Human3.6m

  python eval_KITRO.py --data_path 'data/ProcessedData_CLIFFpred_w2DKP_HM36.pt' >> logs/runkitro_HM36.out 2>&1

  • Corresponding output logs can found at logs/runkitro_HM36.out

Citation

If you find our paper or codes useful, please consider citing our paper:

@InProceedings{KITRO_2024,
    author    = {Yang, Fengyuan and Gu, Kerui and Yao, Angela},
    title     = {KITRO: Refining Human Mesh by 2D Clues and Kinematic-tree Rotation},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {1052-1061}
}

Acknowledgments

Our codes are based on SPIN, CLIFF, SMPLify, and HybrIK and we really appreciate it.