The following repository consists of code to process the data generated by Unreal into a format that could be used for training BEDLAM-CLIFF/BEDLAM-HMR. You could modify the code to support research for tracking, multi-person pose estimation etc. We provide code for processing both SMPL-X and SMPL based ground truth.
Step 1. Register on BEDLAM website.
Step 2. Download BEDLAM 30fps images and ground truth camera in bedlam_data/images from Synthetic Image Data (30fps) section of BEDLAM. Note that only image sequences and ground truth for sequences is needed i.e. -png, -gt
Step 3. Register on SMPL-X website if not done already.
Step 4. Run following script to download SMPL-X model files and BEDLAM animation files in neutral SMPL-X format.
bash download_utils.sh
Step 1. Register on BEDLAM website.
Step 2. Download BEDLAM 30fps images and ground truth camera in bedlam_data/images from Synthetic Image Data (30fps) section of BEDLAM. Note that only image sequences and ground truth for sequences is needed i.e. -png, -gt
Step 3. Register on SMPL website.
Step 4. Run following script to download SMPL model files and BEDLAM animation files in neutral SMPL format.
bash download_utils_smpl.sh
Once the data is downloaded you can run the following script to generate the output *.npz file for each scene. If you want to generate npz for particular scenee you can modify bedlam_scene_names.csv file.
python df_full_body.py --fps 6
For SMPL, run following script:
python df_full_body_smpl.py --fps 6
Note that you can either generate output in 6fps format where every 5th frame information is stored in the output or in 30fps format where every frame is stored. To get the identity of each subject, you can store the sub field.
Note that some of the images are rotated by 90 degree when generate by Unreal. All such images/scenes could be identified using closeup flag in bedlam_scene_names.csv. The processing code make sure that the ground truth is stored in correct format but this means that when loading the image, you need to rotate it by 90 degree so that the projection is correct.
imgname = Name of the images
center = center of the subject
scale = scale of the subject
pose_cam = SMPL-X pose parameter of the subject in camera coordinates
pose_world = SMPL-X pose parameter of the subject in world coordinates
shape = 11 SMPL-X shape componenets
trans_cam = translation of the body in camera coordinates
trans_world = translation of the body in world coordinates
gtkps = 2d joints
cam_int = camera intrinsic matrix
cam_ext = camera extrinsic matrix
gender = gender of the subject
proj_verts = 2d vertices subsampled
motion_info = AMASS motion info, BABEL action label
sub = identity