EasyMocap Doc - Output

En | 中文

Json Format
Export to .bvh

Json Format

The results are saved in json format.

<output_root>
├── keypoints3d
│   ├── 000000.json
│   └── xxxxxx.json
└── smpl
    ├── 000000.jpg
    ├── 000000.json
    └── 000004.json

The data in keypoints3d/000000.json is a list, each element represents a human body.

{
    'id': <id>, # the person ID
    'keypoints3d': [[x0, y0, z0, c0], [x1, y1, z0, c1], ..., [xn, yn, zn, cn]], # x,y,z is the 3D coordinates, c means the confidence of this joint. If the c=0, it means this joint is invisible.
}

The definition of the joints is as body25.

The data in smpl/000000.json is also a list, each element represents the SMPL parameters which is slightly different from official model.

{
    "id": <id>,
    "Rh": <(1, 3)>,
    "Th": <(1, 3)>,
    "poses": <(1, 72/78/87)>,
    "expression": <(1, 10)>,
    "shapes": <(1, 10)>
}

If you use SMPL+H model, the poses contains 22x3+6+6. We use 6 pca coefficients for each hand. 3(jaw, left eye, right eye)x3 poses of head are added for SMPL-X model.

Attention (for SMPL/SMPL-X users)

This parameter is a little different from original SMPL/SMPL-X parameters.

We set the first 3 dimensions of poses to zero, and add a new parameter Rh to represents the global oritentation, the vertices of SMPL model V = RX(theta, beta) + T. Please note that the paramter Rh is not equal to global_orient in SMPL-X model. We take this representation because that changing paramters to new coordinate system in origin is difficult(see this link).

In our representation, you can just use R'@(RX + T) + T' to convert the model, and the new global rotaion and translation is simply written as R'@R and R'@T + T'

To compute the joints locations from these parameters, please refer to ./code/vis_render.py. The key steps are:

# 0. load SMPL model
from smplmodel import load_model
body_model = load_model(args.gender, model_type=args.model)
# 1. load parameters
infos = dataset.read_smpl(nf*step)
# 2. compute joints
joints = body_model(return_verts=False, return_tensor=False, **info)[0]
# 3. compute vertices
vertices = body_model(return_verts=True, return_tensor=False, **info)[0]

Export to bvh format

To export the SMPL results to bvh file, you need to download the SMPL-maya model from the website of SMPL. Place the .fbx model in ./data/smplx/SMPL_maya, it may be like this:

└── smplx
    ├── smpl
    │   ├── SMPL_FEMALE.pkl
    │   ├── SMPL_MALE.pkl
    │   └── SMPL_NEUTRAL.pkl
    ├── SMPL_maya
    │   ├── basicModel_f_lbs_10_207_0_v1.0.2.fbx
    │   ├── basicModel_m_lbs_10_207_0_v1.0.2.fbx
    │   ├── joints_mat_v1.0.2.pkl
    │   ├── README.txt
    │   ├── release_notes_v1.0.2.txt
    │   └── SMPL_maya_plugin_v1.0.2.py
    └── smplx

The Blender is also needed. The <path_to_output_smpl> is usually ${out}/smpl, which contanis the 000000.json, ... of SMPL parameters.

BLENDER_PATH=<path_to_blender>/blender-2.79a-linux-glibc219-x86_64
${BLENDER_PATH}/blender -b -t 12 -P scripts/postprocess/convert2bvh.py -- <path_to_output_smpl> --o <output_path>

We have not implement the export of SMPL+H, SMPL-X model yet. If you are interested on it, feel free to create a pull request to us.

输出

Json格式

关键点重建的结果会输出到${out}/keypoints3d路径下

<out>
├── keypoints3d
│   ├── 000000.json
│   └── xxxxxx.json
└── skel

每个json里面是一个列表，包含了当前帧的所有人，列表里的每一个元素表示一个人，内容如下：

{
    'id': <id>, # 表示人的跟踪的id
    'keypoints3d': [[x0, y0, z0, c0], [x1, y1, z0, c1], ..., [xn, yn, zn, cn]]: # (N, 4)，表示人的关键点坐标，c表示置信度，置信度为0则该关节点不可见
}

关键点的定义使用OpenPose的BODY25格式

导出为bvh格式

这里使用Blender进行导出，测试的Blender版本为2.79。需要先下载SMPL的fbx模型

BLENDER_PATH=<path_to_blender>/blender-2.79a-linux-glibc219-x86_64
${BLENDER_PATH}/blender -b -t 12 -P scripts/postprocess/convert2bvh.py -- <path_to_output_smpl> --o <path_to_bvh>

4.9 KiB Raw Permalink Blame History Unescape Escape