114 lines
4.3 KiB
Markdown
114 lines
4.3 KiB
Markdown
|
<!--
|
|||
|
|
* @Date: 2021-03-07 14:41:22
|
|||
|
|
* @Author: Qing Shuai
|
|||
|
|
* @LastEditors: Qing Shuai
|
|||
|
|
* @LastEditTime: 2021-03-13 21:42:11
|
|||
|
|
* @FilePath: /EasyMocap/doc/02_output.md
|
|||
|
|
-->
|
|||
|
|
# EasyMocap Doc - Output
|
|||
|
|
[En](Output) | [中文](#输出)
|
|||
|
|
|
|||
|
|
## Contents
|
|||
|
|
1. [Json Format](#json-format)
|
|||
|
|
2. [Export to .bvh](#export-to-bvh-format)
|
|||
|
|
|
|||
|
|
## Json Format
|
|||
|
|
The results are saved in `json` format.
|
|||
|
|
```bash
|
|||
|
|
<output_root>
|
|||
|
|
├── keypoints3d
|
|||
|
|
│ ├── 000000.json
|
|||
|
|
│ └── xxxxxx.json
|
|||
|
|
└── smpl
|
|||
|
|
├── 000000.jpg
|
|||
|
|
├── 000000.json
|
|||
|
|
└── 000004.json
|
|||
|
|
```
|
|||
|
|
The data in `keypoints3d/000000.json` is a list, each element represents a human body.
|
|||
|
|
```bash
|
|||
|
|
{
|
|||
|
|
'id': <id>, # the person ID
|
|||
|
|
'keypoints3d': [[x0, y0, z0, c0], [x1, y1, z0, c1], ..., [xn, yn, zn, cn]], # x,y,z is the 3D coordinates, c means the confidence of this joint. If the c=0, it means this joint is invisible.
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
The definition of the joints is as [body25](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/02_output.md#pose-output-format-body_25).
|
|||
|
|
|
|||
|
|
The data in `smpl/000000.json` is also a list, each element represents the SMPL parameters which is slightly different from official model.
|
|||
|
|
```bash
|
|||
|
|
{
|
|||
|
|
"id": <id>,
|
|||
|
|
"Rh": <(1, 3)>,
|
|||
|
|
"Th": <(1, 3)>,
|
|||
|
|
"poses": <(1, 72/78/87)>,
|
|||
|
|
"expression": <(1, 10)>,
|
|||
|
|
"shapes": <(1, 10)>
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
We set the first 3 dimensions of `poses` to zero, and add a new parameter `Rh` to represents the global oritentation, the vertices of SMPL model V = RX(theta, beta) + T.
|
|||
|
|
|
|||
|
|
If you use SMPL+H model, the poses contains `22x3+6+6`. We use `6` pca coefficients for each hand. `3(jaw, left eye, right eye)x3` poses of head are added for SMPL-X model.
|
|||
|
|
|
|||
|
|
To compute the joints locations from these parameters, please refer to `./code/vis_render.py`. The key steps are:
|
|||
|
|
```python
|
|||
|
|
# 0. load SMPL model
|
|||
|
|
from smplmodel import load_model
|
|||
|
|
body_model = load_model(args.gender, model_type=args.model)
|
|||
|
|
# 1. load parameters
|
|||
|
|
infos = dataset.read_smpl(nf*step)
|
|||
|
|
# 2. compute joints
|
|||
|
|
joints = body_model(return_verts=False, return_tensor=False, **info)[0]
|
|||
|
|
# 3. compute vertices
|
|||
|
|
vertices = body_model(return_verts=True, return_tensor=False, **info)[0]
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Export to bvh format
|
|||
|
|
To export the SMPL results to bvh file, you need to download the SMPL-maya model from the website of SMPL. Place the `.fbx` model in `./data/smplx/SMPL_maya`, it may be like this:
|
|||
|
|
```bash
|
|||
|
|
└── smplx
|
|||
|
|
├── smpl
|
|||
|
|
│ ├── SMPL_FEMALE.pkl
|
|||
|
|
│ ├── SMPL_MALE.pkl
|
|||
|
|
│ └── SMPL_NEUTRAL.pkl
|
|||
|
|
├── SMPL_maya
|
|||
|
|
│ ├── basicModel_f_lbs_10_207_0_v1.0.2.fbx
|
|||
|
|
│ ├── basicModel_m_lbs_10_207_0_v1.0.2.fbx
|
|||
|
|
│ ├── joints_mat_v1.0.2.pkl
|
|||
|
|
│ ├── README.txt
|
|||
|
|
│ ├── release_notes_v1.0.2.txt
|
|||
|
|
│ └── SMPL_maya_plugin_v1.0.2.py
|
|||
|
|
└── smplx
|
|||
|
|
```
|
|||
|
|
The Blender is also needed. The `<path_to_output_smpl>` is usually `${out}/smpl`, which contanis the `000000.json, ...` of SMPL parameters.
|
|||
|
|
```bash
|
|||
|
|
BLENDER_PATH=<path_to_blender>/blender-2.79a-linux-glibc219-x86_64
|
|||
|
|
${BLENDER_PATH}/blender -b -t 12 -P scripts/postprocess/convert2bvh.py -- <path_to_output_smpl> --o <output_path>
|
|||
|
|
```
|
|||
|
|
We have not implement the export of SMPL+H, SMPL-X model yet. If you are interested on it, feel free to create a pull request to us.
|
|||
|
|
|
|||
|
|
-----
|
|||
|
|
|
|||
|
|
# 输出
|
|||
|
|
## Json格式
|
|||
|
|
关键点重建的结果会输出到`${out}/keypoints3d`路径下
|
|||
|
|
```bash
|
|||
|
|
<out>
|
|||
|
|
├── keypoints3d
|
|||
|
|
│ ├── 000000.json
|
|||
|
|
│ └── xxxxxx.json
|
|||
|
|
└── skel
|
|||
|
|
```
|
|||
|
|
每个json里面是一个列表，包含了当前帧的所有人，列表里的每一个元素表示一个人，内容如下：
|
|||
|
|
```json
|
|||
|
|
{
|
|||
|
|
'id': <id>, # 表示人的跟踪的id
|
|||
|
|
'keypoints3d': [[x0, y0, z0, c0], [x1, y1, z0, c1], ..., [xn, yn, zn, cn]]: # (N, 4)，表示人的关键点坐标，c表示置信度，置信度为0则该关节点不可见
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
关键点的定义使用OpenPose的[BODY25格式](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/02_output.md#pose-output-format-body_25)
|
|||
|
|
|
|||
|
|
## 导出为bvh格式
|
|||
|
|
这里使用Blender进行导出，测试的Blender版本为2.79。需要先下载SMPL的fbx模型
|
|||
|
|
```bash
|
|||
|
|
BLENDER_PATH=<path_to_blender>/blender-2.79a-linux-glibc219-x86_64
|
|||
|
|
${BLENDER_PATH}/blender -b -t 12 -P scripts/postprocess/convert2bvh.py -- <path_to_output_smpl> --o <path_to_bvh>
|
|||
|
|
```
|