# EasyMocap **EasyMocap** is an open-source toolbox for **markerless human motion capture**. ## Results |:heavy_check_mark: Skeleton|:heavy_check_mark: SMPL| |----|----| |![repro](doc/feng/repro_512.gif)|![repro](doc/feng/smpl_512.gif)|| |:black_square_button: Whole Body|:black_square_button: [Detailed Mesh](https://zju3dv.github.io/neuralbody/)| |----|----| |

| ## Installation ### 1. Download SMPL models To download the *SMPL* model go to [this](http://smpl.is.tue.mpg.de) (male and female models) and [this](http://smplify.is.tue.mpg.de) (gender neutral model) project website and register to get access to the downloads section. **Place them as following:** ```bash data └── smplx ├── J_regressor_body25.npy └── smpl ├── SMPL_FEMALE.pkl ├── SMPL_MALE.pkl └── SMPL_NEUTRAL.pkl ``` ## Quick Start We provide an example multiview dataset[OneDrive](). After downloading the dataset, you can run the following example scripts. ```bash data=path/to/data out=path/to/output # 0. extract the video to images python3 scripts/preprocess/extract_video.py ${data} # 1. example for skeleton reconstruction python3 code/demo_mv1pmf_skel.py ${data} --out ${out} --vis_det --vis_repro --undis --sub_vis 1 7 13 19 # 2. example for SMPL reconstruction python3 code/demo_mv1pmf_smpl.py ${data} --out ${out} --end 300 --vis_smpl --undis --sub_vis 1 7 13 19 ``` ## Not Quick Start ### 0. Prepare Your Own Dataset ```bash zju-ls-feng ├── extri.yml ├── intri.yml └── videos ├── 1.mp4 ├── 2.mp4 ├── ... ├── 8.mp4 └── 9.mp4 ``` The input videos are placed in `videos/`. Here `intri.yml` and `extri.yml` store the camera intrinsici and extrinsic parameters. For example, if the name of a video is `1.mp4`, then there must exist `K_1`, `dist_1` in `intri.yml`, and `R_1((3, 1), rotation vector of camera)`, `T_1(3, 1)` in `extri.yml`. The file format is following [OpenCV format](https://docs.opencv.org/master/dd/d74/tutorial_file_input_output_with_xml_yml.html). ### 1. Run [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) ```bash python3 scripts/preprocess/extract_video.py ${data} --openpose ``` ### 2. Run the code ### 3. Output The results are saved in `json` format. ```bash - ├── keypoints3d │ ├── 000000.json │ └── xxxxxx.json └── smpl ├── 000000.jpg ├── 000000.json └── 000004.json ``` The data in `keypoints3d/000000.json` is a list, each element represents a human body. ```bash { 'id': , 'keypoints3d': [[x0, y0, z0, c0], [x1, y1, z0, c1], ..., [xn, yn, zn, cn]] } ``` The data in `smpl/000000.json` is also a list, each element represents the SMPL parameters which is slightly different from official model. ```bash { "id": , "Rh": <(1, 3)>, "Th": <(1, 3)>, "poses": <(1, 72)>, "shapes": <(1, 10)> } ``` We set the first 3 dimensions of `poses` to zero, and add a new parameter `Rh` to represents the global oritentation, the vertices of SMPL model V = RX(theta, beta) + T. ## Acknowledgements Here are some great resources we benefit: - SMPL models and layer is borrowed from MPII [SMPL-X model](https://github.com/vchoutas/smplx). - Some functions are borrowed from [SPIN](https://github.com/nkolot/SPIN), [VIBE](https://github.com/mkocabas/VIBE), [SMPLify-X](https://github.com/vchoutas/smplify-x) We also would like to thank Wenduo Feng for the example data. ## Contact Please open an issue if you have any questions. ## Citation This project is the base of our other works: [iMocap](https://zju3dv.github.io/iMoCap/), [Neural Body](https://zju3dv.github.io/neuralbody/) ```bibtex @inproceedings{dong2020motion, title={Motion capture from internet videos}, author={Dong, Junting and Shuai, Qing and Zhang, Yuanqing and Liu, Xian and Zhou, Xiaowei and Bao, Hujun}, booktitle={European Conference on Computer Vision}, pages={210--227}, year={2020}, organization={Springer} } ```