Study Plan

Research_Material - Paper_List | Prog - Programming | QF-Quantitative Finance

2021-5

• Mesh deformation (cage based)

• Mesh control (bone based)

  • Skeleton-Aware Networks for Deep Motion Retargeting (SIGGRAPH'20) (code) (project)
  • Learning Skeletal Articulations with Neural Blend Shapes (SIGGRAPH'21) (code)
  • HeterSkinNet: A Heterogeneous Network for Skin Weights Prediction (CGIT)

• Physical control

2020-11

• Basic

  • Motion builder
  • Deepmind-research (code)

• 3D human

  • Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation (Arxiv) (shows generalization of 3D pose estimation)
  • View-Invariant Probabilistic Embedding for Human Pose (eccv'20) (code) (pose invariant)
  • Multi-Scale Networks for 3D Human Pose Estimation with Inference Stage Optimization (Arxiv) (shows generalization)
  • CAPE: Clothed Auto-Person Encoding (CVPR'20) (code) (interesting, just invariant of SMPL)
  • XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera (SIGGRAPH'20)

• Retargeting

  • Skeleton-Aware Networks for Deep Motion Retargeting (SIGGRAPH'20) (code) (project)
  • Unpaired Motion Style Transfer from Video to Animation (SIGGRAPH'20) (code) (Application for skeleton-aware task)

• Control

  • Character Controllers using Motion VAEs (SIGGRAPH'20) (prject) (exciting direction, worth reading)
  • Accurate Face Rig Approximation with Deep Differential Subspace Reconstruction (SIGGRAPH'20) (related work may be useful)
  • A scalable Approach to Control Diverse Behaviors for Physically Simulated Characters (SIGGRAPH'20) (code) (obtain scalable and diverse character)

• Human Character

  • Catch & Carry: Reusable Neural Controllers for Vision-Guided Whole-Body Tasks (SIGGRAPH'20) (deepmind; need testing for platform)
  • Example-driven Virtual Cinematography by Learning Camera Behaviors (SIGGRAPH'20) (code) (exciting and promising; very good work to follow)
  • Robust Motion In-betweening (SIGGRAPH'20) (character between frames)

• Robot

  • Fast and Flexible Multilegged Locomotion Using Learned Centroidal Dynamics (SIGGRAPH'20) (code) (How to establish platform for extend research)
  • Learned Motion Matching (SIGGRAPH'20) (interactive robot and ground)
  • Model Predictive Control with a Visuomotor System for Physics-based Character Animation (SIGGRAPH'20) (project)

• Virtual Reality

  • Holographic Optics for Thin and Lightweight Virtual Reality (SIGGRAPH'20)

• Render

  • Enlighten Me: Importance of Brightness and Shadow for Character Emotion and Appeal (SIGGRAPH'20) (more details)

• Flow

  • Constraint Bubbles and Affine Regions: Reduced Fluid Models for Efficient Immersed Bubbles and Flexible Spatial Coarsening (SIGGRAPH'20) (code)
  • Fast and Scalable Turbulent Flow Simulation with Two-Way Coupling (SIGGRAPH'20) (project)
  • Lagrangian Neural Style Transfer for Fluids (SIGGRAPH'20) (image to 3D fluid transfer)
  • Wave Curves: Simulating Lagrangian water waves on dynamically deforming surfaces (SIGGRAPH'20) (enhance detail of a water surface simulation)

• Non-rigid

  • Homogenized Yarn-Level Cloth (SIGGRAPH'20) (project) (platform for clothing + skeleton)
  • Incremental Potential Contact: Intersection- and Inversion-free, Large-Deformation Dynamics (SIGGRAPH'20) (project) (code) (worth doing this + make person or object soft)
  • Interface Quadrature Material Point Method for Non-sticky Strongly Two-Way Coupled Nonlinear Solids and Fluids (SIGGRAPH'20) (project) (traditional)
  • Phong Deformation: A Better CO Interpolant for Embedded Deformation (SIGGRAPH'20) (make mesh soft + math)
  • Projective Dynamics with Dry Frictional Contact (project) (hair + cloth based on one author)
  • Robust Eulerian-On-Lagrangian Rods (project) (cloth)

2020-10

• 3D human
  • Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose (ECCV'20) (code)

2020-7

• 3D human
  • DeepCap: Monocular Human Performance Capture Using Weak Supervision (cvpr'20)(project)
  • TexMesh: Reconstructing Human Texture and Geometry from Monocular Video([eccv'20])(project)
• Inplicit field
  • Learning Implicit Fields for Generative Shape Modeling (cvpr'19)
  • BAE-NET: Branched Autoencoder for Shape Co-Segmentation (iccv'19)
  • BSP-Net: Generating Compact Meshes via Binary Space Partitioning (cvpr'20)

2020-6

I start one new task about fluid construction and robot control in Huawei. My research have little progress, i need focus.

Reading

• 3D control
  • TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting (cvpr'20)(code)
  • Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild (cvpr'20)
• 3D face
  • Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild (cvpr'20)(code)
  • Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images (cvpr'20)
  • RetinaFace: Single-shot Multi-level Face Localisation in the Wild (cvpr'20)

2020-5

I submitted one paper for NIPS.

Reading

• 3D Matching and Control
  • Human Motion Mapping to a Robot arm with Redundancy Resolution (paper'14)
  • Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets (3DV'19)(code)
  • RigNet: Neural Rigging for Articulated Characters (SIGGRAPH'20)(code)

2020-4

Reading

• 3D Mesh Reconstruction
  • C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion (arXiv)(code)
• Dense Pose
  • DensePose: Dense Human Pose Estimation In The Wild (CVPR'18)(code)
  • Canonical Surface Mapping via Geometric Cycle Consistency (ICCV'19)(code)
  • SCOPS: Self-Supervised Co-Part Segmentation (CVPR'19)
  • Slim DensePose: Thrifty Learning from Sparse Annotations and Motion Cues (CVPR'19)
• 3D Pose Estimation
  • Predicting Camera Viewpoint Improves Cross-dataset Generalization for 3D Human Pose Estimation (arXiv)(code)

2020-3

Work hard for NIPS and TPAMI. Preparing supplementary material for ECCV.

Study

• Basic
  • Human Pose Paper (paper)
  • Kaolin (intro)(project)

Reading

• 3D Mesh Reconstruction from image
  • HoloPose: Holistic 3D Human Reconstruction In-The-Wild (CVPR'19)(proj)
  • Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning (CVPR'19)(code)
• 3D Mesh Reconstruction from video
  • Expressive Body Capture: 3D Hands, Face, and Body from a Single Image (CVPR'19)(proj)
  • Exploiting temporal context for 3D human pose estimation in the wild (CVPR'19)(code)
  • VIBE: Video Inference for Human Body Pose and Shape Estimation (arXiv)(code)
  • HEMlets PoSh: Learning Part-Centric Heatmap Triplets for 3D Human Pose and Shape Estimation (ICCV'19)
• Unsupervised 3D Mesh Reconstruction
  • PoseNet3D: Unsupervised 3D Human Shape and Pose Estimation (arXiv)
  • Self-supervised Learning of Motion Capture(NIPS'17)(code)
  • TexturePose: Supervising Human Mesh Estimation with Texture Consistency (ICCV'19) (code)
  • Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop (ICCV'19) (code)
• 3D reconstrunction
  • On the Continuity of Rotation Representations in Neural Networks (CVPR'19)
  • Learning to Estimate 3D Human Pose and Shape from a Single Color Image (CVPR'18)
  • Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer (NIPS'19)(code)
  • Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning (CVPR'19)(code)
  • VIBE: Video Inference for Human Body Pose and Shape Estimation (arXiv)(code)
  • Neural 3D Mesh Renderer (CVPR'18)(code)
  • learning 3d human dynamics from video (CVPR'19)(code)
  • Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image (ECCV'16) sm
  • MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak Shape Supervision (CVPR'20)
  • Meta3D: Single-View 3D Object Reconstruction from Shape Priors in Memory (ECCV'20)
• 3D Detection
  • Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud (CVPR'20)(code)
• 3D Pose Estimation
  • Metric-Scale Truncation-Robust Heatmaps for 3D Human Pose Estimation (FG)
  • Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPS (CVPR'20)

2020-2

A very busy month for submitting ECCV and CVPR rebuttal.

Luckily, our paper "Deep Kinematics Analysis for Monocular 3D Pose Estimation" has been accepted by CVPR.

2020-1

Super busy but less productive. Chinese new year is comming, I wish all Chinese could be safe under the threat of 2019-nCoV, hope everything works well.

Reading

• Graph

  • Multi-Stage Self-Supervised Learning for Graph Convolutional Networks (arXiv)

• 3D Mesh Reconstruction

  • Self-supervised Learning of Motion Capture(NIPS'17)(code)
  • VIBE: Video Inference for Human Body Pose and Shape Estimation (arXiv)(code)

• 2D Pose Estimation

  • Pose Neural Fabrics Search (arXiv)(code)

• Loss

  • Bayesian Loss for Crowd Count Estimation with Point Supervision (iccv19)(code)

• Unsupervised

  • Object landmark discovery through unsupervised adaptation (NIPS'19)(code)

• 3D Pose Estimation

  • On Boosting Single-Frame 3D Human Pose Estimation via Monocular Videos (iccv19)
  • XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera (arXiv)
  • RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimation(CVPR'19)(code)
  • Bottom-up Higher-Resolution Networks for Multi-Person Pose Estimation (arXiv)(code)
  • The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation (arXiv)(zhihu)
  • MaskedFusion: Mask-based 6D Object Pose Detection (arXiv)(code)

2019-12

Fortunately, i am invited to the award ceremony of IJSAI 2019.

A very busy month for preparing papers and final exams.

Study

• Basic
  • PyQt5 (Tutorial)
  • Depthwise Separable Convolution (Youtobe)
  • MobileNetV1(paper)
• GNN
  • CS224W: Machine Learning with Graphs (project)
  • Deep Learning on Graphs: a roadmap (github)
• pytorch acceleration
  • dali (install guide)(code)
  • apex (offical guide)(教程)

Reading

• Detection
  • EfficientDet: Scalable and Efficient Object Detection (arXiv)(zhihu)
  • Detectron2 (project)
  • ThunderNet: Towards Real-time Generic Object Detection (ICCV19)
  • ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design(arXiv)
  • Light-Head R-CNN: In Defense of Two-Stage Object Detector(arXiv)
• NAS
  • RC-DARTS: Resource Constrained Differentiable Architecture Search (arXiv)
  • Understanding and Robustifying Differentiable Architecture Search (ICLR'20 oral)(code)(review)
  • EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (arXiv)(code)
  • Blockwisely Supervised Neural Architecture Search with Knowledge Distillation (arXiv)
  • Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search (arXiv)(code)
• Pruning
  • Global Sparse Momentum SGD for Pruning Very Deep Neural Networks (NIPS'19)(code)
• Render
  • Few-shot Video-to-Video Synthesis (NIPS'19)(code)
  • Fashion++: Minimal Edits for Outfit Improvement (ICCV19)(code) : Borrow from BicycGAN and pix2pixHD
  • DeepFovea: Neural Reconstruction for Foveated Rendering and Video Compression using Learned Statistics of Natural Videos (Facebook Reality Labs)
  • Animating Landscape: Self-Supervised Learning of Decoupled Motion and Appearance for Single-Image Video Synthesis (TOG'19)(project)(code)
• RL
  • Neural Painters: A learned differentiable constraint for generating brushstroke paintings (arXiv)(code)
• Tracking
  • You Only Look Once: Unified, Real-Time Object Detection (arXiv)

2019-11

Main focus: preparing for ICML and ECCV.

Prepare CVPR submissions and supplementary materials.

Study

• Graph Convolutional Network (Graph本质解析)

Reading

• Unsupervised 3D Pose Estimation
  • Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation (CVPR'19)
  • Unsupervised Keypoint Learning for Guiding Class-Conditional Video Prediction (NIPS'19)
  • Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning (NIPS'18)
  • Domes to Drones: Self-Supervised Active Triangulation for 3D Human Pose Reconstruction(NIPS'19)
  • Unsupervised 3D Pose Estimation with Geometric Self-Supervision
• Graph
  • Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation(IJCAI'18)
  • Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition(AAAI'18)
  • Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition (CVPR'19)
• NAS
  • ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware (ICLR19)(code)
  • FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search (facebook)(code)
  • Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching (arXiv)

2019-10

Work hard for CVPR2020 and PRCV2019 Challenge workshop (Rank 8th).

2019-9

Start for PHD in Vision and Learning Lab supervised by Bingbing Ni and Wenjun Zhang.

Before 2019

• ILSVRC 2015: Classification+localization with additional training data (Rank 1st).
• ILSVRC 2016: Object detection/tracking from video with additional training data (Rank 1st).
• ILSVRC 2016: Object detection from video with provided/additional training data (Rank 1st).
• ILSVRC 2017: Object detection with provided/additional training data (Rank 1st).
• DAVIS Challenge 2016（just in experiments）: Unsupervised Video Segmentation (When i was intern in MSRA supervised by Yan Lv and Xiulian Peng in 2018) (Rank 1st)