Publication and Technical Report Details
Patch-based 3D Human Pose Refinement
State-of-the-art 3D human pose estimation approaches typically estimate pose from the entire RGB image in a single forward run. In this paper, we develop a post-processing step to refine 3D human pose estimation from body part patches. Using local patches as input has two advantages. First, the fine details around body parts are zoomed in to high resolution for preciser 3D pose prediction. Second, it enables the part appearance to be shared between poses to benefit rare poses. In order to acquire informative representation of patches, we explore different input modalities and validate the superiority of fusing predicted segmentation with RGB. We show that our method consistently boosts the accuracy of state-of-the-art 3D human pose methods.
State-of-the-art 3D human pose estimation approaches typically estimate pose from the entire RGB image in a single forward run. In this paper, we develop a post-processing step to refine 3D human pose estimation from body part patches. Using local patches as input has two advantages. First, the fine details around body parts are zoomed in to high resolution for preciser 3D pose prediction. Second, it enables the part appearance to be shared between poses to benefit rare poses. In order to acquire informative representation of patches, we explore different input modalities and validate the superiority of fusing predicted segmentation with RGB. We show that our method consistently boosts the accuracy of state-of-the-art 3D human pose methods.
DeepSkeleton: Skeleton Map for 3D Human Pose Regression
We present a novel intermediate feature representation named skeleton map for regression. It distills structural context from irrelavant properties of RGB image e.g. illumination and texture. It is simple, clean and can be easily generated via deconvolution network. For the first time, we show that training regression network from skeleton map alone is capable of meeting the performance of state-of-the-art 3D human pose estimation works. We further exploit the power of multiple 3D hypothesis generation to obtain reasonbale 3D pose in consistent with 2D pose detection. The effectiveness of our approach is validated on challenging in-the-wild dataset MPII and indoor dataset Human3.6M.
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
We strive to answer two questions: What is the current state of 3D hand pose estimation? And, what are the next challenges that need to be tackled? Following the successful Hands In the Million Challenge (HIM2017), we investigate 11 state-of-the-art methods on three tasks: single frame 3D pose estimation, 3D hand tracking, and hand pose estimation during object interaction. We analyze the performance of different CNN structures with regard to hand shape, joint visibility, view point and articulation distributions.
Model-based Deep Hand Pose Estimation
We propose a model based deep learning approach that adopts a forward kinematics based layer to ensure the geometric validity of estimated poses. For the first time, we show that embedding such a non-linear generative process in deep learning is feasible for hand pose estimation. Our approach is verified on challenging public datasets NYU and ICVL and achieves state-of-the-art performance.
--------------------------------------------------This is the separation line----------------------------------------------
Figure 2. The wall close to Norman Y. Mineta San Jose International Airport.
Figure 3. Human skeleton exhibition in UC Berkeley Art Museum and Pacific Film Archive (BAMPFA).
Fun facts
1. Did you know that after 1880, more and more of Monet’s paintings eliminate the human figure; later they even cut out such solid reminders of human existence as bridges, houses and boats. One wonders what part the early death of his wife Camille, his favorite model, played in this renunciation.
2. Gaugin complained to Pissarro in 1881 about the repetitive nature of the landscape Impressionists' subjects: "The way Durand-Ruel
is operating with Sisley and Monet who are churning out pictures at top speed, he will soon have four hundred paints which he will not be able to get rid of. Hmm.
OK. Fine. People in this decade especially churned out a massive amount of deep learning papers, most of which are colossal in network scale and grand in purported far-fetched "insights" n "arguments". FINE. I AM TOTALLY OK WITH THAT. I will get used to it.
Someday there might even be a grand opening of Museum of Modern Deep Learning and Contemporary Computer Vision Blend. Just kidding.
3.
2. Gaugin complained to Pissarro in 1881 about the repetitive nature of the landscape Impressionists' subjects: "The way Durand-Ruel
is operating with Sisley and Monet who are churning out pictures at top speed, he will soon have four hundred paints which he will not be able to get rid of. Hmm.
OK. Fine. People in this decade especially churned out a massive amount of deep learning papers, most of which are colossal in network scale and grand in purported far-fetched "insights" n "arguments". FINE. I AM TOTALLY OK WITH THAT. I will get used to it.
Someday there might even be a grand opening of Museum of Modern Deep Learning and Contemporary Computer Vision Blend. Just kidding.
3.
Mood Booster
The Cinematic Orchestra - A Caged Bird/Imitations of Life
Señorita
If I Killed Someone For You
LES INDES GALANTES
Señorita
If I Killed Someone For You
LES INDES GALANTES
Places in Shanghai
Pier 39 (Jing'an Kerry Centre Shopping Mall)
SHAKE SHACK (several locations)
Peet's (personally recommend the IAPM one)
Fascino (W Nanjing Rd)
81BAKERY (Daxue Rd, near Fudan Univ.)
%Arabica
T Plus
茶巭道
SHAKE SHACK (several locations)
Peet's (personally recommend the IAPM one)
Fascino (W Nanjing Rd)
81BAKERY (Daxue Rd, near Fudan Univ.)
%Arabica
T Plus
茶巭道
Places in New York
Riverbank State Park (Ice Skating)
Candlelight @ Corpus Christi Catholic Church
Bobst
Strand
Candlelight @ Corpus Christi Catholic Church
Bobst
Strand