Download PDF

Summary

Qianru Sun is currently a second-year post-doctor in the department of Computer Vision and Multimodal Computing, Max-Planck Institute for Informatics, Saarbruecken, Germany. Collaborators are Prof. Dr. Bernt Schiele and Dr. Mario Fritz. She holds a PhD degree from the School of Electronics Engineering and Computer Science, Peking University since Jan. 2016. Her research interests include computer vision and pattern recognition. Specific experiences have been made in human action recognitionanomaly event detection in videossocial relation recognition in photos and person image generation for both low-resolution re-identification images and high-resolution fashion photos.

Experiences

Mar 2016now

Postdoctoral researcher

Max-Planck Institute for Informatics

Department: Computer Vision and Multimodal Computing

Collaborators: Prof. Dr. Bernt Schiele and Dr. Mario Fritz

Other collaborators in KU Leuven: Liqian Ma, Xu jia, Prof. Tinne Tuytelaars and Prof. Luc Van Gool

Interests: Social relation recognition, Conditional image generation

Sep 2010Jan 2016

PhD

Peking University

Supervisor: Prof. Hong Liu

Open Lab on Human Robot Interaction (HRI Lab) 

http://robotics.szpku.edu.cn/c/

School of Electronics Engineering and Computer Science

Major: Electronic Information Science and Technology (Rank: top 10%)   

Interests: Human action recognition and anomaly detection in videos

Oct 2014Jan 2015

PhD exchange student

The University of Tokyo

Machine Intelligence Lab (Harada Ushiku Lab)

http://www.mi.t.u-tokyo.ac.jp/

Information Science and Technology

Project: Anomalous activity detection in surveillance videos.

Sep 2006Jul 2010

Bachelor of Science

Nanjing University of Posts and Telecommunications

Major: Information and Computing Science   (Rank: top 1)   

Main courses: mathematics courses and basic computer science courses, e.g., probability theory, mathematical analysis, numerical approximation, advanced algebra, and digital signal processing

Project Involved

Mar 2016Feb 2019

Privacy Implications of Visual Data Dissemination

Methods and Tools for Understanding and Controlling Privacy in Social Photosets

Understand what information can be extracted from public data, how can persons are linked, how does this compare to traditional social networks, how can we model and evaluate different possible actions a user can take. My work includes the analysis of egocentric videos people upload to public networks and making efficient models to understanding the aims, hobbies, and possible intentions contained in videos.

May 2013Dec 2016

Embedded Bionic Vision System for Human-Machine Interaction in Complicated Scenarios

with Ministry of Science and Technology

Implementing bionic vision in mobile robots to achieve target detection and different recognition tasks, e.g., person re-identification, and spoken/action command recognition. My work includes the design and implement of algorithms for daily activity recognition and  interactive command reception.

Sep 2012Aug 2014

Intelligent Surveillance System

for Public Service Technology Platform

Building a demonstration system implementing intelligent surveillance and salient event summary storage. My work includes the design and implement of algorithms for human behavior recognition in surveillance environments and storing event abstracts correspondingly.

Apr 2011Mar 2013

Abnormal Event Detection and Object Tracking for Monitoring System

with Shenzhen Science and Technology Innovation Committee

Developing algorithms for abnormal behavior detection and real-time alarm to build an efficient monitoring system through network cameras.  My work includes the design and implement of algorithms for human activity modeling and anomalous event detection in surveillance environments.

Research Experience

Pose guided person image generation 

  • Proposing the Pose Guided Person Generation Network (PG^2) that allows to synthesize person images in arbitrary poses, based on an image of that person and a novel pose. This framework utilizes the pose information explicitly and consists of two key stages: pose integration and image refinement. In the first stage the condition image and the target pose are fed into a U-Net-like network to generate an initial but coarse image of the person with the target pose. The second stage then refines the initial and blurry result by training a U-Net-like generator in an adversarial way. (NIPS 2017)

Domain-based social relation recognition in daily life photos

  • Investigating the social psychology articles and proposing a domain-based approach to implementing the recognition of social relations in social life photos. Proposing a corresponding dataset with relation annotations and agreement analysis. (CVPR 2017)

Local spatial-temporal feature based human action modeling and classification

  • Modeling co-occurrence and directional co-occurrence relationships between local feature pairs to enhance the representation capability of bag-of-words model. 
  • Studying local feature encoding method to extract informative descriptors meanwhile controlling the descriptor dimension. Specifically, we make use of clustering residual vectors to construct informative a hierarchical bag-of-words model for human action description.

Action sequenciality based ongoing human activity recognition 

  • Modeling long-range human activity by Recurrent Self-organizing Map (RSOM), in which currently observed features and their temporal information can be synchronously encoded. Traditional RSOM is greatly improved to have more strength for modeling changing time sequences. The RSOM trajectory is generated for activity representation. Since the special structure of trajectory, a combination of Dynamic Time Warping (DTW) distance and Edit distance, named DTW-E, is designed to measure its structural dissimilarity, then to infer the activity immediately.

Online anomalous event detection in surveillance scenes

  • Developing an unsupervised model called online Growing Neural Gas (online GNG) to learn surveillance scenes with changing crowd density and behavior types. Neighbor-related strategies are proposed to adjust dominant learning operations and tune main parameters in a totally adaptive manner.

Mentoring Experience

Liqian Ma, PhD student in KU Leuven                                                                                      2016 - now

Being the leader of sub Visual Group in HRI lab                                                                     2013 - 2015

  • Members: Qianru Sun, Zilin Liang, Yuke Sun, Qiaoduo Zhang, Lu Tian and Mengyuan Liu

Assisting Prof. Hong Liu to mentor Master Thesis                                                                 2014 - 2015       

  • Yuke Sun. Thesis: Person Re-identification System Based on Online Metric Learning. 2015.
  • Qiaoduo Zhang. Thesis: Human Action Classification Based on Sequential Bag-of-Words Model.  2015.
  • Zilin Liang. Thesis: Human Object Tracking Based on Online Machine Learning. 2014.

Digital Image Processing -- Clustering Analysis,  graduate course (six class hours)        2014 - 2015

Publication

  1.  Liqian Ma, Qianru Sun*, Xu Jia*, Bernt Schiele, Tinne Tuytelaars, Luc Van Gool. Pose Guided Person Image Generation. NIPS 2017. (In press. Acceptance rate 20.9%) [Paper]
  2. Qianru Sun, Bernt Schiele and Mario Fritz. A Domain Based Approach to Social Relation Recognition. CVPR 2017, Pages 3481-3490. (Acceptance rate 29.9%) [Paper]
  3. Qianru Sun, Hong Liu and Tatsuya Harada. Online Growing Neural Gas for Anomaly Detection in Changing Surveillance Scenes. Pattern Recognition. April 2017, Volume 64, Pages 187-201. [Paper]
  4. Qianru Sun, Hong Liu, Liqian Ma and Tianwei Zhang. A Novel Hierarchical Bag-of-Words Model for Compact Action Representation. NeuroComputing, Volume 174 Part B, 22 January 2016, Pages 722–732. [Paper]
  5. Qianru Sun, Hong Liu, Mengyuan Liu and Tianwei Zhang. Activity Prediction by Mapping Key Grouplets to Recurrent Self-Organizing Map. NeuroComputing, Volume 177, 12 February 2016, Pages 427–440. [Paper]
  6. Mengyuan Liu, Hong Liu, Qianru Sun, Tianwei Zhang and Runwei Ding. Salient Pairwise Spatio-temporal Interest Points for Real-time Activity Recognition. CAAI Transactions on Intelligence Technology, Volume 1, Issue 1, January 2016, Pages 14–29. [Paper]
  7. Liqian Ma, Hong Liu, Liang Hu, Can Wang, and Qianru Sun. Orientation Driven Bag of Appearances for Person Re-identification. arXiv preprint arXiv:1605.02464. [Paper]
  8. Hong Liu, Mengyuan Liu and Qianru Sun. Learning Directional Co-occurrence for Human Action Classification. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2014), May 4-9, 2014, Florence, Italy. [Paper(PDF)]
  9. Mengyuan Liu, Hong Liu and Qianru Sun. Action Classification by Exploring Directional Co-occurrence of Weighted STIPS. IEEE International Conference on Image Processing (ICIP 2014), Paris, France. [Paper(PDF)]
  10. Yuke Sun, Hong Liu and Qianru Sun. Online Learning on Incremental Distance Metric for Person Re-identification. International Conference on Robotics and Biomimetics (ROBIO 2014), December 5-10, 2014, Bali, Indonesia. [Paper(PDF)]
  11. Hong Liu, Qiaoduo Zhang and Qianru Sun. Human Action Classification Based on Sequential Bag-of-Words Model. International Conference on Robotics and Biomimetics (ROBIO 2014), December 5-10, 2014, Bali, Indonesia. [Paper(PDF)]
  12. Hong Liu, Zilin Liang and Qianru Sun. Robust Tracking with Discriminative Ranking Middle-level Patches. International Journal of Advanced Robotic Systems, 2014, 11:68. doi: 10.5772/ 58430. [Paper(PDF)]
  13. Qianru Sun, Wenmin Wang and Hong Liu. A study of human actions representation in video sequences. CAAI Transactions on Intelligent Systems, 8(3): 189-198, June 2013.
  14. Qianru Sun, Hong Liu. Inferring Ongoing Human Activities Based on Recurrent Self-Organizing Map Trajectory. The 24th British Machine Vision Conference (BMVC 2013), Pages 11.1-11.10, September 9-13, 2013, Bristol, U.K.. [Paper(PDF)]
  15. Qianru Sun, Hong Liu. Learning Spatio-Temporal Co-occurrence Correlograms for Efficient Human Action classification. IEEE International Conference on Image Processing (ICIP 2013), Pages 3220-3224, September 15-18, 2013, Melbourne, Australia. [Paper(PDF)]
  16. Qianru Sun, Hong Liu. Action Disambiguation Analysis Using Normalized Google-Like Distance Correlogram. The 11th Asian Conference on Computer Vision (ACCV 2012 oral), Pages 425-437, November 5-9, 2012, Daejeon, Korea. (Acceptance Rate 3.6%) [Paper(PDF)]

Honor & Award

  • Mar 2016  Lise Meitner Award, by Max Planck Institute for Informatics
  • Sep 2014  Universiade International Scholarship by Shenzhen Universiade Scholarship Foundation
  • Oct 2013  China Reconstruct Scholarship  by PKU
  • Oct 2011  Social Work Award  by PKU
  • Aug 2011  Volunteer Forerunner Individual  by Universiade Organizing Committee
  • Oct 2011  Social Work Award  by PKU
  • Oct 2010-2015  Doctoral Second-class Scholarship by PKU
  • Oct 2008-2009  Tong Ding Second-class Scholarship by NJUPT 
  • Oct 2007-2009  Pivot of Merit Student  by NJUPT
  • Oct 2007-2009  School First-class Scholarship  by NJUPT
  • Oct 2007  National Scholarship  by National Ministry of Education

External Activities

  • Invited speaker in the ODAR Workshop of CVPR 2017
  • Invited speaker in the Joint Lecture Series (MPI & Saarland University)
  • SPLINE 2016, Technical committee member
  • CAAI Transactions on Intelligence Technology, Editorial Board Member
  • ICCV 2017, Reviewer
  • ICRA 2014 and 2015, Reviewer
  • ISWC 2016, Reviewer
  • IEEE Transactions on Multimedia, Reviewer
  • International Journal of Computer Vision, Reviewer
  • NeuroComputing, Reviewer
  • IEEE Transactions on Circuits and Systems for Video Technology, Reviewer