Visual Recognition of Families In the Wild

Presenters

Joseph P Robinson, Ming Shao, Siyu Xia, Yun Fu

Description

Automatic kinship recognition has relevance in an abundance of applications. For starters, aiding forensic investigations, as kinship is a powerful cue that could narrow the search space (e.g., knowledge that the Boston Bombers were brothers could have helped identify the suspects sooner). In short, there are many beneficiaries that could result from such technologies: whether the consumer (e.g., automatic photo library management), scholar (e.g., historic lineage & genealogical studies), data analyzer (e.g., social-media-based analysis), investigator (e.g., cases of missing children and human trafficking– for instance, it is unlikely that a missing child found online would be in any database, however, more than likely a family member would be), or even refugees. Besides application-based problems, and as already hinted, kinship is a powerful cue that could serve as a face attribute capable of greatly reducing the search space in more general face-recognition problems. With our FIW database, we can pose this relatively new and challenging problem at sizes much greater than ever before (e.g., kinship verification with 644,000 face pairs, opposed to just 2,000 and family classification with 1,000 families opposed to just 101. In the end, we also hope FIW serves as a rich resource to further bridge the semantic gap of facial recognition based problems to the broader human-computer interaction incentive.

A fair question to ask–if so useful, why is kinship recognition capable not found or even a prototype for any real-world problem? I mean, great efforts have been spent to advance automatic kinship recognition technologies dating back to 2010. We found the reasoning to be 2-fold:

1. Existing image datasets for kinship recognition tasks are not large enough to capture and reflect the true data distributions of the families of the world.
2. Kin-based relationships in the visual domain are less discriminant than other, more conventional problems (e.g., facial recognition or object classification), as there exists many hidden factors that affect the facial appearances among different family members.

Both points were addressed with the introduction of our FIW database, with data distributions to properly represent real world scenarios available at scales much larger than ever before. We expect the larger, more complex data will pave way to modern day data driven methods (i.e., deep learning) to be used in these problems much more effectively than before possible.

In this tutorial, we will introduce the background information, progress leading us up to this point, several current state-of-the-art algorithms on the various views of the kinship recognition problem (e.g., verification, classification, tri-subject). We then will cover our FIW image collection, along with the challenges it has been used in, the winners with their deep learning approaches. This tutorial will conclude with a list of future research directions.
Outline :

1. Introduction and Background
2. Various Views of Kinship Recognition
3. Families In the Wild (FIW) and Annual Challenges
4. Keynote Speaker
5. Conclusions and Future Work

Bios

Joseph P. Robinson received a BS in electrical & computer engineering (‘14) and is pursuing a PhD in computer engineering at Northeastern University (NEU), where he also works part-time faculty: designed & taught undergrad course in Data Analytics (4.95/5.00 student rating, 3 semesters). His research work is in applied machine vision, with emphasis on faces, deep learning, multimedia, and large databases. He led team on TRECVid d´eebut, then built many image & video datasets (e.g., Families In the Wild). Robinson served as organizing chair and host of various workshops & challenges (e.g., NECV17, RFIWACM-MM17, RFIWFG18, AMFGCVPR18, FacesMMICME18), tutorials (ACM-MM18), PC member (e.g., CVPR, FG, MIRP, MMEDIA, AAAI), reviewer (e.g., IEEE Transactions on Biomedical Circuits and Systems, Image Processing, Pattern Analysis and Machine Intelligence), and lead positions like president of IEEENEU & Relations Officer of IEEE SAC R1 Region. He did NSF REUs (‘10 & ‘11); co-op at Analogic Corporation & BBN Technology; interned at MIT Lincoln Labs (‘14), System & Technology Research (‘16 & ‘17), and Snap Inc. (i.e., Snapchat) (‘18).

Ming Shao is an active researcher in the fields of face recognition, large-scale social media analytics, and has published over 50 articles in the flagship conferences/journals and books. He serves many leading conferences/workshops in the vision and data mining fields, e.g., the web-chair of AMFG2013, program co-chair of IEEE Big Data Workshop 2016/2017, AMFG2017, ACM-MM RFIW Workshop 2017, FG RFIW Data Challenge 2018. He is the TPC member of IJCAI2017, AAAI2017/2018, and FG2017/2018, ICMLA2016, AMFG2015, ECCV 2016 Workshop, and reviewers of many prestigious journals including IEEE TPAMI, TNNLS, TKDE, TIP, TMM, IJCV.

Siyu Xia received his BE and MS degrees in automation engineering from Nanjing University of Aeronautics and Astronautics, Nanjing, China, in 2000 and 2003, respectively, and the PhD degree in pattern recognition and intelligence system from Southeast University, Nanjing, China, in 2006. He is currently working as an associate professor in the School of Automation at Southeast University, Nanjing, China. His research interests include object detection, applied machine learning, social media analysis, and intelligent vision systems.

Yun Fu (S’07-M’08-SM’11-F19) received the B.Eng. degree in information engineering and the M.Eng. degree in pattern recognition and intelligence systems from Xian Jiaotong University, China, respectively, and the M.S. degree in statistics and the Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana-Champaign, respectively. He is an interdisciplinary faculty member affiliated with College of Engineering and the College of Computer and Information Science at Northeastern University since 2012. His research interests are Machine Learning, Computational Intelligence, Big Data Mining, Computer Vision, Pattern Recognition, and Cyber-Physical Systems. He has extensive publications in leading journals, books/book chapters and international conferences/workshops He serves as associate editor, chairs, PC member and reviewer of many top journals and international conferences/workshops. He received seven Prestigious Young Investigator Awards from NAE, ONR, ARO, IEEE, INNS, UIUC, Grainger Foundation; nine Best Paper Awards from IEEE, IAPR, SPIE, SIAM; many major Industrial Research Awards from Google, Samsung, and Adobe, etc. He is currently an Associate Editor of the IEEE Transactions on Neural Networks and Leaning Systems (TNNLS). He is fellow of IEEE, IAPR, OSA and SPIE, a Lifetime Distinguished Member of ACM, Lifetime Member of AAAI and Institute of Mathematical Statistics, member of ACM Future of Computing Academy, Global Young Academy, AAAS, INNS and Beckman Graduate Fellow during 2007-2008.