The current state of domain adaptation in person re-ID

Introduction

Person re-identification (PreID) is the task of associating images or video frames of the same person taken from different cameras or from the same camera at different times. PreID has a number of applications, including automated contact tracing, missing child detection, and real-time athlete statistics summaries at sporting events. PreID can be categorized along a number of dimensions, including: level of supervision, source, and modality. In this article, we consider unsupervised or weakly supervised, multi-source, unimodal PreID specifically (and drop the adjectives at this point).

Sample image motivating missing child detection through PreID. Photo by Chris Barbalis on Unsplash

The UDA Gap

Unsupervised domain adaptation (UDA) is the task of training a statistical model on labeled data from one or more source domains to achieve better performance on data from one or more target domains, with access to only unlabeled data in the target domain. The PreID UDA gap is a failure to bridge lessons learned from the source domain(s) to target domain(s). This gap manifests as a trained model’s failed detection of a person in the target domain, despite this person’s presence in the source domain. Common causes of the UDA gap include a change in: person’s clothing, viewing angle, camera source, lighting conditions.

A Complicating Factor

A close-set problem is one in which no unknown classes arise during testing. In the case of close-set object classification, one or more known classes are in an image or not. PreID is an open-set problem, as unknown classes (specific persons) will be present during test time. Since inter-class differences are far more subtle for PreID than for broad object detection, directly applying close-set object detection solutions to PreID may “pull different identities close in the feature space” [2]. Thus, addressing the UDA gap is complicated by this property of PreID, and requires models that are specifically designed for an open-set problem.

A Few CVPR21 PreID Stand-Outs

IEEE’s CVPR conference is one of the most popular among the computer vision community. 26 PreID papers were accepted to CVPR21 this year, showcasing advances over the past year. Below is a summary of 3 papers that advance different aspects of PreID.

Figure 1 — Overall architecture of Bai et al., which adds RDBSN and MDIF modules to mutual mean-teaching [2].
Figure 2 — Architecture of Hong et al., which has appearance and shape streams that are fused during training [3].
Figure 3 — Architecture of Li et al., which comprises a pixel content-based transformer encoder and a part prototype-based transformer decoder [4].

Conclusions

PreID is prone to the UDA gap, which is further complicated by PreID being an open-set problem. A change of appearance, posture, or camera angle can result in poor PreID performance, which is exacerbated over time by diluted re-id accuracy as new classes (persons) are continually observed, thanks to the open-set nature of PreID. Given this challenge, PreID consistently draws time and attention from the computer vision community. CVPR21 highlighted a number of advances in PreID this year that addressed aspects of the unsupervised domain adaptation gap. We reviewed three papers that addressed: utilizing multiple source datasets, person change of clothing, and person occlusion. The papers demonstrate sound progress in PreID, but also indicate that ongoing research is needed to exceed human-level performance in the real world.

References

[1] Xuan Zhang, Hao Luo, Xing Fan, Weilai Xiang, Yixiao Sun, Qiqi Xiao, Wei Jiang, Chi Zhang, Jian Sun; AlignedReID: Surpassing Human-Level Performance in Person Re-Identification (arXiv), 2018.

About the Author

Christopher Farah is a Senior Data Scientist at Anno.Ai. Chris has over 14 years of experience conducting spatial data mining research in the healthcare and national security sectors. Chris has a Bachelors in Chemical Engineering from The Cooper Union, a MA in Mathematics from St. Louis University, and a PhD in Spatial Information Science and Engineering from the University of Maine.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Anno.Ai

Anno.Ai

Operationalizing applied machine learning for the mission.