Multimodal Profiling Analytics
Background
Singapore represents a vibrant trade and service-oriented economy. It is inherently a multi-racial society and home to a truly diverse populace. The varied cultures and demographics of this country present unique challenges as well as opportunities for government agencies, businesses, and service providers alike. In this environment, understanding the demographics, preferences, and affective states of people will be highly beneficial for commerce, urban safety, security, and many other areas.
Goals
The aim of our multimodal profiling analytics (MMPA) project is recognizing individuals’ attributes. We are considering multiple long-term attributes (age, gender, ethnicity, personality type) together with short-term or transient attributes (gait/posture, affect, attention, fatigue, engagement), using both explicit and subtle cues. Our MMPA approach involves the use of multiple sensor modalities (visual, cognitive, and physiological) as well as combining different information sources to infer the aforementioned attributes for informed decision-making.
Application Areas
We focus on two application areas where we believe profiling will be particularly beneficial:
- Retail commerce. Businesses can gain immensely by identifying different sections of the population and understanding their buying behavior. For example, knowing what clothes people of different genders, age groups and ethnicities wear can enable fashion outlets to reach out to different target groups more effectively, e.g. through targeted advertising. Likewise, sensing affect and attention can help generate genuine and timely feedback to shops and service-based businesses as compared to the contemporary use of survey questionnaires. Using such approaches to assess customer satisfaction can help raise the overall quality of service.
- Safety and security. Fatigue represents a major concern for critical personnel doing repetitive tasks, such as security, immigration, or customs officers. It also poses serious risks to the workforce, especially to drivers and construction workers, and generally impedes productivity. Fatigue assessment requires constant monitoring of the targets’ physical and mental states, which can be accomplished by monitoring body posture, head and eye movements, as well as measuring cognitive and physiological signals (e.g. analyzing brain signals and muscle activity in the neck and lower-back), and evaluating them in the context of the given task.
Results
Please refer to our publications.
Below is a video showcasing our facial expression analysis technology.