Continual and Multimodal Learning for Internet of Things

September 13, 2020 • Cancun, Mexico

A UbiComp 2020 Workshop


Internet of Things (IoT) provides streaming, large-amount, and multimodal sensing data over time. The statistical properties of these data are often significantly different by sensing modalities and temporal traits, which are hardly captured by conventional learning methods. Continual and multimodal learning allows integration, adaptation and generalization of the knowledge learnt from previous experiential data collected with heterogeneity to new situations. Therefore, continual and multimodal learning is an important step to improve the estimation, utilization, and security of real-world data from IoT devices.

Call for Papers

This workshop aims to explore the intersection and combination of continual machine learning and multimodal modeling with applications in Internet of Things. The workshop welcomes works addressing these issues in different applications and domains, such as human-centric sensing, smart cities, health and wellness, privacy and security, etc. We aim at bringing together researchers from different areas to establish a multidisciplinary community and share the latest research.

We focus on the novel learning methods that can be applied on streaming multimodal data:

  • continual learning
  • transfer learning
  • federated learning
  • few-shot learning
  • multi-task learning
  • reinforcement learning
  • learning without forgetting
  • individual and/or institutional privacy
  • balance on-device and off-device learning
  • manage high volume data flow

  • We also welcome continual learning methods that target:

  • data distribution changed caused by the fast-changing dynamic physical environment
  • missing, imbalanced, or noisy data under multimodal sensing scenarios

  • Novel applications or interfaces on streaming multimodal data are also related topics.

    As examples, the data modalities include but not limited to: WiFi, GPS, RFID, vibration, accelerometer, pressure, temperature, humidity, biochemistry, image, video, audio, speech, natural language, virtual reality, etc.

    Important Dates

  • Submission deadline: July 10, 2020
  • Notification of acceptance: July 24, 2020
  • Deadline for camera ready version: July 31, 2020
  • Workshop: September 13, 2020
  • Submit Now

    Submission Guidelines

    Please submit papers using the ACM SIGCHI portrait template. We invite papers of varying length from 2 to 6 pages, plus additional pages for the reference; i.e., the reference page(s) are not counted to the limit of 6 pages. Accepted papers will be included in the ACM Digital Library and supplemental proceedings of the conference. Reviews are not double-blind, and author names and affiliations should be listed.


    Edge-Assisted Collaborative IoT Machine Intelligence, Speaker: Archan Misra, Singapore Management University

    Abstract: To support real-time & sustainable machine intelligence that exploits the rapid growth in sensor deployments and wearable devices (e.g., inertial, radar), there is a need to optimize the execution of machine learning (ML) pipelines on resource-constrained embedded devices. Through this talk, I shall introduce the paradigm of collaborative machine intelligence (CMI), where the sensing and ML pipelines on individual wearable and IoT devices collaborate, in real-time. I shall describe why CMI also requires the evolution of an edge node, from merely a resource for offloaded computation, to a platform for coordination among IoT devices. Using an exemplar video surveillance application, I will describe how IoT-based CMI can provide dramatic reductions in sensing energy, while also improving accuracy and minimizing communication overheads. I shall also describe how CMI can be leveraged to support (a) ultra-low power (and even battery-less operation) of wearable devices and (b) resource-efficient instruction comprehension for natural human-machine interfaces.

    Speaker Bio: Archan Misra is Vice Provost (Research) and Professor of Computer Science at Singapore Management University (SMU). He is the Director of SMU’s Center for Applied Smart-Nation Analytics (CASA), which is developing pervasive technologies for smart city infrastructure and applications. Archan has led a number of multi-million dollar, large-scale research initiatives at SMU, including the LiveLabs research center, and is a current recipient of the prestigious Investigator grant (from Singapore’s National Research Foundation) for sustainable man-machine interaction intelligence. Over a 20+ year research career spanning both academics and industry (at IBM Research and Bellcore), Archan has published on, and practically deployed, technologies spanning wireless networking, mobile & wearable sensing and urban mobility analytics. His current research interests lie in ultra-low energy execution of machine intelligence algorithms using wearable and IoT devices. Archan holds a Ph.D. from the University of Maryland at College Park, and chaired the IEEE Computer Society's Technical Committee on Computer Communications (TCCC) from 2005-2007.


    Workshop Chairs (Feel free to contact us by, if you have any questions.)
  • Susu Xu (Qualcomm AI Research)
  • Tong Yu (Samsung Research America)
  • Shijia Pan (UC Merced)

  • Advising Committee
  • Pei Zhang (Carnegie Mellon University)
  • Hae Young Noh (Stanford University)
  • Joseph Soriaga (Qualcomm AI Research)
  • Steve Gu (AiFi Inc.)
  • Hongxia Jin (Samsung Research America)

  • Technical Program Committee
  • Xiaoxi Zhang (Carnegie Mellon University)
  • Sheng Li (University of Georgia)
  • Xidong Pi (Uber ATG)
  • Zhihan Fang (Rutgers University)
  • Wei Ma (The Hong Kong Polytechnic University)
  • Adeola Bannis (Carnegie Mellon University)
  • Mostafa Mirshekari (Stanford University)
  • Rui Ma (Tsinghua University)
  • Shanghang Zhang (UC Berkeley)
  • Aniruddha Basak (Amazon)
  • Handong Zhao (Adobe Research)

  • Agenda (Cancun Time)

    Welcome! (9:00 - 9:15)
    Keynote (9:15 - 10:00), Speaker: Archan Misra, Singapore Management University
    Session 1: Multimodal Sensing and Learning for Smart Health (10:00 - 11:00)
  • RCH: Robust Calibration Based on Historical Data for Low-Cost Air Quality Sensor Deployments, Guodong Li, Rui Ma, Xinyu Liu, Yue Wang, Lin Zhang
  • Spinal curve assessment of idiopathic scoliosis with a small dataset via a multi-scale keypoint estimation approach, Liu Tianyu, Yang Yukang, Wang Yu, Sun Ming, Fan Wenhui, Wu Cheng, Cody Bunger
  • Digital Twin - A Machine Learning Approach to Predict Individual Stress Levels in Extreme Environments, Dr. Constantin Stefan Scheuermann, Thomas Binderberger, Nadine von Frankenberg, Dr. med Andreas Werner

  • Lunch (11:00 - 12:30)
    Session 2: Continual and Multimodal Learning for Smart Building (12:30 - 13:30)
  • Space Utilization and Activity Recognition using 3D Stereo Vision Camera inside an Educational Building, Anooshmita Das Das, Krister, Mikkel Baun Kjærgaard
  • Fine-grained Activities Recognition with Coarse-grained Labeled Multi-modal Data, Zhizhang Hu, Tong Yu, Yue Zhang, Shijia Pan
  • Accurate Trajectory Prediction in a Smart Building using Recurrent Neural Networks, Anooshmita Das, Mikkel Baun Kjærgaard

  • Session 3: System Evaluation and Calibration (13:30 - 13:50)
  • Evaluation of Federated Learning Aggregation Algorithms, Sannara Ek, François Portet, Philippe Lalanda and German Vega

  • Registration/Doors Close (14:00)

    Note: For each paper presentation, there are 15 minutes for presentation and 5 minutes for Q&A.

    Copyright © All Rights Reserved | This template is made with by Colorlib