For a robot to perceive object properties with multiple sensory
modalities, it needs to interact with the object through action. This
interaction requires that an agent be embodied (i.e., the robot
interacts with the environment through a physical body within that
environment). A major challenge is to get a robot to interact with the
scene in a way that is quick and efficient. Furthermore, learning to
perceive and reason about objects in terms of multiple sensory
modalities remains a longstanding challenge in robotics. Multiple lines
of evidence from the fields of psychology and cognitive science have
demonstrated that humans rely on multiple senses (e.g., audio, haptics,
tactile, etc.) in a broad variety of contexts ranging from language
learning to learning manipulation skills. Nevertheless, most object
representations used by robots today rely solely on visual input (e.g., a
3D object model) and thus, cannot be used to learn or reason about
non-visual object properties (weight, texture, etc.).
This major question we want to address is, how do we collect large
datasets from robots exploring the world with multi-sensory inputs and
what algorithms can we use to learn and act with this data? For
instance, at several major universities, there are robots that can
operate autonomously (e.g., navigate throughout the building, manipulate
objects, etc.) for long periods of time. Such robots could potentially
generate large amount of multi-modal sensory data, coupled with the
robot's actions. While the community has focused on how to deal with
visual information (e.g., deep learning for visual features from large
scale databases), there has been far fewer explorations of how to
utilize and learn from the very different scales of data collected from
very different sensors. Specific challenges include the fact that
different sensors produce data at different sampling rates and different
resolutions. Furthermore, data produced by a robot acting in the world
is typically not independently and identically distributed (a common
assumption of machine learning algorithms) as the current data point
often depends on previous actions.
Questions of Interest:
- How should theories from the fields of psychology and cognitive
science be utilized in robotics and AI in the context of multi-sensory
perception and action?
- What sensory inputs should a robot use when learning manipulation skills?
- What are the representations that we can use to couple multi-sensory perception and action?
- What should the relationship between multi-sensory perception and action be?
- How much multi-sensory data do we need and how should it be collected?
- What multi-sensory representations are suitable for knowledge transfer between robots that have different bodies and sensors?
- What learning techniques can be used to perform real-time
perception and decision making using multi-sensory inputs in the real
physical world?
- How can Reinforcement Learning algorithms be designed to operate on multi-sensory data gathered by embodied physical agents?
- What are new approaches for addressing real-time constraints in embodied perception?
- What machine learning methods currently exist for auditory, visual, and haptic data?
- What are the useful deep learning representations and algorithms for non-visual sensory modalities?
Photo at the end of the meetings:
Important Dates [top]
October 1st, 2016: Submissions open
October 28th November 11th, 2016: Deadline for submissions (extended due to multiple requests)
November 25th, 2016 December 2nd, 2016: Notification of Acceptance Decisions
March 27-29, 2017: Symposium
Registration Information[top]
Registration Form: https://www.regonline.com/sss17
All accepted authors, invited speakers, symposium participants, and
other invited attendees must register by February 17, 2017.
Participation will be open to active participants as well as interested
individuals on a first-come, first-served basis. All registrations
should be completed by March 10, 2017. Registrations will be accepted
until March 27 via the online form, but earlier registration is
preferred.
Invited Speakers:
Dieter Fox
Department of Computer Science and Engineering
University of Washington
Allison Yamanashi Leib
Department of Psychology
UC Berkeley
Charlie Kemp
Department of Biomedical Engineering
Georgia Tech
Katherine J. Kuchenbecker
Haptic Intelligence Department, Max Planck Institute for Intelligent Systems
and Mechanical Engineering and Applied Mechanics Department, University of Pennsylvania
Oliver Brock
TU Berlin
Moqian Tian
Meta Company
Alexander Stoytchev
Department of Electrical and Computer Engineering
Iowa State University
Byron Boots
School of Interactive Computing
Georgia Tech
Schedule:
March 27th
8:30 - 9:00 |
Check-in |
|
9:00 - 9:45 |
Symposium Introduction |
|
9:45 - 10:30 |
Invited Speaker: Alexander Stoytchev |
Bootstrapping Common Sense: A Developmental Approach to Robotics |
10:30-11:00 |
Coffee Break / Discussion |
|
11:00-11:45 |
Poster Spotlight Presentations |
|
11:45-12:30 |
Invited Speaker: Byron Boots |
TBD |
12:30-2:00 |
Group Lunch |
@ Stanford Food Court |
2:00-2:45 |
Invited Speaker: Moqian Tian |
Building Multimodal Robots: What Can We Borrow From Neuroscience? |
2:45-3:30 |
Poster Session #1 |
|
3:30-4:00 |
Coffee Break / Discussion |
|
4:00-5:00 |
Breakout Session #1 |
|
5:00-5:30 |
Day 1 Summary / Discussion |
|
6:00-7:00 |
Reception |
|
March 28th
9:00 - 9:45 |
Invited Speaker: Charlie Kemp |
Multimodal Sensing for Assistive Robots |
9:45-10:30 |
Invited Speaker: Allison Yamanashi Leib |
Human Visual Perception of Complex Environments |
10:30-11:00 |
Coffee Break / Discussion |
|
11:00-11:45 |
Invited Speaker: Oliver Brock |
A Pattern for Perception: From Multi-Modal Sensor Data to Task-Relevant Information |
11:45-12:30 |
Selected Paper Presentations |
|
12:30-2:00 |
Group Lunch |
@ Stanford Food Court |
2:00-2:45 |
Invited Speaker: Katherine Kuchenbecker |
Haptic Intelligence in Robotics |
2:45-3:30 |
Invited Speaker: Dieter Fox |
TBD |
3:30-4:00 |
Coffee Break / Discussion |
|
4:00-4:45 |
Poster Session #2 |
|
4:45-5:30 |
Breakout Session #2 |
|
6:00-7:00 |
AAAI Spring Symposium Series Plenary |
|
March 29th
9:00 - 9:30 |
Invited Speaker: Jivko Sinapov |
Learning From and About Humans: A Multi-Modal Approach |
9:45-10:30 |
Breakout Session / Closing Remarks |
|
10:30-11:00 |
Coffee Break |
|
11:00-12:30 |
Group Brunch |
|
2:00 |
Outing / Sightseeing Activity (TBD) |
|
Paper Submissions [top]
We welcome student abstract submissions describing prior or ongoing
work related to multisensory perception and embodied agents. Types of
submission may include (but not limited to):
- Theoretical contributions
- Empirical studies and analysis
- System demonstrations
- Ongoing and planned research
- Position papers and/or papers that describe novel problems
Submissions should be 2-4 pages in length, plus an extra page for references, in AAAI format.
Submissions should be sent by email to aaai2017sss.imopea -- AT -- gmail.com by November 11th, 2016.
Exceptional submissions will be invited to submit a full paper to a special journal issue on the topic of the symposium.
Organizers [top]
Vivian Chu
http://www.cc.gatech.edu/~vchu7
Ph.D. Candidate
School of Interactive Computing, Georgia Institute of Technology
Jivko Sinapov
http://www.cs.utexas.edu/~jsinapov
Clinical Assistant Professor (a.k.a. glorified post-doc)
Department of Computer Science, University of Texas at Austin
Jeannette Bohg
https://am.is.tuebingen.mpg.de/person/jbohg
Senior Research Scientist
Autonomous Motion Department, MPI for Intelligent Systems, Tübingen, Germany
Sonia Chernova
http://www.cc.gatech.edu/~chernova/
Catherine M. and James E. Allchin Early-Career Assistant Professor
School of Interactive Computing, Georgia Institute of Technology
Andrea L. Thomaz
www.ece.utexas.edu/speakers/andrea-l-thomaz
Associate Professor
Department of Computer Science, University of Texas at Austin