![]() |
|
About Me
- I am a recent PhD graduate from the Department of Computer Science at Tufts University. My research was conducted in the Multimodal Learning, Interaction, and Perception Lab, under the guidance of Prof. Jivko Sinapov. My dissertation research focused on developing innovative frameworks, both theoretically and practically, facilitating the transfer of multi-sensory object representations among robots. This ensures that newly developed robots can build upon existing knowledge rather than starting from scratch.
- I earned my MS in computer science at Tufts University in 2018 and completed BE in computer science and engineering at Chhattisgarh Swami Vivekanand Technical University in 2014.
- My research focuses on Transfer Learning in Robots, Deep Learning for Multi-modal Classification, Learning and Manipulation for Robots, Grounding of Knowledge, and Development of Representations. I specialize in developing deep learning methods to integrate different perceptions of robots into multimodal systems.
Ph.D. Dissertation
Gyan Tatiya, Learning and Transferring Implicit Knowledge of Object Properties Across Robots via Interactive Perception, Ph.D. Dissertation, Department of Computer Science, Tufts University, Medford, MA, February 2024. [Dissertation] [BibTeX] |
Publications (Conference/Journal/Workshop Papers)
![]() |
Gyan Tatiya, Jonathan Francis, Ho-Hsiang Wu, Yonatan Bisk, and Jivko Sinapov MOSAIC: Learning Unified Multi-Sensory Object Property Representations for Robot Learning via Interactive Perception In proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan, May 13-17, 2024. [Paper] [Code] [Dataset] [Dataset Description] [BibTeX] |
![]() |
Gyan Tatiya, Jonathan Francis, and Jivko Sinapov Cross-Tool and Cross-Behavior Perceptual Knowledge Transfer for Grounded Object Recognition Computing Research Repository (CoRR), 2023. [Code] [Dataset] [Dataset Description] [Dataset Visualization] |
![]() |
Gyan Tatiya, Jonathan Francis, and Jivko Sinapov Transferring Implicit Knowledge of Non-Visual Object Properties Across Heterogeneous Robot Morphologies In proceedings of the IEEE International Conference on Robotics and Automation (ICRA), London, UK, May 29 – June 02, 2023. [Paper] [Code] [Dataset] [Dataset Description] [Dataset Visualization] [BibTeX] |
![]() |
Gyan Tatiya, Jonathan Francis, Luca Bondi, Ingrid Navarro, Eric Nyberg, Jivko Sinapov, Jean Oh Knowledge-driven Scene Priors for Semantic Audio-Visual Embodied Navigation Computing Research Repository (CoRR), 2022. [BibTeX] |
![]() |
Yash Shukla, Christopher Thierauf, Ramtin Hosseini, Gyan Tatiya, and Jivko Sinapov ACuTE: Automatic Curriculum Transfer from Simple to Complex Environments In proceedings of the 21th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Auckland, New Zealand (Virtual), May 9-13, 2022. [Code] [BibTeX] |
![]() |
Shivam Goel*, Gyan Tatiya*, Matthias Scheutz and Jivko Sinapov NovelGridworlds: A Benchmark Environment for Detecting and Adapting to Novelties in Open Worlds In proceedings of the AAMAS 2021 Workshop: Adaptive and Learning Agents (ALA), London, UK (Virtual), May 3-7, 2021. [Paper] [Code] [Video] [BibTeX] |
![]() |
Faizan Muhammad, Vasanth Sarathy, Gyan Tatiya, Shivam Goel, Saurav Gyawali, and Mateo Guaman, Jivko Sinapov and Matthias Scheutz A Novelty-Centric Agent Architecture for Changing Worlds In proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), London, UK (Virtual), May 3-7, 2021. [BibTeX] |
![]() |
Gyan Tatiya, Ramtin Hosseini, Michael C. Hughes and Jivko Sinapov A Framework for Sensorimotor Cross-Perception and Cross-Behavior Knowledge Transfer for Object Categorization Frontiers in Robotics and AI, Vol. 7, special issue titled ViTac: Integrating Vision and Touch for Multimodal and Cross-Modal Perception, 2020. [Paper] [Code] [Dataset] [Dataset Visualization] [BibTeX] |
![]() |
Gyan Tatiya, Yash Shukla, Michael Edegware and Jivko Sinapov Haptic Knowledge Transfer Between Heterogeneous Robots using Kernel Manifold Alignment In proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA (Virtual), October 25-29, 2020. [Paper] [Code] [Dataset] [Raw Dataset] [Dataset Visualization] [BibTeX] |
![]() |
Gyan Tatiya, Ramtin Hosseini, Michael C. Hughes and Jivko Sinapov Sensorimotor Cross-Behavior Knowledge Transfer for Grounded Category Recognition In proceedings of the Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), Oslo, Norway, Aug. 19-22, 2019. [Paper] [Poster] [Code] [Dataset] [Dataset Description] [BibTeX] |
![]() |
Gyan Tatiya and Jivko Sinapov Sensorimotor Cross-Perception Knowledge Transfer for Grounded Category Recognition In proceedings of the ICRA 2019 Workshop: Integrating Vision and Touch for Multimodal and Cross-modal Perception (ViTac), Montreal, Canada, May 23, 2019. [Paper] [Poster] [Dataset] [Dataset Description] [BibTeX] |
![]() |
Gyan Tatiya and Jivko Sinapov Deep Multi-Sensory Object Category Recognition Using Interactive Behavioral Exploration In proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Montreal, Canada, May 20-24, 2019. [Paper] [Poster] [Code] [Dataset] [Dataset Description] [BibTeX] |
Personal Projects
NovelGridworlds | Python, OpenAI Gym, Stable Baselines, June 2020 - Present - Created a Python package for an OpenAI custom Gym environment that simulates novel situations in grid worlds. - Trained Reinforcement Learning agents on NovelGridworlds tasks using Stable Baselines. |
Twitter Hashtags Polarity | Python, Tweepy, PySpark, Google Cloud Dataproc, September - December 2019 - Created an application that fetches live Twitter data in a stream and computes polarity of tweets with hashtags. |
Fetch Pick and Place | Python, ROS, Gazebo, January - May 2019 - Implemented Grasp, Pick, Hold, Shake and Place behaviors on Fetch robot. |
Hacking "the State of the Art" UR5 | Python, ROS, MoveIt, Gazebo, January - May 2019 - Implemented Pick and Place behaviors using MoveIt on UR5 Arm + RobotiQ 85 gripper |
FlashVocab | Java, SQLite, XML, Google Cloud Storage, September - December 2017 - Developed an android app for learning 1000 GRE words with pictures & repetition scheduling with flashcards. |
https://scene-maker.appspot.com | Python, webapp2, Jinja2, Google Cloud - Vision, Datastore, Storage, September - December 2017 - Created a web application that generates 3D photo album based on labels, landmark, colors found in photos. |
Spaceship Simulation Game | JavaScript, HTML, CSS, September - December 2017 - Designed and developed a Spaceship Simulation Game. |
Hand-Written Digit Recognition | Python, NumPy, Matplotlib, September - December 2016 - Implemented the Backpropagation algorithm for neural networks to the task of hand-written digit recognition. - Evaluated performance of network architectures with different depths and widths. |
Text Classification | Python, NumPy, September - December 2016 - Trained two variants of the Naive Bayes classifier with smoothing and applied it to classify text into two different categories. - Trained two variants of the Naive Bayes classifier with smoothing and classified two text categories. |
Context Sensitive Spelling Corrector | Python, NLTK, Matplotlib, November - December 2016 - Implemented Context Word method and Bigram Hidden Markov Model to correct contextual errors in the text. - Improved baseline accuracy by 10%. |
Language Modelling | Python, NLTK, September - December 2016 - Built a bigram language model for the Brown corpus. - Calculated bigrams probability using the Maximum Likelihood Estimate, Laplace smoothing, Simple linear interpolation & Deleted Interpolation Algorithm, and calculated the perplexity for each. |
Statistical Parsing | Python, NLTK, September - December 2016 - Built a Probabilistic Context-Free Grammars (PCFG) parser using the Treebank corpora. - Implemented the probabilistic CKY algorithm for parsing a test sentence using your learned PCFG. |
Social Networking Site | PHP, MySQL, JavaScript, CSS & Ajax, January - May 2016 - Developed a social networking site that brings together several technologies like PHP, MySQL, JavaScript, CSS & Ajax. Users can create account, find other users and add them as friends or follow them. Users can also post public or private messages and update/edit their status & upload profile picture. |
Mobile Controlled Robot Based on Dual Tone Multi Frequency (DTMF) | C, January - May 2014 - Developed a robot controlled by a mobile phone from anywhere around the world. - Incorporated various hardware such as a Microcontroller (ATMEGA 16), DC Motors, LCD, DTMF decoder. |
Coursework
- Fall 2019: COMP-0119-01 Big Data
- Spring 2019: COMP-0150-08 Probabilistic Robotics for HRI
- Fall 2018: COMP 150-03: Bayesian Deep Learning
- Fall 2018: COMP 150-04: Reinforcement Learning
- Fall 2017: COMP 150-07: Computer Vision
- Fall 2017: COMP 150-MOB: Mobile Development
- Fall 2017: COMP 150-09: Cloud Applications
- Fall 2017: COMP 86-01 Object-Oriented Programming for Graphical User Interfaces
- Spring 2017: COMP 171-01: Human-Computer Interaction
- Spring 2017: COMP 160: Introduction to Algorithms
- Spring 2017: COMP 131: Artificial Intelligence
- Fall 2016: COMP 150-02: Human-Robot Interaction
- Fall 2016: COMP 150-04: Natural Language Processing
- Fall 2016: COMP 135-01: Introduction to Machine Learning