3 Column Software
Open-source projects
Navigating Uncertainty: Diffusion-based User Intention Estimation for Wheelchair Assistance
A novel diffusion-based multimodal framework for predicting wheelchair user intentions under uncertainty. Published at IEEE Transactions on Robotics 2025. .
Interface Matters: Comparing First and Third-Person Perspective Interfaces for Bi-Manual Robot Shoe Lacing Behavioural Cloning
The repository provides 3D models and assembly instructions for the GuMi teleoperation device, along with a ROS package that includes scripts for calibration and command publishing. Published at ICRA 2025..
DanceMVP: Self-Supervised Learning for Multi-Task Primitive-Based Dance Performance Assessment via Transformer Text Prompting
A novel method for human dance motion evaluation. Published at AAAI 2024. .
Forecasting Bimanual Object Manipulation Sequences From Unimanual Observations
A novel method for forecasting bimanual object manipulation sequences from unimanual observations. Published at AAAI 2024. .
Robot-Assisted-Dressing-Pipeline
Code to accompany our Science Robotics paper on "". .
Action-Conditioned Generation of Bimanual Object Manipulation Sequences
A novel method that receives an action label to generate a motion sequence of a person performing bimanual object manipulation. Published at AAAI 2023. .
Holo-Spot Unity App and ROS packages (IROS 2022)
This project includes the Unity application and ROS packages needed for controlling Boston Dynamics' Spot using the Microsoft HoloLens 2.
Using Eye Gaze to Forecast Human Pose in Everyday Pick and Place Actions
A novel method that uses gaze information to predict the object being fixated, as well as the human pose the instant the fixated object is picked or placed. Published at . .
Message-Passing Framework for Vision Prediction Stability in Human-Robot Interaction
To stabilise vision prediction results used in HRI, we present a message-passing framework (MPF) that uses the memory of individual modules to correct each other's outputs. The framework includes pose estimation, object detection, hand-object detection, and gaze estimation.
What is the Patient Looking At? Robust Gaze-Scene Intersection Under Free-Viewing Conditions
For gaze tracking under free-viewing conditions where there is large parallax between the user's head pose and the camera, we propose a fast and robust method to intersect the participant's gaze with the scene.
Using a Single Input to Forecast Human Pose in Everyday Pick and Place Actions
A method that uses an input from a single timestep to directly forecast the human pose the instant a pick or place action is performed, as well as the time taken to complete the action. Published at ICASSP 2022.
Multitask Variational Autoencoding of Human-to-Human Handover
A method that directly forecasts the human pose and object orientation in human-to-human handover sequences. Published at IROS 2021.
Event-based vision model estimation
Entropy minimisation framework for event-based vision model estimation (see ). .
RT-BENE blink estimation
A robust blink estimator based on the RT-GENE dataset (see ). (includes ROS code and link to the dataset).
RT-GENE gaze estimation
A new robust gaze estimation framework applicable in natural environments with large camera-subject distances (see ). (includes ROS code and link to the dataset).
Shared Control
An obstacle avoidance algorithm in C++ that has been adapted for shared control on differential-drive robots (see for description of its use on a robotic wheelchair). (ROS package).
TRACA visual object tracking
A new context-aware correlation filter based tracking framework to achieve both high computational speed and state-of-the-art performance among real-time trackers (see ). (includes ROS code).
iCub-HRI
A set of convenience wrappers related to perception, object manipulation and social interaction for the iCub humanoid robot (see ).
Attentional Visual Tracking
Visual tracking with an attentional mechanism that chooses a subset of the associated correlation filters for increased robustness and computational efficiency (see ). . Please see also our TRACA CVPR2018 paper below.
Markerless Perspective Taking
C++ / Python framework which implements markerless perspective Taking (see ).
Lifelong Augmentation of Multi-Modal Streaming Autobiographical Memories
C++ framework which implements an Autobiographical Memory geared towards the developmental robotics community (see ).
Kinematic Structure Correspondences
Matlab package which implements learning correspondences between kinematic structures (see ). The download includes the dataset.
Kinematic Structure Learning
Matlab package which implements unsupervised learning of kinematic structures (see Chang and Demiris CVPR2015).
C++ package which implements offline and online unsupervised learning of kinematic structures based on 3D data (see ).
HAMMER
HAMMER is a C++ library to easily instantiate cognitive architectures based on the simulation theory of mind.
Online Temporal Learning Library
The Online Temporal Learning C++ library implements several memory structures and includes two “high-level” online learning methods.
Datasets
911今日黑料Dance Dataset
This dataset contains human dance motion and music for different genres, choreographies and expertise levels (see ). .
RT-BENE Blink Estimation Dataset
This dataset contains blink labels for the RT-GENE dataset (see ). (includes source code).
RT-GENE Gaze Estimation Dataset
This dataset contains face and eye images along with the ground truth head pose and eye gaze angles (see ). (includes source code).
Kinematic Structure Correspondences Dataset
This dataset contains kinematic structures and their correspondences, see Chang et al. CVPR2016. The download includes the CVPR2016 source code.
Kinematic Structure Learning Dataset
This dataset contains various kinematic structures which can be used for unsupervised complex kinematic structure learning (see Chang and Demiris CVPR2015).
iCub Grasp Dataset
This dataset contains grasping data for nine-everyday objects collected using the iCub humanoid platform.
Other contributions
In addition, we have contributed code to the following open-source robotics projects:
