-
Green Screen Augmentation Enables Scene Generalisation in Robotic Manipulation
Authors:
Eugene Teoh,
Sumit Patidar,
Xiao Ma,
Stephen James
Abstract:
Generalising vision-based manipulation policies to novel environments remains a challenging area with limited exploration. Current practices involve collecting data in one location, training imitation learning or reinforcement learning policies with this data, and deploying the policy in the same location. However, this approach lacks scalability as it necessitates data collection in multiple loca…
▽ More
Generalising vision-based manipulation policies to novel environments remains a challenging area with limited exploration. Current practices involve collecting data in one location, training imitation learning or reinforcement learning policies with this data, and deploying the policy in the same location. However, this approach lacks scalability as it necessitates data collection in multiple locations for each task. This paper proposes a novel approach where data is collected in a location predominantly featuring green screens. We introduce Green-screen Augmentation (GreenAug), employing a chroma key algorithm to overlay background textures onto a green screen. Through extensive real-world empirical studies with over 850 training demonstrations and 8.2k evaluation episodes, we demonstrate that GreenAug surpasses no augmentation, standard computer vision augmentation, and prior generative augmentation methods in performance. While no algorithmic novelties are claimed, our paper advocates for a fundamental shift in data collection practices. We propose that real-world demonstrations in future research should utilise green screens, followed by the application of GreenAug. We believe GreenAug unlocks policy generalisation to visually distinct novel locations, addressing the current scene generalisation limitations in robot learning.
△ Less
Submitted 10 July, 2024;
originally announced July 2024.
-
Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulation
Authors:
Xiao Ma,
Sumit Patidar,
Iain Haughton,
Stephen James
Abstract:
This paper introduces Hierarchical Diffusion Policy (HDP), a hierarchical agent for multi-task robotic manipulation. HDP factorises a manipulation policy into a hierarchical structure: a high-level task-planning agent which predicts a distant next-best end-effector pose (NBP), and a low-level goal-conditioned diffusion policy which generates optimal motion trajectories. The factorised policy repre…
▽ More
This paper introduces Hierarchical Diffusion Policy (HDP), a hierarchical agent for multi-task robotic manipulation. HDP factorises a manipulation policy into a hierarchical structure: a high-level task-planning agent which predicts a distant next-best end-effector pose (NBP), and a low-level goal-conditioned diffusion policy which generates optimal motion trajectories. The factorised policy representation allows HDP to tackle both long-horizon task planning while generating fine-grained low-level actions. To generate context-aware motion trajectories while satisfying robot kinematics constraints, we present a novel kinematics-aware goal-conditioned control agent, Robot Kinematics Diffuser (RK-Diffuser). Specifically, RK-Diffuser learns to generate both the end-effector pose and joint position trajectories, and distill the accurate but kinematics-unaware end-effector pose diffuser to the kinematics-aware but less accurate joint position diffuser via differentiable kinematics. Empirically, we show that HDP achieves a significantly higher success rate than the state-of-the-art methods in both simulation and real-world.
△ Less
Submitted 6 March, 2024;
originally announced March 2024.
-
In-Hand Cube Reconfiguration: Simplified
Authors:
Sumit Patidar,
Adrian Sieler,
Oliver Brock
Abstract:
We present a simple approach to in-hand cube reconfiguration. By simplifying planning, control, and perception as much as possible, while maintaining robust and general performance, we gain insights into the inherent complexity of in-hand cube reconfiguration. We also demonstrate the effectiveness of combining GOFAI-based planning with the exploitation of environmental constraints and inherently c…
▽ More
We present a simple approach to in-hand cube reconfiguration. By simplifying planning, control, and perception as much as possible, while maintaining robust and general performance, we gain insights into the inherent complexity of in-hand cube reconfiguration. We also demonstrate the effectiveness of combining GOFAI-based planning with the exploitation of environmental constraints and inherently compliant end-effectors in the context of dexterous manipulation. The proposed system outperforms a substantially more complex system for cube reconfiguration based on deep learning and accurate physical simulation, contributing arguments to the discussion about what the most promising approach to general manipulation might be. Project website: https://rbo.gitlab-pages.tu-berlin.de/robotics/simpleIHM/
△ Less
Submitted 23 August, 2023;
originally announced August 2023.
-
A deep convolutional neural network model for rapid prediction of fluvial flood inundation
Authors:
Syed Kabir,
Sandhya Patidar,
Xilin Xia,
Qiuhua Liang,
Jeffrey Neal,
Gareth Pender,
.
Abstract:
Most of the two-dimensional (2D) hydraulic/hydrodynamic models are still computationally too demanding for real-time applications. In this paper, an innovative modelling approach based on a deep convolutional neural network (CNN) method is presented for rapid prediction of fluvial flood inundation. The CNN model is trained using outputs from a 2D hydraulic model (i.e. LISFLOOD-FP) to predict water…
▽ More
Most of the two-dimensional (2D) hydraulic/hydrodynamic models are still computationally too demanding for real-time applications. In this paper, an innovative modelling approach based on a deep convolutional neural network (CNN) method is presented for rapid prediction of fluvial flood inundation. The CNN model is trained using outputs from a 2D hydraulic model (i.e. LISFLOOD-FP) to predict water depths. The pre-trained model is then applied to simulate the January 2005 and December 2015 floods in Carlisle, UK. The CNN predictions are compared favourably with the outputs produced by LISFLOOD-FP. The performance of the CNN model is further confirmed by benchmarking against a support vector regression (SVR) method. The results show that the CNN model outperforms SVR by a large margin. The CNN model is highly accurate in capturing flooded cells as indicated by several quantitative assessment matrices. The estimated error for reproducing maximum flood depth is 0 ~ 0.2 meters for the 2005 event and 0 ~ 0.5 meters for the 2015 event at over 99% of the cells covering the computational domain. The proposed CNN method offers great potential for real-time flood modelling/forecasting considering its simplicity, superior performance and computational efficiency.
△ Less
Submitted 16 September, 2020; v1 submitted 20 June, 2020;
originally announced June 2020.
