-
DART: Implicit Doppler Tomography for Radar Novel View Synthesis
Authors:
Tianshu Huang,
John Miller,
Akarsh Prabhakara,
Tao Jin,
Tarana Laroia,
Zico Kolter,
Anthony Rowe
Abstract:
Simulation is an invaluable tool for radio-frequency system designers that enables rapid prototyping of various algorithms for imaging, target detection, classification, and tracking. However, simulating realistic radar scans is a challenging task that requires an accurate model of the scene, radio frequency material properties, and a corresponding radar synthesis function. Rather than specifying…
▽ More
Simulation is an invaluable tool for radio-frequency system designers that enables rapid prototyping of various algorithms for imaging, target detection, classification, and tracking. However, simulating realistic radar scans is a challenging task that requires an accurate model of the scene, radio frequency material properties, and a corresponding radar synthesis function. Rather than specifying these models explicitly, we propose DART - Doppler Aided Radar Tomography, a Neural Radiance Field-inspired method which uses radar-specific physics to create a reflectance and transmittance-based rendering pipeline for range-Doppler images. We then evaluate DART by constructing a custom data collection platform and collecting a novel radar dataset together with accurate position and instantaneous velocity measurements from lidar-based localization. In comparison to state-of-the-art baselines, DART synthesizes superior radar range-Doppler images from novel views across all datasets and additionally can be used to generate high quality tomographic images.
△ Less
Submitted 6 March, 2024;
originally announced March 2024.
-
Stop Hiding The Sharp Knives: The WebAssembly Linux Interface
Authors:
Arjun Ramesh,
Tianshu Huang,
Ben L. Titzer,
Anthony Rowe
Abstract:
WebAssembly is gaining popularity as a portable binary format targetable from many programming languages. With a well-specified low-level virtual instruction set, minimal memory footprint and many high-performance implementations, it has been successfully adopted for lightweight in-process memory sandboxing in many contexts. Despite these advantages, WebAssembly lacks many standard system interfac…
▽ More
WebAssembly is gaining popularity as a portable binary format targetable from many programming languages. With a well-specified low-level virtual instruction set, minimal memory footprint and many high-performance implementations, it has been successfully adopted for lightweight in-process memory sandboxing in many contexts. Despite these advantages, WebAssembly lacks many standard system interfaces, making it difficult to reuse existing applications.
This paper proposes WALI: The WebAssembly Linux Interface, a thin layer over Linux's userspace system calls, creating a new class of virtualization where WebAssembly seamlessly interacts with native processes and the underlying operating system. By virtualizing the lowest level of userspace, WALI offers application portability with little effort and reuses existing compiler backends. With WebAssembly's control flow integrity guarantees, these modules gain an additional level of protection against remote code injection attacks. Furthermore, capability-based APIs can themselves be virtualized and implemented in terms of WALI, improving reuse and robustness through better layering. We present an implementation of WALI in a modern WebAssembly engine and evaluate its performance on a number of applications which we can now compile with mostly trivial effort.
△ Less
Submitted 6 December, 2023;
originally announced December 2023.
-
High Resolution Point Clouds from mmWave Radar
Authors:
Akarsh Prabhakara,
Tao Jin,
Arnav Das,
Gantavya Bhatt,
Lilly Kumari,
Elahe Soltanaghaei,
Jeff Bilmes,
Swarun Kumar,
Anthony Rowe
Abstract:
This paper explores a machine learning approach for generating high resolution point clouds from a single-chip mmWave radar. Unlike lidar and vision-based systems, mmWave radar can operate in harsh environments and see through occlusions like smoke, fog, and dust. Unfortunately, current mmWave processing techniques offer poor spatial resolution compared to lidar point clouds. This paper presents R…
▽ More
This paper explores a machine learning approach for generating high resolution point clouds from a single-chip mmWave radar. Unlike lidar and vision-based systems, mmWave radar can operate in harsh environments and see through occlusions like smoke, fog, and dust. Unfortunately, current mmWave processing techniques offer poor spatial resolution compared to lidar point clouds. This paper presents RadarHD, an end-to-end neural network that constructs lidar-like point clouds from low resolution radar input. Enhancing radar images is challenging due to the presence of specular and spurious reflections. Radar data also doesn't map well to traditional image processing techniques due to the signal's sinc-like spreading pattern. We overcome these challenges by training RadarHD on a large volume of raw I/Q radar data paired with lidar point clouds across diverse indoor settings. Our experiments show the ability to generate rich point clouds even in scenes unobserved during training and in the presence of heavy smoke occlusion. Further, RadarHD's point clouds are high-quality enough to work with existing lidar odometry and mapping workflows.
△ Less
Submitted 16 July, 2023; v1 submitted 18 June, 2022;
originally announced June 2022.
-
Quantum Extremal Learning
Authors:
Savvas Varsamopoulos,
Evan Philip,
Herman W. T. van Vlijmen,
Sairam Menon,
Ann Vos,
Natalia Dyubankova,
Bert Torfs,
Anthony Rowe,
Vincent E. Elfving
Abstract:
We propose a quantum algorithm for `extremal learning', which is the process of finding the input to a hidden function that extremizes the function output, without having direct access to the hidden function, given only partial input-output (training) data. The algorithm, called quantum extremal learning (QEL), consists of a parametric quantum circuit that is variationally trained to model data in…
▽ More
We propose a quantum algorithm for `extremal learning', which is the process of finding the input to a hidden function that extremizes the function output, without having direct access to the hidden function, given only partial input-output (training) data. The algorithm, called quantum extremal learning (QEL), consists of a parametric quantum circuit that is variationally trained to model data input-output relationships and where a trainable quantum feature map, that encodes the input data, is analytically differentiated in order to find the coordinate that extremizes the model. This enables the combination of established quantum machine learning modelling with established quantum optimization, on a single circuit/quantum computer. We have tested our algorithm on a range of classical datasets based on either discrete or continuous input variables, both of which are compatible with the algorithm. In case of discrete variables, we test our algorithm on synthetic problems formulated based on Max-Cut problem generators and also considering higher order correlations in the input-output relationships. In case of the continuous variables, we test our algorithm on synthetic datasets in 1D and simple ordinary differential functions. We find that the algorithm is able to successfully find the extremal value of such problems, even when the training dataset is sparse or a small fraction of the input configuration space. We additionally show how the algorithm can be used for much more general cases of higher dimensionality, complex differential equations, and with full flexibility in the choice of both modeling and optimization ansatz. We envision that due to its general framework and simple construction, the QEL algorithm will be able to solve a wide variety of applications in different fields, opening up areas of further research.
△ Less
Submitted 5 May, 2022;
originally announced May 2022.
-
A Hybrid mmWave and Camera System for Long-Range Depth Imaging
Authors:
Akarsh Prabhakara,
Diana Zhang,
Chao Li,
Sirajum Munir,
Aswin Sankanaryanan,
Anthony Rowe,
Swarun Kumar
Abstract:
mmWave radars offer excellent depth resolution even at very long ranges owing to their high bandwidth. But their angular resolution is at least an order-of-magnitude worse than camera and lidar systems. Hence, mmWave radar is not a capable 3-D imaging solution in isolation. We propose Metamoran, a system that combines the complimentary strengths of radar and camera to obtain accurate, high resolut…
▽ More
mmWave radars offer excellent depth resolution even at very long ranges owing to their high bandwidth. But their angular resolution is at least an order-of-magnitude worse than camera and lidar systems. Hence, mmWave radar is not a capable 3-D imaging solution in isolation. We propose Metamoran, a system that combines the complimentary strengths of radar and camera to obtain accurate, high resolution depth images over long ranges even in high clutter environments, all from a single fixed vantage point. Metamoran enables rich long-range depth imaging with applications in security and surveillance, roadside safety infrastructure and wide-area mapping. Our approach leverages the high angular resolution from cameras using computer vision techniques, including image segmentation and monocular depth estimation, to obtain object shape. Our core contribution is a method to convert this object shape into an RF I/Q equivalent, which we use in a novel radar processing pipeline to help declutter the scene and capture extremely weak reflections from objects at long distances. We perform a detailed evaluation of Metamoran's depth imaging capabilities in 400 diverse scenes. Our evaluation shows that Metamoran estimates the depth of static objects up to 90 m and moving objects up to 305 m and with a median error of 28 cm, an improvement of 13$\times$ compared to a naive radar+camera baseline and 23$\times$ compared to monocular depth estimation.
△ Less
Submitted 24 February, 2022; v1 submitted 14 June, 2021;
originally announced June 2021.
-
Data-driven Thermal Model Inference with ARMAX, in Smart Environments, based on Normalized Mutual Information
Authors:
Zhanhong Jiang,
Jonathan Francis,
Anit Kumar Sahu,
Sirajum Munir,
Charles Shelton,
Anthony Rowe,
Mario Bergés
Abstract:
Understanding the models that characterize the thermal dynamics in a smart building is important for the comfort of its occupants and for its energy optimization. A significant amount of research has attempted to utilize thermodynamics (physical) models for smart building control, but these approaches remain challenging due to the stochastic nature of the intermittent environmental disturbances. T…
▽ More
Understanding the models that characterize the thermal dynamics in a smart building is important for the comfort of its occupants and for its energy optimization. A significant amount of research has attempted to utilize thermodynamics (physical) models for smart building control, but these approaches remain challenging due to the stochastic nature of the intermittent environmental disturbances. This paper presents a novel data-driven approach for indoor thermal model inference, which combines an Autoregressive Moving Average with eXogenous inputs model (ARMAX) with a Normalized Mutual Information scheme (NMI). Based on this information-theoretic method, NMI, causal dependencies between the indoor temperature and exogenous inputs are explicitly obtained as a guideline for the ARMAX model to find the dominating inputs. For validation, we use three datasets based on building energy systems-against which we compare our method to an autoregressive model with exogenous inputs (ARX), a regularized ARMAX model, and state-space models.
△ Less
Submitted 10 June, 2020;
originally announced June 2020.
-
Tracking Motion and Proxemics using Thermal-sensor Array
Authors:
Chandrayee Basu,
Anthony Rowe
Abstract:
Indoor tracking has all-pervasive applications beyond mere surveillance, for example in education, health monitoring, marketing, energy management and so on. Image and video based tracking systems are intrusive. Thermal array sensors on the other hand can provide coarse-grained tracking while preserving privacy of the subjects. The goal of the project is to facilitate motion detection and group pr…
▽ More
Indoor tracking has all-pervasive applications beyond mere surveillance, for example in education, health monitoring, marketing, energy management and so on. Image and video based tracking systems are intrusive. Thermal array sensors on the other hand can provide coarse-grained tracking while preserving privacy of the subjects. The goal of the project is to facilitate motion detection and group proxemics modeling using an 8 x 8 infrared sensor array. Each of the 8 x 8 pixels is a temperature reading in Fahrenheit. We refer to each 8 x 8 matrix as a scene. We collected approximately 902 scenes with different configurations of human groups and different walking directions. We infer direction of motion of a subject across a set of scenes as left-to-right, right-to-left, up-to-down and down-to-up using cross-correlation analysis. We used features from connected component analysis of each background subtracted scene and performed Support Vector Machine classification to estimate number of instances of human subjects in the scene.
△ Less
Submitted 25 November, 2015;
originally announced November 2015.
