Hierarchical Neural Architecture Search for Deep Stereo Matching (with Xuelian Cheng, Yiran Zhong, Mehrtash Harandi, Yuchao Dai, Xiaojun Chang, Hongdong Li and Zongyuan Ge)

This paper shows how to apply neural architecture search to the structured problem of stereo matching, where the network has to both extract features and compare them across two views. Rather than searching at a single level, the method searches simultaneously over cell structure (the local feature operations) and network structure (how the cells are connected for matching and aggregation), producing architectures that outperform hand-designed stereo networks like PSMNet on KITTI and Middlebury while using far fewer parameters.

[NeurIPS 2020 paper]

EMPNet: Neural Localisation and Mapping Using Embedded Memory Points (with Gil Avraham, Yan Zuo and Thanuja Dharmasiri)

This paper shows how to compute dense descriptors over images that can be used to augment ICP for localisation from large baselines.

[ICCV 2019 paper]

Real-time joint semantic segmentation and depth estimation using asymmetric annotations (with Vladimir Nekrasov, Thanuja Dharmasiri, Andrew Spek, Chunhua Shen and Ian Reid)

This paper shows the benefits of simultaneously estimating semantics and depth for a monocular image input stream.  The resulting network can perform this estimate at 13ms per frame, enabling it to be used in real-time systems.

[ICRA 2019 paper]

Eng: End-to-end neural geometry for robust depth and pose estimation using cnns (with Thanuja Dharmasiri and Andrew Spek)

This paper shows how to compute camera motion using networks to estimate depth per frame and optical flow between frames with uncertainty.  These estimates and uncertainties are then combined using conventional optimisation to obtain motion.

[ACCV 2018 paper]

Relative pose based redundancy removal: Collaborative RGB-D data transmission in mobile visual sensor networks (with Xiaoqin Wang, Ahmet Şekercioğlu, Vincent Frémont, Enrico Natalizio and Isabelle Fantoni)

This paper shows how two robots that are exploring the same environment can share RGBD data to fill in gaps in each other's sensory data while simultaneously estimating their relative pose in the world.

[Sensors 2018 paper]

Efficient Subpixel Refinement with Symbolic Linear Predictors (with Vincent Lui, Jonathon Geeves and Winston Yii)

This paper introduces a method for very rapid sub-pixel refinement of correspondences.  The method precomputes an update matrix that applies to the pixel intensity errors.  The update matrix is itself computed from a quadratic and linear form of the pixels in the image patch.  Finally the method also enables an estimation of the precision that will be obtained from a given reference patch.

[CVPR 2018 paper]

Updated notes on Lie groups

The notes on Lie groups have been updated.  The theory chapter has been split into two chapters for Lie groups and projective geometry.  Sections on the Lie bracket and conics have been added.  Notes available here.

Australian Centre for Robotic Vision

The Australian Research Council have just awarded us $19M to establish a Centre of Excellence in Robotic Vision.  The centre will address some of the key challenges in enabling robots to use vision to operate in unstructured and dynamic environments alongside humans.

[ARC Announcement]

A Unified Rolling Shutter and Motion Blur Model for 3D Visual Registration (with Maxime Meilland and Andrew Comport)

This paper shows how rolling shutter and motion blur can be handled in a unified way within a SLAM framework by computing the predicted appearance (and derivatives) of each row in the image based on its individual exposure duration and 6 DoF motion.

[ICCV 2013 paper]

A Real-Time Distributed Relative Pose Estimation Algorithm for RGB-D Camera Equipped Visual Sensor Networks (with Max Wang and Ahmet Sekercioglu)

In this paper, we present a distributed, peer-to-peer algorithm for relative pose estimation in a network of mobile robots equipped with RGB-D cameras acting as a visual sensor network.  Our algorithm uses the depth information to estimate the relative pose of a robot when camera sensors mounted on different robots observe a common scene from different angles of view.

[ICDSC 2013 paper]

Reduced dimensionality EKF for SLAM (with Dinesh Gamage)

This paper presents a method for reducing the computational complexity of Kalman Filters where large numbers of dimensions have no process noise (e.g. in SLAM).  The method reduces the dimensionality of the filter by removing dimensions which have been accurately measured, retaining just the unknown dimensions in the filter.

[BMVC 2013 paper]

An Iterative 5-pt Algorithm for Fast and Robust Essential Matrix Estimation (with Vincent Lui)

This paper presents a novel algorithm for calculating epipolar geometry from 5 correspondences. The algorithm directly solves for the orientation of each camera relative to the baseline that separates them and is able to impose the half plane constraints that arise from the requirement that visible landmarks must be in front of both cameras. The algo- rithm is conceptually simple, and provides numerically stable solutions that are used as a hypothesis generator within RANSAC. It is significantly faster than existing methods and is comfortably able to provide frame-rate performance on real data.

[2103 BMVC paper]

Algorithmic methodologies for FPGA-based vision (with Yoong Kang Lim and Lindsay Kleeman)

This paper presents a methodology for developing computer vision algorithms for FPGAs and shows how the methodology can be implemented in two case studies.

[2012 Machine Vision and Applications paper]

Distributed visual processing for augmented reality (with Winston Yii and Wai Ho Li)

This paper presents a system which combines smartphones with networked infrastructure and fixed sensors and shows how these elements can be combined to deliver real-time augmented reality.  We use a Kinect to generate dynamic trackable models of the environment as it changes at video frame rate.

[ISMAR 2012 paper]

Corner matching refinement for monocular pose estimation (with Dinesh Gamage)

Many tasks in computer vision rely on accurate detection and matching of visual landmarks (e.g. image corners) between two images.  This paper presents a method for refining the coordinates of correspondences directly.  Thus given some coordinates in the first image, our goal is to maximise the accuracy of the estimate of the coordinates in second image corresponding to the same real world point without being too concerned about which real world point is being matched.

[Draft BMVC 2012 paper]

Robust egomotion estimation using ICP in inverse depth coordinates (with Dennis Lui, Titus Tang and Wai Ho Li)

This paper presents a 6 degrees of freedom egomotion estimation method using Iterative Closest Point (ICP) for low cost and low accuracy range cameras. Instead of Euclidean coordinates, the method uses inverse depth coordinates which better conforms to the error characteristics of raw sensor data. Extensive experiments were performed to evaluate different combinations of error metrics and parameters. The result is a real-time system that is accurate and robust across a variety of motion trajectories.

[ICRA 2012 paper]

Rapidly constructed appearance models for tracking in augmented reality applications (with Jeremiah Neubert and John Pretlove)

This paper shows how a user can rapidly construct a 3D model that can be used for visual tracking.  The system uses point features for initialization and edge features for tracking.

[2011 Machine Vision and Applications paper]

Visual localisation of a robot with an external RGBD sensor (with Winston Yii, Nalika Damayanthi and Wai Ho Li)

This paper presents a novel approach to visual localisation that uses a camera on the robot coupled wirelessly to an external RGB-D sensor. Unlike systems where an external sensor observes the robot, our approach merely assumes the robots camera and external sensor share a portion of their field of view. Experiments were performed using a Microsoft Kinect as the external sensor and a small mobile robot. The robot carries a smartphone, which acts as its camera, sensor processor, control platform and wireless link. Computational effort is distributed between the smartphone and a host PC connected to the Kinect. Experimental results show that the approach is accurate and robust in dynamic environments with substantial object movement and occlusions.  This work won the best student paper prize at ACRA 2011.
[ACRA 2011 paper]

Rapid scene reconstruction on mobile phones from panoramic images (with Qi Pan, Clemens Arth, Gerhard Reitmayr and Ed Rosten)

This work presents a novel system that allows for the generation of a coarse 3D model of the environment within several seconds on mobile smartphones. By using a very fast and flexible algorithm a set of panoramic images is captured to form the basis of wide field-of-view images required for reliable and robust reconstruction. A cheap on-line space carving approach based on Delaunay triangulation is employed to obtain dense, polygonal, textured representations. The use of an intuitive method to capture these images, as well as the efficiency of the reconstruction approach allows for an application on recent mobile phone hardware, giving visually pleasing results almost instantly.
[ISMAR 2011 paper]

Deterministic sample consensus with multiple match hypotheses (with Paul McIlroy, Ed Rosten and Simon Taylor)

This paper proposes a deterministic scheme for selecting correspondences from feature matching to generate motion hypotheses. The method combines matching scores, ambiguity and the past performance of motion hypotheses generated by the matches, to estimate the probability that a feature match is correct. At every stage the best correspondences are chosen to generate a hypothesis. This method will therefore only spend time on poor or ambiguous matches when the best correspondences have proven themselves to be unsuitable. The result is a system that is able to operate efficiently on ambiguous data and is suitable for implementation on devices with limited computing resources. 
[2010 BMVC Paper]