Research Interests

Intelligent Robots

Elegant Motion

Interactive Perception

Dexterous Bi-Manual Manipulation

Reasoning through Uncertainty

Long-Horizon Planning

Creative Problem Solving

Common Sense Reasoning

Exploration Strategies

Experience

Robotics R&D Intern at NVIDIA (Jun 2023 - Feb 2024)

I designed and trained DextrAH-G: a safe, continuously reacting pixels-to-action policy that achieves state-of-the-art dexterous grasping in the real world and was trained entirely in simulation using RL and teacher-student distillation with a geometric fabric controller

Robotics R&D Intern at NVIDIA (May 2022 - Aug 2022)

I built an obstacle-aware global navigation system for robotic arms using a combination of deep reinforcement learning and supervised learning to quickly move through complex obstacle distributions without getting stuck in local minima

Deep Reinforcement Learning Research at UBC Motion Control and Character Animation Lab (Sep 2021 - Jun 2022)

I developed a quadruped robot control policy for discontinuous terrain (gaps, stepping stones) using a combination of learning-based and model-based control for robust sim-to-real transfer.

Autopilot Software Engineering Intern at Tesla (May 2021 - Aug 2021)

I developed safety-critical C++ software for Tesla’s Full Self-Driving Beta system for planning and trajectory optimization.

Embedded Systems Software Intern at Tesla (May 2020 - Aug 2020)

I developed firmware for Tesla seat motors, heaters, pumps, and sensors.

Software Engineer in Robotics at Kardium (Sep 2019 - Dec 2019)

I built robotic systems that construct and validate a medical device that treats cardiac disease.

Robotics Software Developer at Open Robotics and Google Summer of Code (May 2019 - Aug 2019)

I developed ROS software in C++ for an autonomous surface-vessel simulator called Virtual RobotX (VRX), and published this work as a coauthor at the OCEANS 2019 MTS/IEEE Seattle conference.

Robot Localization and Navigation Developer at A&K Robotics (Jan 2018 - Apr 2018)

I improved the quality assurance and development speed on an autonomous floor-cleaning robot.

Publications

DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabrics

Tyler Lum*, Martin Matak*, Viktor Makoviychuk, Ankur Handa, Arthur Allshire, Tucker Hermans, Nathan D. Ratliff, Karl Van Wyk.

In Submission

Get a Grip: Multi-Finger Grasp Evaluation at Scale Enables Robust Sim-to-Real Transfer

Tyler Lum*, Albert H. Li*, Preston Culbertson, Krishnan Srinivasan, Aaron Ames, Mac Schwager, Jeannette Bohg.

In Submission

Reinforcement Learning Enables Real-Time Planning and Control of Agile Maneuvers for Soft Robot Arms

Rianna Jitosho*, Tyler Lum*, Allison Okamura, Karen Liu.

Conference on Robot Learning (CoRL) 2023

What Makes Certain Pre-Trained Visual Representations Better for Robotic Learning?

Kyle Hsu, Tyler Lum, Ruohan Gao, Shixiang Shane Gu, Jiajun Wu, Chelsea Finn.

NeurIPS 2022 Foundation Models for Decision Making Workshop

Imaging Biomarker Knowledge Transfer for Attention-Based Diagnosis of COVID-19 in Lung Ultrasound Videos

Tyler Lum, Mobina Mahdavi, Oron Frenkel, Christopher Lee, Fatemeh Taheri Dezaki, Mohammad H. Jafari, Nathan Van Woudenberg, Ang Nan Gu, Purang Abolmaesumi, Teresa Tsang.

MICCAI Advances in Simplifying Medical UltraSound (ASMUS) 2021

Toward Maritime Robotic Simulation in Gazebo

Brian Bingham, Carlos Agüero, Michael McCarrin, Joseph Klamo, Joshua Malia, Kevin Allen, Tyler Lum, Marshall Rawson, Rumman Waqar.

OCEANS 2019 MTS/IEEE SEATTLE

Projects

Data-driven Mixed Martial Arts (MMA) Automated Scoring

When an MMA fight ends without a finish, judges render a decision based on a scoring system that can be quite subjective and prone to error. To address this issue, I built a dataset consisting of thousands of UFC fights, and used it to train an ML-based automated scoring system for MMA fights to produce unbiased fight decisions.

Virtual RobotX (VRX): Simulator for Autonomous Surface Vessels

VRX is an autonomous boat simulator that uses ROS and Gazebo for physics, rendering, and communication. I developed software to add features for customization and control, enhance the simulator's overall realism, and improve the infrastructure for competitions that use VRX.

UBC Sailbot: Fully Autonomous Sailboat

I led the development of pathfinding and perception algorithms for a fully autonomous sailboat named Raye. Raye will be the first fully autonomous sailboat to sail from Victoria to Maui (over 4000km) in 2022.

Vim Configuration for Fast Text Editing

Vim is a highly efficient and configurable text editor that I love. I shared the Vim configuration file I use for fast text editing, and I wrote articles about Vim that have over 60,000 views.

Contactless Vitals Monitoring System

Research from Press Ganey Associates found that the average ER wait time in 2009 was 6 hours, which is plenty of time for a patient's condition to worsen significantly before they are seen. To address this issue, we used a Kinect sensor to monitor the breathing of multiple people simultaneously, and send an alert if the breathing rates are outside of normal ranges.

Fully Autonomous, Multi-Robot System

We designed and built a fully autonomous, multi-robot system from scratch in 13 weeks. We developed mechanical, electrical, and software systems that allowed the robots to navigate through a complex obstacle course.

Volta Flex: Electromyography (EMG) Garment for Injury Recovery

The VOLTA Flex V1 is a garment with integrated EMG sensors that supports injury recovery. It allows users to visualize their muscle activation during strengthening exercises to support faster recovery.

Lumina: An Improved Toolkit for Central Venous Access

When a patient has critical liver deterioration and a rapid drop in blood pressure, physicians must set up a central venous access line to help normalize their blood pressure. If this is not done quickly enough, the patient will die. However, the current process is cumbersome and error-prone. We worked with an emergency room doctor to develop tools for faster setup of a central venous access line.