Create and improve various AI support functions for dexterous teleoperation avatar through robot learning.

Research topics: Robot Learning via Deep State-Space Modeling

• Embodied AI: Proposed Diff-Control, an action diffusion policy incorporating ControlNet from the domain of image generation to robot actions generation.
• Created a multimodal learning framework (α-MDF) using attention mechanism and differentiable filtering, which conducts state estimation in latent space with multiple modalities. Experimented on real-world tasks and validated the system on both rigid body robots and soft robots.
• Developed differentiable Ensemble Kalman Filters framework incorporating algorithmic priors for robot imitation learning, i.e., learning system dynamics, missing observations, and learning representations in high-dimensional space.
• Deployed the differentiable filtering framework with smartwatch for ubiquitous robot control tasks, i.e., remote teleoperation, drone piloting.

Worked on multiple projects with different research items across computer vision, differentiable filters, optimizations.

• Developed and refined Multi-object tracking (MOT) algorithms using Bayes Filter for Video Analytics for indoor and outdoor cameras.
• Developed monocular depth prediction models with varied advanced architecture-Vision Transformer (ViT) and multi-scale local planar guidance (LPG) blocks
• Developed multi-objective optimization technique base on Frank-Wolfe algorithm for training across multiple datasets
• Proposed Depth model shows 0.117, 0.416 on abs REL and RMS error metrics and 0.868 on d1 metric on NYU depth testset