Hi! I'm a MS Robotics student at the University of Pennsylvania's GRASP Laboratory, where I have been fortunate to work with Jason Ma, Edward Hu and Dinesh Jayaraman. My research interests include vision-language models, autonomous driving and robot learning. I am passionate about developing generalizable agents that can about and execute fine-grained manipulation, such as brewing cup of coffee latte with an espresso machine.
Education
University of Pennsylvania
M.S.E. in Robotics
Advisor: Prof. Dinesh Jayaraman
Zhejiang University
B.E. in Electronic and Computer Engineering
University of Illinois at Urbana-Champaign
B.S. in Computer Engineering
Advisor: Prof. Said Mikki
Experience
Graduate Research Assistant — University of Pennsylvania
Advisor: PhD Candidate Jason Ma & Prof. Dinesh Jayaraman
Building VLMs-based Manipulation System via Diffusion Policy for fine-grained tasks. Reproduced ReKep, SPHINX and pi0-FAST.
Multimodal MLE Intern — IDEA-Computer Vision & Robotics Department
Advisor: Dr. Xiaoke Jiang & Prof. Lei Zhang
Build, train and optimize a small human-centric video captioning model based on Grounding DINO 1.6Pro.
Team Leader & System Architect — Intelligent Pour-Over Coffee Machine Project
Advisor: Prof. Said Mikki
Led development of automated coffee machine with novel pour-over mechanism. Implemented FSM control system using Raspberry Pi 5B with focus on food-grade quality and safety.
Research Assistant — International Design Institute of ZJU
Advisor: Prof. Wei Xiang
Developed a LLMs-based framework for human-machine co-driving scenario. Submitted paper 'Visionary Co-Driver: LLMs Enhance Driver Risk Perception with ARHUD' as third author.
News
Mar 2025
ICRA 2025 paper accepted!
ZeroMimic: Distilling Robotic Manipulation Skills from Web Videos is accepted by ICRA 2025!
Sep 2024
Research at PAL Lab
Hello Pilly! Started RA at PAL Lab working with Jason Ma and Prof. Dinesh Jayaraman
June 2024
Dual Degree Graduation
Graduated from ZJU and UIUC with dual degree of B.S. in CompE + B.Eng. in ECE
Publications
IEEE Transactions on Intelligent Transportation Systems 2025
Visionary Co-Driver: LLMs Enhance Driver Risk Perception with ARHUD
Xiang Wei, Ziyue Lei, Jie Wang, Qi Zheng, Yingying Huang
Using LLMs to analyze autonomous driving road scene, modeling the behavior of pedestrians. Construct a HUD system to evaluate the driver interaction.
ICICT 2024
🏆 Oral Presentation
Real-time V2V Communication Network Cooperative Control System through Distributed Database,
Xinwen Zhu, Zihao Li, Yuxuan Jiang, Jiazhen Xu, Jie Wang, and Xuyang Ba
Using distributed databases for collision avoidance, verified on Quanser Car at crossroad scenario.
Patent 2022
终端机和一体机: A Thin Client and All-in-One Computer Design
Wang, Xiang, Wang, Jie
China Patent No. CN 216852698 U. Issued: June 28, 2022. Shenzhen Weidi Cloud Technology Co., LTD.
Portfolio
2025: Course Blogs of CIS7000: Real-World Robot Learning
A comprehensive collection of paper reading notes and algorithm reproductions for ESE6800/CIS7000 at UPenn. Learning from Prof. Antonio Loquercio and Prof. Dinesh Jayaraman on cutting-edge robotics research. 🤖📚
2024: CIS5190 RL-based High-Speed Drone Racing
Advisor: Prof. Antonio LoquercioBuilt a drone racing simulation with curriculum tracks (ellipse, split-S, UZH) using OmniDrones on IsaacSim & IssacLab. Designed a PPO-based reward function, achieving 40.7% success rate and 7.94 m/s average speed on a 127m track. Trained 150M frames across 250 parallel environments in 8 hours on 2 x NVIDIA A6000 GPUs. Explored domain randomization and trajectory-based methods for robustness.
2024: PhiloCoffee Club
A student club exploring the crossroads of coffee and philosophy, finding insights through the shared experience of a delicious cup of coffee.
2023: CS438 Wireless Network Analysis Tool
A Python-based analytical tool for comprehensive wireless network analysis, featuring signal strength heatmaps and roaming mechanisms analysis. Developed in collaboration with UIUC IT Network team to study campus Wi-Fi infrastructure. 📡
2023: Shell Eco-marathon Autonomous Programming Competition
Developed autonomous driving modules using ROS and tested in CARLA simulator with Unreal 4 Engine. Focused on efficient path planning for Shell Eco-marathon competition. 🚗
2022: ECE391: MentOS - Multi-terminal Operating System
Developed a Linux-like OS kernel using C and x86, implementing core features like paging virtual memory, IDT, GDT, and i8259-based interrupt controller. Created device drivers for RTC, keyboard, PIT and a read-only file system. Established system call linkage between user and kernel space, with support for single CPU task scheduling and multiple terminal switching. ⚙️
2021: RoboMaster University Robotics Competition
Assisted in developing and optimizing vision algorithms for object detection via YOLOv4, contributing to auto-aiming and shooting system for real-time target engagement, improving hit accuracy by 20%. Supported the integration of vision systems on Jetson Nano with embedded platforms on DJI manifold2, achieving a 30 FPS during live competition on the industrial camera. 🤖
2021: Simulation and Modeling of Unmanned Vehicle Safety Test for Baidu Apollo D-kit
Assembled Lidar, Radar and Camera modules for Baidu Apollo D-kit Self-driving Test Vehicle. Constructed high- solution Lidar maps across ZJU international campus. Improved perception module for traffic state estimation based on collected data and literature review. 🚗