Sicheng Lu

Research

I study embodied AI and humanoid robot learning, with the operating room as the driving application: vision-language navigation, visuomotor policies, sim-to-real transfer, and perception from surgical video to embodied agents. I have worked on how a humanoid can use a room's ambient cameras to plan and replan its actions [1], and I am now training an end-to-end vision-language navigation policy for the Unitree G1 in a reconstructed operating room, with sim-to-real transfer as the goal. Before that, I built video foundation models for surgical perception [2] and generated training data for recognizing corroded screws [3].

News

Fall 2026: Course Assistant for Computer Integrated Surgery I with Prof. Russell H. Taylor.
Mar 2026: SurgRec is on arXiv, with public code and weights.
Jan 2026: Joined the ARCADE Lab at Johns Hopkins.
Aug 2025: Started the M.S.E. in Robotics at Johns Hopkins.

Projects

* Equal contribution. Also on Google Scholar.

[1] ActOR: Ambient AI-Based Agentic Orchestration for Humanoid Autonomy in Operating Rooms

Han Zhang*, Sicheng Lu*, Aleks Santari, Jeffrey K. Jopling, Masaru Ishii, Mathias Unberath

Under review

An LLM agent plans locomotion and manipulation skills for a Unitree G1 humanoid over a digital twin fused from ambient and head cameras, and replans from execution feedback. In simulation, ambient perception reduces exploration and closed-loop feedback improves multi-step success; the system also ran on the robot in a physical mock OR.
[2] Scaling Video Pretraining for Surgical Foundation Models

Sicheng Lu*, Zikai Xiao*, Jianhui Wei*, Danyu Sun, Qi Lu, Keli Hu, Yang Feng, Jian Wu, Zongxin Yang, Zuozhu Liu

arXiv preprint arXiv:2603.29966, 2026

SurgRec is a reproducible pretraining recipe for surgical video, built on 10,535 videos (214.5M frames) and evaluated on a 16-dataset benchmark across four clinical domains. SurgRec-MAE gains +3.18 points over VideoMAE on average (+5.43 in macro accuracy), and SurgRec-JEPA +2.61 over V-JEPA.
arXiv Code Weights
BibTeX
```
@article{lu2026scaling,
  title   = {Scaling Video Pretraining for Surgical Foundation Models},
  author  = {Lu, Sicheng and Xiao, Zikai and Wei, Jianhui and Sun, Danyu and
             Lu, Qi and Hu, Keli and Feng, Yang and Wu, Jian and
             Yang, Zongxin and Liu, Zuozhu},
  journal = {arXiv preprint arXiv:2603.29966},
  year    = {2026}
}
```

Figure 3 of the RustGAN paper: basic patterns and degrees of screw rusting.

[3] RustGAN: A Method to Generate Rusty Screw with Physicochemical Constraints

Sicheng Lu, Chen Xu, Qiong Lin, Yicong Gao, Zhiliang He

International Conference on Advanced Mechatronic Systems (ICAMechS), pp. 7–12, 2024

A physics-constrained GAN for scarce defect data in disassembly vision. Adding the generated images to training raised VGG16 rust classification accuracy from 91.2% to 98.7%.

Paper

BibTeX

@inproceedings{lu2024rustgan,
  title     = {{RustGAN}: A Method to Generate Rusty Screw with
               Physicochemical Constraints},
  author    = {Lu, Sicheng and Xu, Chen and Lin, Qiong and Gao, Yicong and
               He, Zhiliang},
  booktitle = {International Conference on Advanced Mechatronic Systems (ICAMechS)},
  pages     = {7--12},
  year      = {2024},
  doi       = {10.1109/ICAMechS63130.2024.10818721}
}

Research Experience

2026–now

Graduate Researcher, ARCADE Lab, Laboratory for Computational Sensing and Robotics, Johns Hopkins University. Advisor: Prof. Mathias Unberath.

ActOR [1]: I built the framework, ran it on a Unitree G1 in a physical mock OR, and was solely responsible for the simulation ablations in ORs reconstructed from real rooms.

End-to-end vision-language navigation (ongoing): I am training a G1 visuomotor policy from egocentric RGB and language, with large-scale demonstrations from a privileged teacher in a reconstructed OR.

2025–2026

Research Assistant, ZJU-UIUC Institute, Zhejiang University. Advisor: Prof. Zuozhu Liu.

SurgRec [2]: I curated 10,535 surgical videos and led pretraining with balanced sampling, benchmark construction, evaluation, and the manuscript.

2024

Undergraduate Researcher, Zhejiang University. Advisor: Prof. Yicong Gao.

RustGAN [3]: a physics-constrained GAN for scarce defect data in disassembly vision.

Education

2025–2027: M.S.E. in Robotics, Johns Hopkins University (expected May 2027)
2021–2025: B.S. in Mechanical Engineering, minor in ECE, University of Illinois Urbana-Champaign (dual degree with Zhejiang University)
2021–2025: B.Eng. in Mechanical Engineering, Zhejiang University (ZJU-UIUC Institute)

Teaching

Fall 2026: Course Assistant, EN.601.455/655 Computer Integrated Surgery I, Johns Hopkins University, with Prof. Russell H. Taylor
Spring 2024: Teaching Assistant, Introductory Solid Mechanics, ZJU-UIUC Institute, with Prof. Jiahuan Cui