Xin Ding (丁鑫)

I am a PhD student at the University of Science and Technology of China (USTC), School of Information Science and Technology, expected to graduate in 2027. I am currently a research intern at Microsoft Research Asia. Previously, I interned at Huawei Noah's Ark Lab, where I was awarded the Huawei Innovation Pioneer (Top 2% of Central Research Institute, the only intern recipient).

My research focuses on multimodal large language models, and embodied agents. I have published papers in top international conferences in artificial intelligence and computer vision, including ICCV, CVPR, ECCV, ICLR, and ICML, and received the 2025 ICLR Spotlight. I have received the National Scholarship (Top 1%), the Huawei 2012 Star, and the Microsoft Star of Tomorrow.

Email  /  Phone: 18584891264

🔥 News & Highlights

Demos

🔥 Zetta (ζ) — Closed-loop physical intelligence with online learning

🔥 Zeva — In-context causal learning for generalizable embodied manipulation

StreamMind — Full frame rate streaming video dialogue

Em-Garde — Proactive streaming video understanding

AdaNav — Adaptive reasoning for vision-language navigation

Research

I am currently interested in the continual self-evolution of embodied agents, including high-level planning, memory management and low-level World-Action Models (WAMs).

Embodiment

Zeva-Ego learns action-centered physical priors from labeled and unlabeled egocentric video, then uses ICCL to adapt from robot action-effect feedback with frozen parameters. Scaling to 10K hours of Ego video improves RoboTwin success from 63.8% to 75.3%, while accumulated interaction experience raises success from 58% to 89% within four attempts.

A breakthrough in embodied AI that enables robots to experience "Aha Moment" through closed-loop online learning. Zetta achieves self-evolving physical intelligence by continuously learning from interaction with the environment, demonstrating unprecedented adaptability and problem-solving capabilities in real-world robotic tasks.

An In-Context Causal Learning World-Action Model (WAM) that enables robots to learn the causal impact of their actions on the environment from past interactions and apply this knowledge to future tasks. Zeva learns how to complete tasks from human demonstrations or self-exploration without training, extracting causality between actions and state changes on-the-fly during execution. Achieves 83.3% success on real-world manipulation and demonstrates strong success-rate scaling from 57% to 90% across self-evolution iterations, with robust cross-task generalization.

A state-conditioned memory compilation framework that dynamically selects and compiles relevant memories based on the agent’s current execution state into guidance. It improves over no-memory baselines across Qwen backbones (up to +129%), matches or approaches frontier closed-source systems(GPT and Gemini), and reduces latency by up to 60%..

A training-free framework for embodied skill self-evolution that performs skill-aware reflection over trajectories to distinguish skill errors from execution lapses and selectively revise skill content. It consistently improves embodied task success on ALFWorld and EmbodiedBench, enabling a frozen Qwen3.5-27B executor to reach 93.28% success on ALFWorld and outperform GPT-5.2 as a direct agent by 31.58%.

A unified KV-cache management framework for MoT-based vision-language-action models that enables cross-task KV sharing and cross-frame continuous batching for efficient parallel inference. It achieves up to 3.7× speedup while supporting simultaneous high-throughput language generation and real-time action control without performance degradation, validated on both NVIDIA 4090, Jetson AGX Thor, and a real humanoid robot..

A uncertainty-based adaptive reasoning for VLN agents via UAR Block and a heuristic-to-RL training curriculum. Achieves 3× reasoning cost reduction and surpasses closed-source models trained on millions of samples using only 6K training examples.

A fully on-device embodied intelligence system with a hierarchical architecture integrating a real-time StreamBrain, modular skill experts, and dynamic scene-episode memory for low-latency, privacy-preserving interaction. It achieves up to 7.3× faster local processing and enables real-time embodied tasks such as navigation, memory recall, and multimodal interaction on a mobile robot without relying on cloud infrastructure.

Multimodal Large Language Models

A streaming video understanding framework that dcouples semantic parsing from streaming perception; achieves 13fps real-time inference with F1 improvement of 10%+ on StreamingBench and OVO-Bench over existing Streaming VideoLLMs.

Decouples perception and cognition via Event-Perception Feature Extractor (EPFE) and cognition gate, breaking the 100fps high-frame-rate video processing bottleneck. Achieves SOTA on multiple benchmarks; deployed in Microsoft Xbox game engine and Asiainfo anomaly detection.

Model Compression

Joint optimization of quantization and KV cache for efficient image generation. Achieves W4A4 quantization with accuracy drop within 2%–9% and FP16-level quality at W8A8.

Cross-block dependency reconstruction for post-training quantization of LLMs. First to achieve W4A4 ultra-low-bit quantization of large models; accuracy gap within 1% of FP16 at W4A16 and W8A8. Deployed in Huawei Pangu and other production LLM acceleration projects.

Reinforcement learning-based active sampling scheduler for denoising diffusion models, improving sample quality and efficiency.

First quantization algorithm for SAM. Gradient-based feature redistribution and joint quantization step optimization. Achieves lossless quantization at W8A8 (accuracy gap <1% vs FP32); deployed in Huawei HiSilicon on-device deployment.

Template from Jon Barron.