Xin Ding (丁鑫)

I am a PhD student at the University of Science and Technology of China (USTC), School of Information Science and Technology, expected to graduate in 2027. I am currently a research intern at Microsoft Research Asia. Previously, I interned at Huawei Noah's Ark Lab, where I was awarded the Huawei Innovation Pioneer (Top 2% of Central Research Institute, the only intern recipient).

My research focuses on multimodal large language models, and embodied agents. I have published papers in top international conferences in artificial intelligence and computer vision, including ICCV, CVPR, ECCV, ICLR, and ICML, and received the 2025 ICLR Spotlight. I have received the National Scholarship (Top 1%), the Huawei 2012 Star, and the Microsoft Star of Tomorrow.

Email  /  CV (中文)  /  Scholar  /  Github

Demos

StreamMind — Full frame rate streaming video dialogue

Em-Garde — Proactive streaming video understanding

AdaNav — Adaptive reasoning for vision-language navigation

Research

I am currently interested in the continual self-evolution of embodied agents, including high-level planning, memory management and low-level World-Action Models (WAMs).

Embodied Agents

A state-conditioned memory compilation framework that dynamically selects and compiles relevant memories based on the agent’s current execution state into guidance. It improves over no-memory baselines across Qwen backbones (up to +129%), matches or approaches frontier closed-source systems(GPT and Gemini), and reduces latency by up to 60%..

A training-free framework for embodied skill self-evolution that performs skill-aware reflection over trajectories to distinguish skill errors from execution lapses and selectively revise skill content. It consistently improves embodied task success on ALFWorld and EmbodiedBench, enabling a frozen Qwen3.5-27B executor to reach 93.28% success on ALFWorld and outperform GPT-5.2 as a direct agent by 31.58%.

A unified KV-cache management framework for MoT-based vision-language-action models that enables cross-task KV sharing and cross-frame continuous batching for efficient parallel inference. It achieves up to 3.7× speedup while supporting simultaneous high-throughput language generation and real-time action control without performance degradation, validated on both NVIDIA 4090, Jetson AGX Thor, and a real humanoid robot..

A uncertainty-based adaptive reasoning for VLN agents via UAR Block and a heuristic-to-RL training curriculum. Achieves 3× reasoning cost reduction and surpasses closed-source models trained on millions of samples using only 6K training examples.

A fully on-device embodied intelligence system with a hierarchical architecture integrating a real-time StreamBrain, modular skill experts, and dynamic scene-episode memory for low-latency, privacy-preserving interaction. It achieves up to 7.3× faster local processing and enables real-time embodied tasks such as navigation, memory recall, and multimodal interaction on a mobile robot without relying on cloud infrastructure.

Multimodal Large Language Models

A streaming video understanding framework that dcouples semantic parsing from streaming perception; achieves 13fps real-time inference with F1 improvement of 10%+ on StreamingBench and OVO-Bench over existing Streaming VideoLLMs.

Decouples perception and cognition via Event-Perception Feature Extractor (EPFE) and cognition gate, breaking the 100fps high-frame-rate video processing bottleneck. Achieves SOTA on multiple benchmarks; deployed in Microsoft Xbox game engine and Asiainfo anomaly detection.

Model Compression

Joint optimization of quantization and KV cache for efficient image generation. Achieves W4A4 quantization with accuracy drop within 2%–9% and FP16-level quality at W8A8.

Cross-block dependency reconstruction for post-training quantization of LLMs. First to achieve W4A4 ultra-low-bit quantization of large models; accuracy gap within 1% of FP16 at W4A16 and W8A8. Deployed in Huawei Pangu and other production LLM acceleration projects.

Reinforcement learning-based active sampling scheduler for denoising diffusion models, improving sample quality and efficiency.

First quantization algorithm for SAM. Gradient-based feature redistribution and joint quantization step optimization. Achieves lossless quantization at W8A8 (accuracy gap <1% vs FP32); deployed in Huawei HiSilicon on-device deployment.

Template from Jon Barron.