RetourVoir sur GitHub
HTML MIT

awesome-vla-for-ad

🌐 Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future

W

worldbench

Dernière activité 24 sept. 2026
worldbench/awesome-vla-for-ad

482

étoiles

42

forks

1

issues ouvertes

3dautonomous-drivingawesome-listembodied-ailarge-language-modelsllmmulti-modalmultimodal-large-language-modelsself-drivingvision-languagevision-language-actionvision-language-modelsvlavlm

Ce README est souvent en anglais.

Awesome Logo arXiv Visitors PR's Welcome

😎 Awesome VLA for Autonomous Driving

Autonomous driving has long relied on modular "Perception-Decision-Action" pipelines, whose hand-crafted interfaces and rule-based components often struggle in complex, dynamic, or long-tailed scenarios. Their cascaded structure also amplifies upstream perception errors, undermining downstream planning and control.

This survey reviews vision-action (VA) models and vision-language-action (VLA) models for autonomous driving. We trace the evolution from early VA approaches to modern VLA frameworks, and organize existing methods into two principal paradigms:

  • End-to-End VLA, which integrates perception, reasoning, and planning within a single model.
  • Dual-System VLA, which separates slow deliberation (via VLMs) from fast, safety-critical execution (via planners).

For more details, kindly refer to our 📚 Paper, 🌐 Project Page, and 🤗 HuggingFace Leaderboard.

📚 Citation

If you find this work helpful for your research, please kindly consider citing our paper:

@article{survey_vla4ad,
    title   = {Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future},
    author  = {Tianshuai Hu and Xiaolu Liu and Song Wang and Yiyao Zhu and Ao Liang and Lingdong Kong and Guoyang Zhao and Zeying Gong and Jun Cen and Zhiyu Huang and Xiaoshuai Hao and Linfeng Li and Hang Song and Xiangtai Li and Jun Ma and Shaojie Shen and Jianke Zhu and Dacheng Tao and Ziwei Liu and Junwei Liang},
    journal = {arXiv preprint arXiv:2512.16760},
    year    = {2025},
}
@article{survey_3d_4d_world_models,
    title   = {{3D} and {4D} World Modeling: A Survey},
    author  = {Lingdong Kong and Wesley Yang and Jianbiao Mei and Youquan Liu and Ao Liang and Dekai Zhu and Dongyue Lu and Wei Yin and Xiaotao Hu and Mingkai Jia and Junyuan Deng and Kaiwen Zhang and Yang Wu and Tianyi Yan and Shenyuan Gao and Song Wang and Linfeng Li and Liang Pan and Yong Liu and Jianke Zhu and Wei Tsang Ooi and Steven C. H. Hoi and Ziwei Liu},
    journal = {arXiv preprint arXiv:2509.07996},
    year    = {2025}
}

Table of Contents

1. Vision-Action Models

1️⃣ Action-Only Models

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website GitHub
Selective Memory arXiv
Not All History Helps: Velocity-Aware Selective Memory for Long-Horizon End-to-End Autonomous Driving		 arXiv 2026 - -
MomADv2 arXiv
MomADv2: Reliable Temporal Memory for End-to-End Autonomous Driving		 arXiv 2026 - -
RoG-DAgger arXiv
RoG-DAgger: Rollout-Guided Post-Training for End-to-End Driving		 arXiv 2026 - -
CL4AD arXiv
Scaling Curriculum Learning For Autonomous Driving		 arXiv 2026 - -
VG-SAF arXiv
Variance-Guided Spatial Attention Fusion for Robust End-to-End Driving under Asymmetric Sensor Degradation		 arXiv 2026 - -
LBC arXiv
Learning by Cheating		 CoRL 2020 - GitHub
Latent-DRL arXiv
End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordances		 CVPR 2020 - -
NEAT arXiv
NEAT: Neural Attention Fields for End-to-End Autonomous Driving		 ICCV 2021 - GitHub
Roach arXiv
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach		 ICCV 2021 Website GitHub
WoR arXiv
Learning to Drive from A World on Rails		 ICCV 2021 Website GitHub
TCP arXiv
Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline		 NeurIPS 2022 - GitHub
Urban-Driver arXiv
Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients		 CoRL 2022 Website GitHub
LAV arXiv
Learning from All Vehicles		 CVPR 2022 Website GitHub
TransFuser arXiv
TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving		 TPAMI 2023 - GitHub
GRI arXiv
GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Driving		 Robotics 2023 - -
BEVPlanner arXiv
Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?		 CVPR 2024 - GitHub
Raw2Drive arXiv
Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)		 NeurIPS 2025 - -
RAD arXiv
RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning		 NeurIPS 2025 Website -
TrajDiff arXiv
TrajDiff: End-to-End Autonomous Driving without Perception Annotation		 arXiv 2025 - GitHub
SimScale arXiv
SimScale: Learning to Drive via Real-World Simulation at Scale		 arXiv 2025 Website GitHub
- arXiv
Addressing the Waypoint-Action Gap in End-to-End Autonomous Driving via Vehicle Motion Models		 arXiv 2026 - -
Velocity-Aware Selective Memory arXiv
Not All History Helps: Velocity-Aware Selective Memory for Long-Horizon End-to-End Autonomous Driving		 arXiv 2026 - -
Scaling Curriculum Learning arXiv
Scaling Curriculum Learning For Autonomous Driving		 arXiv 2026 - -
MomADv2 arXiv
MomADv2: Reliable Temporal Memory for End-to-End Autonomous Driving		 arXiv 2026 - -
RoG-DAgger arXiv
RoG-DAgger: Rollout-Guided Post-Training for End-to-End Driving		 arXiv 2026 - -
DriveZero arXiv
DriveZero: End-to-End Driving Beyond Human Demonstrations		 arXiv 2026 - -
EMPlan arXiv
Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving		 arXiv 2026 - -
RADP arXiv
Learning to Explain While Planning: Rule-Aligned Diffusion Planning for Autonomous Driving		 arXiv 2026 - -

2️⃣ Perception-Action Models

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website GitHub
ST-P3 arXiv
ST-P3: End-to-End Vision-Based Autonomous Driving via Spatial-Temporal Feature Learning		 ECCV 2022 - GitHub
UniAD arXiv
Planning-Oriented Autonomous Driving		 CVPR 2023 - GitHub
VAD arXiv
VAD: Vectorized Scene Representation for Efficient Autonomous Driving		 ICCV 2023 - GitHub
OccNet arXiv
Scene as Occupancy		 ICCV 2023 - GitHub
GenAD arXiv
GenAD: Generative End-to-End Autonomous Driving		 ECCV 2024 - GitHub
PARA-Drive CVPR
PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving		 CVPR 2024 Website -
Hydra-MDP CVPRW
Hydra-MDP: End-to-End Multimodal Planning with Multi-Target Hydra-Distillation		 CVPRW 2024 Website GitHub
SparseAD arXiv
SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Driving		 arXiv 2024 - -
GaussianAD arXiv
GaussianAD: Gaussian-Centric End-to-End Autonomous Driving		 arXiv 2024 - -
DiFSD arXiv
DiFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Driving		 arXiv 2024 - GitHub
DriveTransformer arXiv
DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving		 ICLR 2025 - GitHub
SparseDrive arXiv
SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation		 ICRA 2025 - GitHub
DiffusionDrive arXiv
DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving		 CVPR 2025 - GitHub
GoalFlow arXiv
GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Driving		 CVPR 2025 Website GitHub
GuideFlow arXiv
GuideFlow: Constraint-Guided Flow Matching for Planning in End-to-End Autonomous Driving		 arXiv 2025 - GitHub
ETA arXiv
ETA: Efficiency through Thinking Ahead, A Dual Approach to Self-Driving with Large Models		 arXiv 2025 - GitHub
Geo arXiv
Spatial Retrieval Augmented Autonomous Driving		 arXiv 2025 - -
DiffusionDriveV2 arXiv
DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Driving		 arXiv 2025 - GitHub
NaviHydra arXiv
NaviHydra: Controllable Navigation-Guided End-to-End Autonomous Driving with Hydra Distillation		 arXiv 2025 - -
Mimir arXiv
Mimir: Hierarchical Goal-Driven Diffusion with Uncertainty Propagation for End-to-End Autonomous Driving		 arXiv 2025 - GitHub
FROST-Drive arXiv
FROST-Drive: Scalable and Efficient End-to-End Driving with a Frozen Vision Encoder		 arXiv 2026 - -
DrivoR arXiv
Driving on Registers		 arXiv 2026 Website GitHub
SPS arXiv
See Less, Drive Better: Generalizable End-to-End Autonomous Driving via Foundation Models Stochastic Patch Selection		 arXiv 2026 - -
BevAD arXiv
What Matters for Scalable and Robust Learning in End-to-End Driving Planners?		 CVPR 2026 Website GitHub
BeyondSight arXiv
BeyondSight: Object Permanence for End-to-End Autonomous Driving		 arXiv 2026 - -
What's arXiv
What's Hidden Matters: Identifying Planning-Critical Occluded Agents using Vision-Language Models		 arXiv 2026 - -
Plug arXiv
Plug-and-Play Traffic Element Awareness for End-to-End Autonomous Driving		 arXiv 2026 - -
Off arXiv
Off the Rails: Hijacking the Scoring Head in Generative End-to-End Driving Planners with Safety-Violating Adversarial Perturbations		 arXiv 2026 - -
Herding arXiv
Herding End-to-End Autonomous Driving via Neuro-Symbolic Safety Guards		 arXiv 2026 - -
VG-SAF arXiv
Variance-Guided Spatial Attention Fusion for Robust End-to-End Driving under Asymmetric Sensor Degradation		 arXiv 2026 - -

3️⃣ Image-Based World Models

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website GitHub
DriveDreamer arXiv
DriveDreamer: Towards Real-World-Driven World Models for Autonomous Driving		 ECCV 2024 Website GitHub
GenAD arXiv
GenAD: Generalized Predictive Model for Autonomous Driving		 CVPR 2024 - GitHub
Drive-WM arXiv
Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving		 CVPR 2024 Website GitHub
DrivingWorld arXiv
DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT		 arXiv 2024 Website GitHub
Imagine-2-Drive arXiv
Imagine-2-Drive: Leveraging High-Fidelity World Models via Multi-Modal Diffusion Policies		 IROS 2025 Website -
DrivingGPT arXiv
DrivingGPT: Unifying Driving World Modeling and Planning with Multi-Modal Autoregressive Transformers		 ICCV 2025 Website -
Epona arXiv
Epona: Autoregressive Diffusion World Model for Autonomous Driving		 ICCV 2025 Website GitHub
VaViM arXiv
VaViM and VaVAM: Autonomous Driving through Video Generative Modeling		 arXiv 2025 Website GitHub
UniDrive-WM arXiv
UniDrive-WM: Unified Understanding, Planning and Generation World Model For Autonomous Driving		 arXiv 2026 Website -
DwD arXiv
Driving with DINO: Vision Foundation Features as a Unified Bridge for Sim-to-Real Generation in Autonomous Driving		 arXiv 2026 - -
WorldDrive arXiv
Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation		 arXiv 2026 - GitHub
OmniDreams arXiv
NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation		 arXiv 2026 - -
DriveDreamer-Policy arXiv
DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning		 arXiv 2026 - -
DriveVA arXiv
DriveVA: Video Action Models are Zero-Shot Drivers		 ECCV 2026 - GitHub
Xiaomi EV World Model arXiv
Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving		 arXiv 2026 - -
HERMES++ arXiv
HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation		 arXiv 2026 - -
LMGenDrive arXiv
LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving		 arXiv 2026 - -
X-World arXiv
X-World: Controllable Ego-Centric Multi-Camera World Models for Scalable End-to-End Driving		 arXiv 2026 - -
How arXiv
How Can Driving World Models Do Counterfactual Prediction?		 arXiv 2026 - -
Orbis arXiv
Orbis 2: A Hierarchical World Model for Driving		 arXiv 2026 - -
M$^\text{4}$World: arXiv
M$^\text{4}$World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming		 arXiv 2026 - -
Is arXiv
Is Energy Guidance All You Need? Training-Free Norm Injection for Driving World Models		 arXiv 2026 - -
GeoRoute arXiv
GeoRoute: Geometry-Aware Hybrid Inference for Traffic Future-Frame Prediction		 arXiv 2026 - -
Cam2Sim arXiv
Cam2Sim: Neural Scenario Reconstruction for Closed-Loop Autonomous Driving Simulation		 arXiv 2026 - -
RealWeather arXiv
RealWeather: Realistic and Scene-Faithful Weather Translation with Driving World Models		 arXiv 2026 - -
SUV arXiv
SUV: Future Scene Understanding as Video Generation for End-to-End Driving		 arXiv 2026 - -
ZYT-World arXiv
ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation		 arXiv 2026 - -
RoXDrive arXiv
RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts		 arXiv 2026 - -
PhysWAM arXiv
PhysWAM: Physically Consistent World Action Model for Autonomous Driving		 arXiv 2026 - -
SV-WAM arXiv
SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving		 arXiv 2026 - -

4️⃣ Occupancy-Based World Models

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website GitHub
OccWorld arXiv
OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving		 ECCV 2024 Website GitHub
NeMo ECCV
Neural Volumetric World Models for Autonomous Driving		 ECCV 2024 - -
OccVAR OpenReview
OCCVAR: Scalable 4D Occupancy Prediction via Next-Scale Prediction		 OpenReview 2024 - -
RenderWorld arXiv
RenderWorld: World Model with Self-Supervised 3D Label		 arXiv 2024 - -
DFIT-OccWorld arXiv
An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Training		 arXiv 2024 - -
Drive-OccWorld arXiv
Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving		 AAAI 2025 Website GitHub
T³Former arXiv
Temporal Triplane Transformers as Occupancy World Models		 arXiv 2025 - -
OmniNWM arXiv
OmniNWM: Omniscient Driving Navigation World Models		 arXiv 2025 - GitHub
AD-R1 arXiv
AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Models		 arXiv 2025 - -
SparseOccVLA arXiv
SparseOccVLA: Bridging Occupancy and Vision-Language Models via Sparse Queries for Unified 4D Scene Understanding and Planning		 arXiv 2026 - GitHub
GEM arXiv
GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning		 arXiv 2026 - -
OWMDrive arXiv
OWMDrive: Causality-Aware End-to-End Autonomous Driving via 4D Occupancy World Model		 arXiv 2026 - -
GaussianDWM++ arXiv
GaussianDWM++: Language-Grounded 3D Gaussian Driving World Model for Unified Scene Understanding, Editing, and Multi-Modal Generation		 arXiv 2026 - -
V2X-WAM arXiv
V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Driving		 arXiv 2026 - -

5️⃣ Latent-Based World Models

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website GitHub
DA-WAM arXiv
DA-WAM: Decision-Aligned Future Latents for Driving World Models		 arXiv 2026 - -
WA-JEPA arXiv
WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving		 arXiv 2026 - -
GeoWAM arXiv
GeoWAM: Visual Geometry World Action Models for Autonomous Driving		 arXiv 2026 - -
RISE arXiv
RISE: Adaptive Imagination for World Action Models		 arXiv 2026 - -
Covariate-Shift arXiv
Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models		 arXiv 2024 - -
World4Drive arXiv
World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model		 ICCV 2025 - -
WoTE arXiv
End-to-End Driving with Online Trajectory Evaluation via BEV World Model		 ICCV 2025 - GitHub
LAW arXiv
Enhancing End-to-End Autonomous Driving with Latent World Model		 ICLR 2025 - GitHub
SSR arXiv
Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving		 ICLR 2025 - GitHub
Echo-Planning arXiv
Echo Planning for Autonomous Driving: From Current Observations to Future Trajectories and Back		 arXiv 2025 - -
SeerDrive arXiv
Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution		 NeurIPS 2025 - GitHub
Drive-JEPA arXiv
Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving		 arXiv 2026 - GitHub
GraphWorld arXiv
GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving		 arXiv 2026 - -
Unified Driving Tokens arXiv
Unified Driving Tokens: Representation- and Geometry-Guided Discrete Tokenizer for Driving World Models and Planning		 arXiv 2026 - -
EponaV2 arXiv
EponaV2: Driving World Model with Comprehensive Future Reasoning		 arXiv 2026 - -
IDOL arXiv
IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving		 arXiv 2026 - -
ExploreVLA arXiv
ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving		 arXiv 2026 - -
Kinematics-Aware LWM arXiv
Kinematics-Aware Latent World Models for Data-Efficient Autonomous Driving		 arXiv 2026 - -
SparseWorld arXiv
SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation		 arXiv 2026 - -
LWM Survey arXiv
Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges		 arXiv 2026 - -
World Models arXiv
World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications		 arXiv 2026 - -
DynVLA arXiv
DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving		 arXiv 2026 - -
DriveWorld-VLA arXiv
DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving		 arXiv 2026 - -
CRISP arXiv
CRISP: A Spatiotemporal Camera-Radar Backbone for Driving via Forecasting-Based World-Model Pretraining		 arXiv 2026 - -
Geographic arXiv
Geographic Diversity Beats Data Volume for Cross-Domain Generalization in Zero-Label JEPA Driving World Models		 arXiv 2026 - -
Auto arXiv
Auto-JEPA: A Latent World Model of Continuous Intent for End-to-End Autonomous Driving		 arXiv 2026 - -
BrainWAM arXiv
BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving		 arXiv 2026 - -
X-Mind arXiv
X-Mind: Efficient Visual Chain-of-Thought via Predictive World Model for End-to-End Driving		 arXiv 2026 - -
WCog arXiv
WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving		 arXiv 2026 - -
HyWorldVLA arXiv
HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving		 arXiv 2026 - -
LWDrive arXiv
LWDrive: Layer-Wise World-Model-Guided Vision-Language Model Planning for Autonomous Driving		 arXiv 2026 - -
DriveCache arXiv
DriveCache: Action-Aware Caching for Driving World Model Inference		 arXiv 2026 - -
DA-WAM arXiv
DA-WAM: Decision-Aligned Future Latents for Driving World Models		 arXiv 2026 - -
RISE arXiv
RISE: Adaptive Imagination for World Action Models		 arXiv 2026 - -
WA-JEPA arXiv
WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving		 arXiv 2026 - GitHub
GeoWAM arXiv
GeoWAM: Visual Geometry World Action Models for Autonomous Driving		 arXiv 2026 - -
Drive-HWM arXiv
Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving		 arXiv 2026 - -
MM-Future arXiv
MM-Future: Multi-Mode Joint World-Action Modeling for Autonomous Driving		 arXiv 2026 - -
D-JEPA arXiv
D-JEPA: A Decision-Aligned Latent World Model		 arXiv 2026 - -
ForeDrive arXiv
ForeDrive: Foresight-Guided End-to-End Autonomous Driving with a Planning-Relevant Latent World Model		 arXiv 2026 - -
WALT arXiv
WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving		 arXiv 2026 - -
MomWorld arXiv
MomWorld: Momentum-Aware Latent World Model for Long-Horizon Autonomous Driving		 arXiv 2026 - -
AD-E2E-JEPA arXiv
AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving		 arXiv 2026 - GitHub
World4Scorer arXiv
World4Scorer: Outcome-Grounded World Modeling for Autonomous Driving		 arXiv 2026 Website GitHub
ReWAM arXiv
ReWAM: Reciprocal World Action Models for Interactive Autonomous Driving		 arXiv 2026 - GitHub

2. Vision-Language-Action Models

1️⃣ Textual Action Generator

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website GitHub
GAPL arXiv
GAPL: Grounded Action-effect Policy Learning for LLM-Based Trajectory Planning		 arXiv 2026 - -
MAO-LLM arXiv
Multi-Agent Orchestration with the Common-Sense Reasoning Capabilities of LLMs for Autonomous Driving		 arXiv 2026 - -
DriveMLM arXiv
DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving		 arXiv 2023 - GitHub
RAG-Driver arXiv
RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model		 RSS 2024 Website GitHub
RDA-Driver arXiv
Making Large Language Models Better Planners with Reasoning-Decision Alignment		 ECCV 2024 - -
DriveLM arXiv
DriveLM: Driving with Graph Visual Question Answering		 ECCV 2024 Website GitHub
DriveGPT4 arXiv
DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model		 RA-L 2024 Website -
DriVLMe arXiv
DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experience		 IROS 2024 Website GitHub
LLaDA arXiv
Driving Everywhere with Large Language Model Policy Adaptation		 CVPR 2024 Website GitHub
VLAAD WACVW
VLAAD: Vision and Language Assistant for Autonomous Driving		 WACVW 2024 - GitHub
OccLLaMA arXiv
OccLLaMA: A Unified Occupancy-Language-Action World Model for Understanding and Generation Tasks in Autonomous Driving		 arXiv 2024 Website -
Doe-1 arXiv
Doe-1: Closed-Loop Autonomous Driving with Large World Model		 arXiv 2024 Website GitHub
LINGO-2 arXiv
LINGO-2: Driving with Natural Language		 - Website -
SafeAuto arXiv
SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models		 ICML 2025 - GitHub
OpenEMMA arXiv
OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving		 WACV 2025 - GitHub
ReasonPlan arXiv
ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Driving		 CoRL 2025 - GitHub
WKER arXiv
World Knowledge-Enhanced Reasoning Using Instruction-Guided Interactor in Autonomous Driving		 AAAI 2025 - -
OmniDrive arXiv
OmniDrive: A Holistic LLM-Agent Framework for Autonomous Driving with 3D Perception, Reasoning and Planning		 CVPR 2025 - GitHub
S4-Driver arXiv
S4-Driver: Scalable Self-Supervised Driving Multimodal Large Language Model with Spatio-Temporal Visual Representation		 CVPR 2025 Website -
Occ-LLM arXiv
Occ-LLM: Enhancing Autonomous Driving with Occupancy-BasedLarge Language Models		 ICRA 2025 - -
DriveBench arXiv
Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives		 ICCV 2025 Website GitHub
FutureSightDrive arXiv
FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving		 NeurIPS 2025 Website GitHub
ImpromptuVLA arXiv
Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models		 NeurIPS 2025 Website GitHub
Sce2DriveX arXiv
Sce2DriveX: A Generalized MLLM Framework for Scene-to-Drive Learning		 RA-L 2025 - -
EMMA arXiv
EMMA: End-to-End Multimodal Model for Autonomous Driving		 TMLR 2025 Website -
DriveAgent-R1 arXiv
DriveAgent-R1: Advancing VLM-Based Autonomous Driving with Hybrid Thinking and Active Perception		 arXiv 2025 - -
Drive-R1 arXiv
Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning		 arXiv 2025 - -
FastDriveVLA arXiv
FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-Based Token Pruning		 arXiv 2025 - -
WiseAD arXiv
WiseAD: Knowledge Augmented End-to-End Autonomous Driving with Vision-Language Model		 arXiv 2025 Website GitHub
AutoDrive-R² arXiv
AutoDrive-R²: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving		 arXiv 2025 - -
OmniReason arXiv
OmniReason: A Temporal-Guided Vision-Language-Action Framework for Autonomous Driving		 arXiv 2025 - -
OpenREAD arXiv
OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Critic		 arXiv 2025 - GitHub
dVLM-AD arXiv
dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoning		 arXiv 2025 - -
PLA arXiv
A Unified Perception-Language-Action Framework for Adaptive Autonomous Driving		 arXiv 2025 - -
AlphaDrive arXiv
AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning		 arXiv 2025 - GitHub
CoReVLA arXiv
CoReVLA: A Dual-Stage End-to-End Autonomous Driving Framework for Long-Tail Scenarios via Collect-and-Refine		 arXiv 2025 Website GitHub
WAM-Diff arXiv
WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Driving		 arXiv 2025 - GitHub
VLADriveBench arXiv
VLADriveBench: Evaluating CoT-Action Relationship in VLA for Autonomous Driving		 arXiv 2026 - -
BLUE arXiv
BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Driving		 arXiv 2026 - -
DriveMA arXiv
DriveMA: Rethinking Language Interfaces in Driving VLAs with One-Step Meta-Actions		 arXiv 2026 - -
C-CoT arXiv
C-CoT: Counterfactual Chain-of-Thought with Vision-Language Models for Safe Autonomous Driving		 arXiv 2026 - -
MAGNIFIED arXiv
MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning		 arXiv 2026 - -
DriveReward arXiv
DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving		 arXiv 2026 - -
nuReasoning arXiv
nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving		 arXiv 2026 - -
Decision-Making arXiv
Decision-Making with Lightweight Confidence-Aware Language Model for Autonomous Driving		 arXiv 2026 - -
Is arXiv
Is VLA Reasoning Faithful? Probing Safety of Chain-of-Causation in Autonomous Driving Models		 arXiv 2026 - -
ReasonBreak arXiv
ReasonBreak: Probing Vulnerabilities in Reasoning-Enabled Vision-Language-Action Models for Autonomous Driving		 arXiv 2026 - -
Intend, arXiv
Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving		 arXiv 2026 - -
Judge, Then Drive arXiv
Judge, Then Drive: A Critic-Centric Vision Language Action Framework for Autonomous Driving		 arXiv 2026 - -
EvoDrive arXiv
EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents		 arXiv 2026 - -
Unifying arXiv
Unifying Language-Action Understanding and Generation for Autonomous Driving		 arXiv 2026 - -
MindDriver arXiv
MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Driving		 arXiv 2026 - -
HERMES arXiv
HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Driving		 arXiv 2026 - -
Counterfactual VLA arXiv
Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning		 arXiv 2025 - -
OmniDrive-R1 arXiv
OmniDrive-R1: Reinforcement-driven Interleaved Multi-modal Chain-of-Thought for Trustworthy Vision-Language Autonomous Driving		 arXiv 2025 - -
BeLLA arXiv
BeLLA: End-to-End Birds Eye View Large Language Assistant for Autonomous Driving		 arXiv 2025 - -
Reasoning arXiv
Reasoning-aware Speculative Decoding for Efficient Vision-Language-Action Models in Autonomous Driving		 arXiv 2026 - -
What arXiv
What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning		 arXiv 2026 - -
Teaching arXiv
Teaching Vision-Language-Action Models What to See and Where to Look		 arXiv 2026 - -
Deferred arXiv
Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs		 arXiv 2026 - -
Depth arXiv
Depth-Wise Probing and Pruning of the Planning Token in a Driving Vision-Language-Action Model		 arXiv 2026 - -
CritiqueDriveVLM arXiv
CritiqueDriveVLM: From Verifier-Guided Reinforcement Learning to Latent Thought Distillation for Autonomous Driving		 arXiv 2026 - -
FactorDrive arXiv
FactorDrive: Adaptive Multi-Step Reasoning Driven by Planning-Critical Factors for End-to-End Autonomous Driving		 arXiv 2026 - -
XCoT arXiv
XCoT-VLA: Executable Chain-of-Thought for Vision-Language-Action Driving		 arXiv 2026 - -
MVPruner arXiv
MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving		 arXiv 2026 - -
FocusDrive arXiv
FocusDrive: Reasoning with Visual Focus for Autonomous Driving		 arXiv 2026 - -

2️⃣ Numerical Action Generator

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website GitHub
Geo-VLA arXiv
Geo-VLA: Geometry-Aware Vision-Language-Action Planning via Internalization of Map Semantics		 arXiv 2026 - -
RedLight-VLA arXiv
RedLight-VLA: Models for traffic-rule grounding and behavioral emphasis in driving policies		 arXiv 2026 - -
Collab-MMI arXiv
A Collaborative Multi-Modality Interaction for VLA-based End-to-End Autonomous Driving		 arXiv 2026 - -
Drive the Thoughts arXiv
Drive the Thoughts: Runtime Monitoring of VLA Reasoning-Trajectory Consistency		 arXiv 2026 - -
LMDrive arXiv
LMDrive: Closed-Loop End-to-End Driving with Large Language Models		 CVPR 2024 Website GitHub
BEVDriver arXiv
BEVDriver: Leveraging BEV Maps in LLMs for Robust Closed-Loop Driving		 IROS 2025 - -
CoVLA-Agent arXiv
CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Driving		 WACV 2025 Website -
ORION arXiv
ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generation		 ICCV 2025 Website GitHub
SimLingo arXiv
SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment		 CVPR 2025 Website GitHub
DriveGPT4-V2 CVPR
DriveGPT4-V2: Harnessing Large Language Model Capabilities for Enhanced Closed-Loop Autonomous Driving		 CVPR 2025 - -
AutoVLA arXiv
AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning		 NeurIPS 2025 Website GitHub
DriveMoE arXiv
DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving		 arXiv 2025 Website GitHub
DSDrive arXiv
DSDrive: Distilling Large Language Model for Lightweight End-to-End Autonomous Driving with Unified Reasoning and Planning		 arXiv 2025 - -
OccVLA arXiv
OccVLA: Vision-Language-Action Model with Implicit 3D Occupancy Supervision.		 arXiv 2025 - -
VDRive arXiv
VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-End Autonomous Driving		 arXiv 2025 - -
ReflectDrive arXiv
Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving		 arXiv 2025 - GitHub
E3AD arXiv
E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving		 arXiv 2025 - -
LCDrive arXiv
Latent Chain-of-Thought World Modeling for End-to-End Driving		 arXiv 2025 - -
Alpamayo-R1 arXiv
Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail		 arXiv 2025 - -
UniUGP arXiv
UniUGP: Unifying understanding, generation, and planing for end-to-end autonomous driving.		 arXiv 2025 - -
MindDrive arXiv
MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Driving		 arXiv 2025 - -
AdaThinkDrive arXiv
AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Driving		 arXiv 2025 - -
Percept-WAM arXiv
Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Driving		 arXiv 2025 - -
Reasoning-VLA arXiv
Reasoning-VLA: A Fast and General Vision-Language-Action Reasoning Model for Autonomous Driving		 arXiv 2025 - -
SpaceDrive arXiv
SpaceDrive: Infusing Spatial Awareness into VLM-Based Autonomous Driving		 arXiv 2025 - -
OpenDriveVLA arXiv
OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model		 AAAI 2026 Website GitHub
WAM-Flow arXiv
WAM-Flow: Parallel Coarse-to-Fine Motion Planning via Discrete Flow Matching for Autonomous Driving		 CVPR 2026 GitHub
ColaVLA arXiv
ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving		 CVPR 2026 Website GitHub
AutoMoT arXiv
AutoMoT: A Unified Vision-Language-Action Model with Asynchronous Mixture-of-Transformers for End-to-End Autonomous Driving		 arXiv 2026 - -
OneDrive arXiv
OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models		 arXiv 2026 - -
UniDriveVLA arXiv
UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving		 arXiv 2026 - -
VLA-World arXiv
Learning Vision-Language-Action World Models for Autonomous Driving		 arXiv 2026 - -
Metis arXiv
Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation		 arXiv 2026 - -
MindVLA-U1 arXiv
MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving		 arXiv 2026 - -
DVGT-2 arXiv
DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale		 arXiv 2026 - -
VLGA arXiv
VLGA: Vision-Language-Geometry-Action Models for Autonomous Driving		 arXiv 2026 - -
VECTOR-Drive arXiv
VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving		 arXiv 2026 - -
ChainFlow-VLA arXiv
ChainFlow-VLA: Causal Flow Planning with Vision-Language Models		 arXiv 2026 - -
LVDrive arXiv
LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model		 arXiv 2026 - -
StyleVLA arXiv
StyleVLA: Driving Style-Aware Vision Language Action Model for Autonomous Driving		 arXiv 2026 - -
Masked-VLA-Diffusion arXiv
Efficient and Explainable End-to-End Autonomous Driving via Masked Vision-Language-Action Diffusion		 arXiv 2026 - -
Uni-World VLA arXiv
Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving		 arXiv 2026 - -
Does arXiv
Does Visual Information Play a Decisive Role in Vision-Language-Action Model Driving Behavior?		 arXiv 2026 - -
Reasoning arXiv
Reasoning About Traversability: Language-Guided Off-Road 3D Trajectory Planning		 arXiv 2026 - -
SpanVLA arXiv
SpanVLA: Efficient Action Bridging and Learning from Negative-Recovery Samples for Vision-Language-Action Model		 arXiv 2026 - -
Sim2Real-AD arXiv
Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Driving		 arXiv 2026 - -
Drive My Way arXiv
Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving		 arXiv 2026 - -
DriveVLM-RL arXiv
DriveVLM-RL: Neuroscience-Inspired Reinforcement Learning with Vision-Language Models for Safe and Deployable Autonomous Driving		 arXiv 2026 - -
Learning from Mistakes arXiv
Learning from Mistakes: Post-Training for Driving VLA with Takeover Data		 arXiv 2026 - -
SAMoE-VLA arXiv
SAMoE-VLA: A Scene Adaptive Mixture-of-Experts Vision-Language-Action Model for Autonomous Driving		 arXiv 2026 - -
LaST-VLA arXiv
LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving		 arXiv 2026 - -
PixelPilot arXiv
PixelPilot: Scalable Vision-Language-Action Models for End-to-End Autonomous Driving		 arXiv 2026 - -
S-squared arXiv
S-squared-VLA: Decoupling Semantic and Spatial Streams in Vision-Language-Action Models for Autonomous Driving		 arXiv 2026 - -
WAM arXiv
WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA		 arXiv 2026 - -
AnchorVLA arXiv
AnchorVLA: Bridging Discrete Decisions and Continuous Trajectories for Vision-Language-Action Planning		 arXiv 2026 - -
CLEAR arXiv
CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving		 arXiv 2026 - -
Post arXiv
Post-Training in End-to-End Autonomous Driving		 arXiv 2026 - -
Latent arXiv
Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Driving		 arXiv 2026 - -
Inference arXiv
Inference-Time Attention Steering for Vision-Language-Action Driving Models		 arXiv 2026 - -
Geo-VLA arXiv
Geo-VLA: Geometry-Aware Vision-Language-Action Planning via Internalization of Map Semantics		 arXiv 2026 - -
RedLight-VLA arXiv
RedLight-VLA: Models for traffic-rule grounding and behavioral emphasis in driving policies		 arXiv 2026 - -
Collaborative Multi-Modality arXiv
A Collaborative Multi-Modality Interaction for VLA-based End-to-End Autonomous Driving		 arXiv 2026 - -
Qwen-Drive-1.0 arXiv
Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving		 arXiv 2026 Website GitHub
LaPla arXiv
Continuous Actions from Discrete Minds: Latent-Aligned Planning for End-to-End Autonomous Driving		 arXiv 2026 - -
GRAVA arXiv
GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Driving		 arXiv 2026 - GitHub
RAF-VLA arXiv
RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving		 arXiv 2026 - -
FIVE-VLA arXiv
FIVE-VLA: Fast and EffectIVE Autonomous Driving with Recurrent Action Memory		 arXiv 2026 - -
Run-then-Walk arXiv
Sometimes You Gotta Run Before You Can Walk: Run-then-Walk Scheduling Strategy for VLM Autonomous Driving		 arXiv 2026 - -
CAR-VLA arXiv
CAR-VLA: Complexity-Aware and Risk-Adaptive Reasoning for Autonomous Driving		 arXiv 2026 - GitHub
RefineDrive arXiv
RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Driving		 arXiv 2026 - -
AD-Memo arXiv
Vision-Language-Action Autonomous Driving Agent with Language-based Memory		 arXiv 2026 - -

3️⃣ Explicit Action Guidance

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website GitHub
DriveVLM arXiv
DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models		 CoRL 2024 Website -
LeapAD arXiv
Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Driving		 NeurIPS 2024 Website GitHub
FasionAD arXiv
FASIONAD: Fast and Slow Fusion Thinking Systems for Human-Like Autonomous Driving with Adaptive Feedback		 arXiv 2024 - -
Senna arXiv
Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving		 arXiv 2024 - GitHub
DualAD arXiv
DualAD: Dual-Layer Planning for Reasoning in Autonomous Driving		 IROS 2025 Website GitHub
DME-Driver arXiv
DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Driving		 AAAI 2025 - -
SOLVE arXiv
SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving		 CVPR 2025 - -
ReAL-AD arXiv
ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Driving		 ICCV 2025 Website -
LeapVAD arXiv
LeapVAD: A Leap in Autonomous Driving via Cognitive Perception and Dual-Process Thinking		 TNNLS 2025 - -
DiffVLA arXiv
DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Driving		 arXiv 2025 - -
FasionAD++ arXiv
FASIONAD++: Integrating High-Level Instruction and Information Bottleneck in Fast-Slow fusion Systems for Enhanced Safety in Autonomous Driving with Adaptive Feedback		 arXiv 2025 - -
HiST-VLA arXiv
HiST-VLA: A Hierarchical Spatio-Temporal Vision-Language-Action Model for End-to-End Autonomous Driving		 arXiv 2026 - -
Senna-2 arXiv
Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning		 arXiv 2026 - -
From arXiv
From Representational Complementarity to Dual Systems: Synergizing VLM and Vision-Only Backbones for End-to-End Driving		 arXiv 2026 - -
Fast-dDrive arXiv
Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving		 arXiv 2026 - -
DiffVLA++ arXiv
DiffVLA++: Bridging Cognitive Reasoning and End-to-End Driving through Metric-Guided Alignment		 arXiv 2025 - -
RT-VLA arXiv
RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillation		 arXiv 2026 - -
Slow arXiv
Slow Brain, Fast Planner: Latency-Resilient VLM-Augmented Urban Navigation		 arXiv 2026 - -
SimpleVSF arXiv
SimpleVSF: VLM-Scoring Fusion for Trajectory Prediction of End-to-End Autonomous Driving		 arXiv 2025 - -
CoWorld-VLA arXiv
CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving		 arXiv 2026 - -
OneVL arXiv
Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation		 arXiv 2026 - -
WorldVLM arXiv
WorldVLM: Combining World Model Forecasting and Vision-Language Reasoning		 arXiv 2026 - -
NaviDriveVLM arXiv
NaviDriveVLM: Decoupling High-Level Reasoning and Motion Planning for Autonomous Driving		 arXiv 2026 - -
PRAM-R arXiv
PRAM-R: A Perception-Reasoning-Action-Memory Framework with LLM-Guided Modality Routing for Adaptive Autonomous Driving		 arXiv 2026 - -
CorrectAD arXiv
CorrectAD: A Self-Correcting Agentic System to Improve End-to-end Planning in Autonomous Driving		 arXiv 2025 - -
MTRDrive arXiv
MTRDrive: Memory-Tool Synergistic Reasoning for Robust Autonomous Driving in Corner Cases		 arXiv 2025 - -
Think arXiv
Think at 5 Hz, Act at 20 Hz: Asynchronous Fast-Slow Vision-Language-Action Inference for Closed-Loop Driving		 arXiv 2026 - -
DriveVLA arXiv
DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving		 arXiv 2026 - -
FIRE arXiv
FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Driving		 arXiv 2026 - -
FlashDrive arXiv
FlashDrive: Flash Vision-Language-Action Inference for Autonomous Driving		 arXiv 2026 - -
GAPL arXiv
GAPL: Grounded Action-effect Policy Learning for LLM-Based Trajectory Planning		 arXiv 2026 - -
Multi-Agent LLM Orchestration arXiv
Multi-Agent Orchestration with the Common-Sense Reasoning Capabilities of LLMs for Autonomous Driving		 arXiv 2026 - -
LADA arXiv
Less Language, More Latents: Annotation-Efficient VLAs for Driving		 arXiv 2026 - -
RCVLA arXiv
RCVLA: 4D Radar-Grounded Semantic Reasoning and Trajectory Arbitration for Autonomous Driving		 arXiv 2026 - -

4️⃣ Implicit Representations Transfer

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website GitHub
VLP arXiv
VLP: Vision Language Planning for Autonomous Driving		 CVPR 2024 - -
VLM-AD arXiv
VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision		 CoRL 2025 - -
DiMA arXiv
Distilling Multi-modal Large Language Models for Autonomous Driving		 CVPR 2025 - -
DINO-Foresight arXiv
DINO-Foresight: Looking into the Future with DINO		 NeurIPS 2025 Website GitHub
ALN-P3 arXiv
ALN-P3: Unified Language Alignment for Perception, Prediction, and Planning in Autonomous Driving		 arXiv 2025 - -
VERDI arXiv
VERDI: VLM-Embedded Reasoning for Autonomous Driving		 arXiv 2025 - -
VLM-E2E arXiv
VLM-E2E: Enhancing End-to-End Autonomous Driving with Multimodal Driver Attention Fusion		 arXiv 2025 - -
ReCogDrive arXiv
ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving		 arXiv 2025 Website GitHub
InsightDrive arXiv
InsightDrive: Insight Scene Representation for End-to-End Autonomous Driving		 arXiv 2025 - GitHub
NetRoller arXiv
NetRoller: Interfacing General and Specialized Models for End-to-End Autonomous Driving		 arXiv 2025 - GitHub
ViLaD arXiv
ViLaD: A Large Vision Language Diffusion Framework for End-to-End Autonomous Driving		 arXiv 2025 - -
OmniScene arXiv
OmniScene: Attention-Augmented Multimodal 4D Scene Understanding for Autonomous Driving		 arXiv 2025 - -
LMAD arXiv
LMAD: Integrated End-to-End Vision�Language Model for Explainable Autonomous Driving		 arXiv 2025 - -
BEVLM arXiv
BEVLM: Distilling Semantic Knowledge from LLMs into Bird's-Eye View Representations		 ECCV 2026 Website -
Outcome arXiv
Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving		 arXiv 2026 - -
Relational Latent World Models arXiv
Relationally Grounded Latent World Models for Autonomous Driving		 arXiv 2026 - -

3. Datasets & Benchmarks

⏲️ In chronological order, from the earliest to the latest.

1️⃣ Vision-Action Datasets

Dataset Paper Venue Website GitHub
SkyDrive arXiv
SkyDrive: Learning to Drive in a New City from Aerial Traffic Monitoring		 arXiv 2026 - -
BDD100K arXiv
BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning		 CVPR 2020 Website GitHub
nuScenes arXiv
nuScenes: A Multimodal Dataset for Autonomous Driving		 CVPR 2020 Website -
Waymo arXiv
Scalability in Perception for Autonomous Driving: Waymo Open Dataset		 CVPR 2020 Website GitHub
nuPlan arXiv
nuPlan: A Closed-Loop ML-Based Planning Benchmark for Autonomous Vehicles		 arXiv 2021 Website GitHub
Argoverse 2 arXiv
Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting		 NeurIPS 2021 Website GitHub
Bench2Drive arXiv
Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-to-End Autonomous Driving		 NeurIPS 2024 - GitHub
RoboBEV arXiv
Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving		 TPAMI 2025 - GitHub
WOD-E2E arXiv
WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-Tail Scenarios		 arXiv 2025 Website GitHub
navdream arXiv
The Constant Eye: Benchmarking and Bridging Appearance Robustness in Autonomous Driving		 arXiv 2026 - -
SIREN-Bench arXiv
SIREN-Bench: Behavior-Driven Generation and Evaluation of Emergency-Vehicle Interactions		 arXiv 2026 - -
SkyDrive arXiv
SkyDrive: Learning to Drive in a New City from Aerial Traffic Monitoring		 arXiv 2026 - -
DriveDegrade arXiv
A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving		 arXiv 2026 - -
VIPS arXiv
VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulation		 arXiv 2026 Website -
VehDyn arXiv
VehDyn: A Driving World Model Benchmark for Vehicle Dynamics		 arXiv 2026 - -
ExceptionDrive arXiv
ExceptionDrive: A Planning-Oriented Counterfactual Corner-Case Benchmark for Autonomous Driving		 arXiv 2026 - -

2️⃣ Vision-Language-Action Datasets

Dataset Paper Venue Website GitHub
CAViAR arXiv
CAViAR: A Causal Video Dataset for Fine-Grained Accident Reasoning in Real-World Scenarios		 ECCV 2026 - -
V2XBench arXiv
Roadside-Cooperative Autonomous Driving: From Data Platform to Vision-Language End-to-End Reasoning		 arXiv 2026 - -
SIREN-Bench arXiv
SIREN-Bench: Behavior-Driven Generation and Evaluation of Emergency-Vehicle Interactions		 arXiv 2026 - -
DriveDegrade arXiv
A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving		 arXiv 2026 - -
CVaR-Clearance arXiv
Barrier Function Conformal Safety Clearance Certification with CVaR for Driving Trajectory Selection		 arXiv 2026 - -
Planning-Oriented E2E arXiv
Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms		 arXiv 2026 - -
BDD-X arXiv
Textual Explanations for Self-Driving Vehicles		 ECCV 2018 - GitHub
Talk2Car IEEE
Talk2Car: Predicting Physical Trajectories for Natural Language Commands		 IEEE Access 2022 - GitHub
SDN arXiv
DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents		 EMNLP 2022 - GitHub
DriveMLM arXiv
DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving		 arXiv 2023 - GitHub
LMDrive arXiv
LMDrive: Closed-Loop End-to-End Driving with Large Language Models		 CVPR 2024 Website GitHub
DriveLM-nuScenes arXiv
DriveLM: Driving with Graph Visual Question Answering		 ECCV 2024 Website GitHub
HBD arXiv
DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Driving		 AAAI 2025 - -
VLAAD WACVW
VLAAD: Vision and Language Assistant for Autonomous Driving		 WACVW 2024 - GitHub
SUP-AD arXiv
DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models		 CoRL 2024 Website -
NuInstruct arXiv
Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models		 CVPR 2024 - GitHub
WOMD-Reasoning arXiv
WOMD-Reasoning: A Large-Scale Dataset for Interaction Reasoning in Driving		 ICML 2025 Website GitHub
DriveCoT arXiv
DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving		 arXiv 2024 Website -
Reason2Drive arXiv
Reason2Drive: Towards Interpretable and Chain-Based Reasoning for Autonomous Driving		 ECCV 2024 - GitHub
DriveBench arXiv
Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives		 ICCV 2025 Website GitHub
MetaAD arXiv
AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning		 arXiv 2025 Website GitHub
OmniDrive arXiv
OmniDrive: A Holistic LLM-Agent Framework for Autonomous Driving with 3D Perception, Reasoning and Planning		 CVPR 2025 - GitHub
NuInteract arXiv
Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous Driving		 arXiv 2025 - -
DriveAction arXiv
DriveAction: A Benchmark for Exploring Human-like Driving Decisions in VLA Models		 arXiv 2025 - -
ImpromptuVLA arXiv
Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models		 arXiv 2025 Website GitHub
CoVLA arXiv
CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Driving		 WACV 2025 Website -
OmniReason-nuScenes arXiv
OmniReason: A Temporal-Guided Vision-Language-Action Framework for Autonomous Driving		 arXiv 2025 - -
OmniReason-B2D arXiv
OmniReason: A Temporal-Guided Vision-Language-Action Framework for Autonomous Driving		 arXiv 2025 - -
Bench2Drive-VL arXiv
Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Models		 arXiv 2026 - -
V2X-QA arXiv
V2X-QA: A Comprehensive Reasoning Dataset and Benchmark for Multimodal Large Language Models in Autonomous Driving Across Ego, Infrastructure, and Cooperative Views		 arXiv 2026 - -
DriveSpatial arXiv
DRIVESPATIAL: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving		 arXiv 2026 - -
GeoDrive-Bench arXiv
GeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving		 arXiv 2026 - -
Vega arXiv
Vega: Learning to Drive with Natural Language Instructions		 arXiv 2026 - -
doScenes/DVDrive arXiv
A DVDrive Approach for doScenes Instructed Driving Challenge		 arXiv 2026 - -
PedestrianQA arXiv
PEDESTRIANQA: A Benchmark for Vision-Language Models on Pedestrian Intention and Trajectory Prediction		 arXiv 2026 - -
Where arXiv
Where Does the Answer Come From? Benchmarking View-Level Visual Evidence Identification in Multi-View MLLMs for Autonomous Driving		 arXiv 2026 - -
EventDrive arXiv
EventDrive: Event Cameras for Vision-Language Driving Intelligence		 arXiv 2026 - -
CrashSight arXiv
CrashSight: A Phase-Aware, Infrastructure-Centric Video Benchmark for Traffic Crash Scene Understanding and Reasoning		 arXiv 2026 - -
Towards Safe Mobility arXiv
Towards Safe Mobility: A Unified Transportation Foundation Model enabled by Open-Ended Vision-Language Dataset		 arXiv 2026 - -
TRIP-Evaluate arXiv
TRIP-Evaluate: An Open Multimodal Benchmark for Evaluating Large Models in Transportation		 arXiv 2026 - -
DriveJudge arXiv
DriveJudge: Rethinking Autonomous Driving Evaluation with Vision-Language Models		 arXiv 2026 - -
ReactSim-Bench arXiv
ReactSim-Bench: Benchmarking Reactive Behavior World Model Simulation in Autonomous Driving		 arXiv 2026 - -
AnchorDrive arXiv
AnchorDrive: LLM Scenario Rollout with Anchor-Guided Diffusion Regeneration for Safety-Critical Scenario Generation		 arXiv 2026 - -
ScenePilot-Bench arXiv
ScenePilot-4K: A Large-Scale First-Person Dataset and Benchmark for Vision-Language Models in Autonomous Driving		 arXiv 2026 - -
RoboDriveVLM arXiv
RoboDriveVLM: A Novel Benchmark and Baseline towards Robust Vision-Language Models for Autonomous Driving		 arXiv 2025 - -
Is arXiv
Is Your VLM for Autonomous Driving Safety-Ready? A Comprehensive Benchmark for Evaluating External and In-Cabin Risks		 arXiv 2025 - -
CARScenes arXiv
CARScenes: Semantic VLM Dataset for Safe Autonomous Driving		 arXiv 2025 - -
AUTOPILOT arXiv
AUTOPILOT VQA: Benchmarking Vision-Language Models for Incident-Centric Dashcam Understanding		 arXiv 2026 - -
ObsDriveBench arXiv
ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness		 arXiv 2026 - -
CARA arXiv
CARA: Concept-Aware Risk Attention for Interpretable Collision Anticipation		 arXiv 2026 - -
CMU arXiv
CMU-Drive and V2V-VLA: Cooperative Multi-agent Unified Driving with Reasoning Benchmark and Vehicle-to-Vehicle Vision-Language-Action Models		 arXiv 2026 - -
The arXiv
The LAIA Dataset: Labelled Attention for Intelligent Automobiles		 arXiv 2026 - -
VLN arXiv
VLN-AVP: Zero-Shot Vision-Language Navigation with Hybrid Long-Short-Term Memory for Autonomous Valet Parking		 arXiv 2026 - -
Benchmarking arXiv
Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective		 arXiv 2026 - -
SSP arXiv
SSP: An Event-Matched Syn2Sim2Phy Cross-Domain Evaluation Framework for Autonomous Driving VLA Models		 arXiv 2026 - -
CAViAR arXiv
CAViAR: A Causal Video Dataset for Fine-Grained Accident Reasoning in Real-World Scenarios		 arXiv 2026 - GitHub
V2XBench arXiv
Roadside-Cooperative Autonomous Driving: From Data Platform to Vision-Language End-to-End Reasoning		 arXiv 2026 - -
Pedestrian Yielding Bias arXiv
LLM-Driven Autonomous Vehicles Inherit Human Driver Biases in Pedestrian Yielding: Results and Implications From A New Benchmark		 arXiv 2026 - -
AD-DiffBench arXiv
Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language		 arXiv 2026 - GitHub
CASCADE arXiv
CASCADE: A Spatio-Temporal-Causal Reasoning Representation and Dataset for Driving		 arXiv 2026 Website -
CoVLM-Bench arXiv
CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning		 arXiv 2026 - -
AnchorReasoning arXiv
AnchorReasoning: A Visual Grounding and Causal Reasoning Dataset in Long-Tail Autonomous Driving Scenarios		 arXiv 2026 - -
DriveHierarchy arXiv
DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution		 arXiv 2026 - GitHub
CausalDriveBench arXiv
CausalDriveBench: Evaluating Causal Reasoning in Vision-Language-Action Models for Autonomous Driving		 arXiv 2026 - -
doPlan arXiv
doPlan: A Variable-Horizon Dataset for Multi-Stage Language-Conditioned Planning in Autonomous Driving		 arXiv 2026 - GitHub
TrafficSignBench arXiv
TrafficSignBench: Rule-Centric Closed-Loop Evaluation of Traffic-Sign Compliance in Autonomous Driving		 arXiv 2026 Website GitHub

4. Applications

Model Paper Venue Website GitHub
CoLMIN arXiv
CoLMIN: LLM-based Multi-Decision Path Negotiation for Cooperative Autonomous Driving		 arXiv 2026 - -

5. Other Resources

Model Paper Venue Website GitHub
Planning-Oriented E2E Survey arXiv
Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms		 arXiv 2026 - -
Conformal Safety Clearance arXiv
Barrier Function Conformal Safety Clearance Certification with CVaR for Driving Trajectory Selection		 arXiv 2026 - -
Drive the Thoughts arXiv
Drive the Thoughts: Runtime Monitoring of VLA Reasoning-Trajectory Consistency		 arXiv 2026 - GitHub
Language Residue Survey arXiv
Rethinking Language's Role in Efficient VLA for Autonomous Vehicles: Toward Smarter, Trustworthy Driving		 arXiv 2026 - -
MemoryDrivoR arXiv
Driving on Memory		 arXiv 2026 - GitHub
Action-Grounded Reasoning Survey arXiv
Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving		 arXiv 2026 - GitHub
BEV-Forcing arXiv
Towards Zero-Shot Transfer Across Embodiments For Driving VLAs		 arXiv 2026 - GitHub
Hindsight and Foresight arXiv
Drive by Hindsight and Foresight: Tool-Grounded Synergistic Reasoning over Hierarchical Memory for Autonomous Driving		 arXiv 2026 - -
Latent-Space Monitor Audit arXiv
Auditing Latent-Space Monitors for Autonomous Driving		 arXiv 2026 - -
Selective Neuro-Symbolic Reasoning arXiv
From Scene Graphs to Answers: Selective Neuro-Symbolic Reasoning for Autonomous Driving		 arXiv 2026 - -
Metric Validity arXiv
When the Score Becomes the Target: Rethinking Metric Validity in Autonomous Driving		 arXiv 2026 - -
Decide-Then-Explain arXiv
Where Do Embodied Decisions Come From? Rethinking Latent and Explicit Reasoning		 arXiv 2026 - GitHub
VLM Reliability arXiv
Towards Reliable Vision-Language Models for Autonomous Driving		 arXiv 2026 - -

Last updated: October 4, 2026.

Contribuer sur GitHubSite du projet

Projets similaires

awesome-embodied-vla-va-vln

A curated list of state-of-the-art research in embodied AI, focusing on vision-language-action (VLA) models, vision-language navigation (VLN), and related multimodal learning approaches.

Jjonyzhang2023
3,6 k étoiles172

Awesome-LLM4AD

A curated list of awesome LLM/VLM/VLA/World Model for Autonomous Driving(LLM4AD) resources (continually updated)

large-language-modelsvision-language-action-modelvision-language-model
TThinklab-SJTU
1,9 k étoiles113

VLA-Adapter

VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model

Pythonembodied-airoboticsvision-language-action-model
OOpenHelix-Team
2,3 k étoiles210
Explorer d'autres projets