Agentic Workflow for Travel Behavior Prediction with Multimodal LLMs

What changed

This study introduces a novel three-agent workflow designed to bridge the gap between digital data collection and predictive modeling in travel behavior research. Traditionally, these stages have been developed and evaluated independently. The proposed workflow integrates conversational data collection via a chatbot, structured data processing, and behavioral prediction. A key innovation is the use of an image-augmented, stated-preference survey administered by a chatbot. This survey collected mode choices from 92 student commuters across five distinct weather scenarios, resulting in 454 respondent-scenario observations. The research evaluated nine locally deployed large language models (LLMs), with parameter counts ranging from 2 to 35 billion. These LLMs were tested under various configurations, including zero-shot prompting with context, persona-based framing, few-shot learning, and vision-based inputs using the same weather images shown to respondents.

Conventional machine learning models were used as benchmarks. A random forest model achieved 69.6% accuracy in predicting five distinct classes of travel mode choice. The best performing text-only zero-shot LLM reached 69.9% accuracy without task-specific fine-tuning. The study found that incorporating habitual travel information consistently improved LLM predictions. Expert framing generally yielded better results than role-play framing, and persona information was most effective when habitual travel data was absent. Few-shot prompting also enhanced prediction accuracy for several LLMs, with performance gains stabilizing after a small number of examples. The most significant advancement came from vision-capable LLMs. When provided with the same weather images presented to human respondents, the best vision-based configuration achieved a five-class accuracy of 71.5%, surpassing text-only models and indicating the value of visual context.

Why it matters for builders

This research offers builders a blueprint for creating more sophisticated and integrated AI systems. It showcases how conversational agents can be leveraged for active data collection, moving beyond passive surveys. Furthermore, it highlights the potential of multimodal LLMs, specifically their ability to process visual information alongside text, to enhance predictive accuracy in domain-specific applications. The structured evaluation of different prompting strategies and LLM configurations provides valuable insights for optimizing agent performance.

Practical impact

Builders can explore implementing similar agentic workflows for their own data collection and predictive modeling needs. This could involve developing chatbot interfaces for interactive surveys, integrating LLMs for analyzing qualitative responses, and experimenting with multimodal models for tasks where visual context is relevant. The findings suggest that for travel behavior analysis, incorporating habitual information and visual cues can significantly improve prediction models. Developers might consider building tools that facilitate the creation of image-augmented surveys and the evaluation of various LLM prompting techniques for specific prediction tasks.

Caveats and source limits

The study's findings are based on a sample of 92 student commuters, and broader application requires validation with larger and more representative traveler samples. While the research evaluates LLMs ranging from 2 to 35 billion parameters, specific model names and their exact performance metrics beyond the reported accuracies are not detailed. The study focuses on travel mode choice under weather conditions, and its direct applicability to other domains may vary. The exact implementation details of the multi-agent workflow and the specific LLM APIs or frameworks used are not fully elaborated in the provided excerpt.