Agentic Workflow for Travel Behavior Prediction with Multimodal LLMs

What changed

This study proposes a novel three-agent workflow designed to bridge the gap between digital data collection and predictive modeling in travel behavior research. Traditionally, these phases are handled separately. The proposed workflow integrates conversational data collection via a chatbot-administered survey, structured data processing, and behavioral prediction. The survey, augmented with images and presented across five weather scenarios, collected mode choices from 454 student commuters. This data was then used to train and evaluate various models, including multinomial logit, logistic regression, random forest, and nine locally deployed LLMs ranging from 2 to 35 billion parameters.

Why it matters for builders

The research showcases a practical application of agentic systems for complex data analysis tasks. It provides insights into optimizing LLM performance through different prompting strategies (zero-shot, persona, few-shot) and highlights the benefits of multimodal inputs. For builders, this suggests a framework for developing more integrated and accurate AI systems that can handle diverse data types and user interactions.

Practical impact

In terms of predictive accuracy, the random forest model achieved 69.6% five-class accuracy. The best text-only zero-shot LLM reached 69.9% accuracy. Incorporating habitual travel information consistently improved predictions. Expert framing generally outperformed role-play, and persona information was most effective when habitual travel data was scarce. Few-shot prompting showed diminishing returns after a small number of examples. Crucially, the best vision-based LLM configuration, utilizing the same weather images shown to respondents, achieved 71.5% five-class accuracy, indicating that visual context can significantly enhance predictive capabilities for certain models.

Caveats and source limits

The findings are based on a specific dataset of student commuters and five weather scenarios. The evaluation of LLMs was conducted on locally deployed models, and the performance might vary with different model architectures or deployment environments. The study focuses on travel behavior prediction, and its direct applicability to other domains would require further investigation. The research paper is available on arXiv.