PAR3D: A Unified 3D-MLLM for Part-Aware Scene Understanding

PAR3D is a novel 3D-MLLM framework that introduces part-aware representation for comprehensive 3D scene understanding. Unlike existing models that largely focus on objects, PAR3D is designed to understand, reason about, and ground both objects and their constituent parts within 3D environments. To facilitate its development and evaluation, the researchers created ScenePart, a synthetic 3D scene dataset featuring part-level annotations and language instructions. The framework incorporates Part-Aware 3D Representation Learning to enrich visual representations with fine-grained part semantics and utilizes Hierarchical Segmentation Query Generation for grounding part targets through hierarchical object-part queries. Experimental results indicate that PAR3D significantly improves performance in part-level question answering and referring segmentation, while also maintaining strong performance on object-level vision-language tasks.