In this work, we present 3DCoMPaT, a multimodal 2D/3D dataset with 16 million rendered views of more than 10 million stylized 3D shapes carefully annotated at the part-instance level, alongside matching RGB point clouds, 3D textured meshes, depth maps, and segmentation masks.
3DCoMPaT covers 41 shape categories, 275 fine-grained part categories, and 293 fine-grained material classes that can be compositionally applied to parts of 3D objects. We render a subset of one million stylized shapes from four equally spaced views as well as four randomized views, leading to a total of 16 million renderings. This dataset primarily focuses on stylizing 3D shapes at part-level with compatible materials. Parts are segmented at the instance level, with coarse-grained and fine-grained semantic levels.
We introduce a new task, called Grounded CoMPaT Recognition (GCR), to collectively recognize and ground compositions of materials on parts of 3D objects. We present two variations of this task and adapt state-of-art 2D/3D deep learning methods to solve the problem as baselines for future research. We hope our work will help ease future research on compositional 3D Vision.
We begin with 3D shape collection, perform editing, gather material compatibility data, annotate shapes at a fine-grained level, realign misaligned shapes using part annotations, sample materials for each part, and render stylized shapes from various angles.
Grounded CoMPaT Recognition (GCR). Given an input shape, here: a chair, the task consists of (a) recognizing the shape category and (b) segmenting the part-material pairs composing it.
Move the slider to explore the dataset. These samples are illustrative examples, offering insight into the dataset's extensive content, which encompasses 16 million rendered views of over 10 million stylized 3D shapes.
We show below andomly sampled styles for a set of six geometries from the dataset. Stylized shapes are sampled with a random style parameterized by a set of compatible materials.
@inproceedings {slim_3dcompatplus_2023,
title={{3DCoMPaT++}: An improved Large-scale 3D Vision Dataset
for Compositional Recognition},
author={ Habib Slim and Xiang Li and Yuchen Li,
Mahmoud Ahmed and Mohamed Ayman and Ujjwal Upadhyay
Ahmed Abdelreheem and Arpit Prajapati,
Suhail Pothigara and Peter Wonka and Mohamed Elhoseiny },
booktitle={arXiv},
year={2023}
}@inproceedings {li20223d_compat,
title={{3DCoMPaT}: Composition of Materials on Parts of 3D Things},
author={Yuchen Li, Ujjwal Upadhyay, Habib Slim,
Ahmed Abdelreheem, Arpit Prajapati,
Suhail Pothigara, Peter Wonka, Mohamed Elhoseiny},
booktitle={17th European Conference on Computer Vision (ECCV)},
year={2022}
}