3DCoMPaT++: An improved Large-scale 3D Vision Dataset for Compositional Recognition

1KAUST, 2Poly9, 3Qure.AI

3DCoMPaT++ is a multimodal dataset for compositional 3D research with detailed part-level annotations.


In this work, we present 3DCoMPaT, a multimodal 2D/3D dataset with 16 million rendered views of more than 10 million stylized 3D shapes carefully annotated at the part-instance level, alongside matching RGB point clouds, 3D textured meshes, depth maps, and segmentation masks.

3DCoMPaT covers 41 shape categories, 275 fine-grained part categories, and 293 fine-grained material classes that can be compositionally applied to parts of 3D objects. We render a subset of one million stylized shapes from four equally spaced views as well as four randomized views, leading to a total of 16 million renderings. This dataset primarily focuses on stylizing 3D shapes at part-level with compatible materials. Parts are segmented at the instance level, with coarse-grained and fine-grained semantic levels.

We introduce a new task, called Grounded CoMPaT Recognition (GCR), to collectively recognize and ground compositions of materials on parts of 3D objects. We present two variations of this task and adapt state-of-art 2D/3D deep learning methods to solve the problem as baselines for future research. We hope our work will help ease future research on compositional 3D Vision.

⚡ Pipeline

We begin with 3D shape collection, perform editing, gather material compatibility data, annotate shapes at a fine-grained level, realign misaligned shapes using part annotations, sample materials for each part, and render stylized shapes from various angles.

🧩 GCR Task

Grounded CoMPaT Recognition (GCR). Given an input shape, here: a chair, the task consists of (a) recognizing the shape category and (b) segmenting the part-material pairs composing it.

🔎 Dataset

Move the slider to explore the dataset. These samples are illustrative examples, offering insight into the dataset's extensive content, which encompasses 16 million rendered views of over 10 million stylized 3D shapes.

RGB image
Part map
Material map
Depth map
RGB pointcloud
Part labels
Material labels

We show below andomly sampled styles for a set of six geometries from the dataset. Stylized shapes are sampled with a random style parameterized by a set of compatible materials.

📖 Citations

If our work is useful to your research, please consider citing our work by referencing the following research papers for the 3DCoMPaT and 3DCoMPaT++ datasets:


@inproceedings {slim_3dcompatplus_2023,
    title={{3DCoMPaT++}: An improved Large-scale 3D Vision Dataset
    for Compositional Recognition},
    author={ Habib Slim and Xiang Li and Yuchen Li,
    Mahmoud Ahmed and Mohamed Ayman and Ujjwal Upadhyay
    Ahmed Abdelreheem and Arpit Prajapati,
    Suhail Pothigara and Peter Wonka and Mohamed Elhoseiny },

3DCoMPaT v1:

@inproceedings {li20223d_compat,
    title={{3DCoMPaT}: Composition of Materials on Parts of 3D Things},
    author={Yuchen Li, Ujjwal Upadhyay, Habib Slim,
    Ahmed Abdelreheem, Arpit Prajapati,
    Suhail Pothigara, Peter Wonka, Mohamed Elhoseiny},
    booktitle={17th European Conference on Computer Vision (ECCV)},