Passer à la navigation principale Passer à la recherche Passer au contenu principal

Zero-Shot 3D Shape Correspondence

  • King Abdullah University of Science and Technology
  • École Polytechnique

Résultats de recherche: Le chapitre dans un livre, un rapport, une anthologie ou une collection › Contribution à une conférence › Revue par des pairs

27 Citations (Scopus)

Résumé

We propose a novel zero-shot approach to computing correspondences between 3D shapes. Existing approaches mainly focus on isometric and near-isometric shape pairs (e.g., human vs. human), but less attention has been given to strongly non-isometric and inter-class shape matching (e.g., human vs. cow). To this end, we introduce a fully automatic method that exploits the exceptional reasoning capabilities of recent foundation models in language and vision to tackle difficult shape correspondence problems. Our approach comprises multiple stages. First, we classify the 3D shapes in a zero-shot manner by feeding rendered shape views to a language-vision model (e.g., BLIP2) to generate a list of class proposals per shape. These proposals are unified into a single class per shape by employing the reasoning capabilities of ChatGPT. Second, we attempt to segment the two shapes in a zero-shot manner, but in contrast to the co-segmentation problem, we do not require a mutual set of semantic regions. Instead, we propose to exploit the in-context learning capabilities of ChatGPT to generate two different sets of semantic regions for each shape and a semantic mapping between them. This enables our approach to match strongly non-isometric shapes with significant differences in geometric structure. Finally, we employ the generated semantic mapping to produce coarse correspondences that can further be refined by the functional maps framework to produce dense point-to-point maps. Our approach, despite its simplicity, produces highly plausible results in a zero-shot manner, especially between strongly non-isometric shapes.

langue originaleAnglais
titreProceedings - SIGGRAPH Asia 2023 Conference Papers, SA 2023
rédacteurs en chefStephen N. Spencer
EditeurAssociation for Computing Machinery, Inc
ISBN (Electronique)9798400703157
Les DOIs
étatPublié - 10 déc. 2023
Modification externeOui
Evénement2023 SIGGRAPH Asia 2023 Conference Papers, SA 2023 - Sydney, Australie
Durée: 12 déc. 2023 → 15 déc. 2023

Série de publications

NomProceedings - SIGGRAPH Asia 2023 Conference Papers, SA 2023

Une conférence

Une conférence2023 SIGGRAPH Asia 2023 Conference Papers, SA 2023
Pays/TerritoireAustralie
La villeSydney
période12/12/23 → 15/12/23

Empreinte digitale

Examiner les sujets de recherche de « Zero-Shot 3D Shape Correspondence ». Ensemble, ils forment une empreinte digitale unique.

Contient cette citation