Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds

Bjorn Michele; Alexandre Boulch; Gilles Puy; Maxime Bucher; Renaud Marlet

doi:10.1109/3DV53792.2021.00107

Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds

Bjorn Michele
, Alexandre Boulch
, Gilles Puy
, Maxime Bucher
, Renaud Marlet

Valeo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

While there has been a number of studies on Zero-Shot Learning (ZSL) for 2D images,its application to 3D data is still recent and scarce,with just a few methods limited to classification. We present the first generative approach for both ZSL and Generalized ZSL (GZSL) on 3D data,that can handle both classification and,for the first time,semantic segmentation. We show that it reaches or outperforms the state of the art on ModelNet40 classification for both inductive ZSL and inductive GZSL. For semantic segmentation,we created three benchmarks for evaluating this new ZSL task,using S3DIS,ScanNet and SemanticKITTI. Our experiments show that our method outperforms strong baselines,which we additionally propose for this task.

Original language	English
Title of host publication	Proceedings - 2021 International Conference on 3D Vision, 3DV 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	992-1002
Number of pages	11
ISBN (Electronic)	9781665426886
DOIs	https://doi.org/10.1109/3DV53792.2021.00107
Publication status	Published - 1 Jan 2021
Event	9th International Conference on 3D Vision, 3DV 2021 - Virtual, Online, United Kingdom Duration: 1 Dec 2021 → 3 Dec 2021

Publication series

Name	Proceedings - 2021 International Conference on 3D Vision, 3DV 2021

Conference

Conference	9th International Conference on 3D Vision, 3DV 2021
Country/Territory	United Kingdom
City	Virtual, Online
Period	1/12/21 → 3/12/21

Keywords

3D data
Frugal learning
GZSL
Generalized Zero Shot learning
Point cloud
Semantic segmentation
Transfer learning
ZSS
Zero Shot Semantic Segmentation
Zero-Shot learning

Access to Document

10.1109/3DV53792.2021.00107

Cite this

Michele, B., Boulch, A., Puy, G., Bucher, M., & Marlet, R. (2021). Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds. In Proceedings - 2021 International Conference on 3D Vision, 3DV 2021 (pp. 992-1002). (Proceedings - 2021 International Conference on 3D Vision, 3DV 2021). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/3DV53792.2021.00107

@inproceedings{c47a7f716e2a4cde99fc11ccb55995a7,

title = "Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds",

abstract = "While there has been a number of studies on Zero-Shot Learning (ZSL) for 2D images,its application to 3D data is still recent and scarce,with just a few methods limited to classification. We present the first generative approach for both ZSL and Generalized ZSL (GZSL) on 3D data,that can handle both classification and,for the first time,semantic segmentation. We show that it reaches or outperforms the state of the art on ModelNet40 classification for both inductive ZSL and inductive GZSL. For semantic segmentation,we created three benchmarks for evaluating this new ZSL task,using S3DIS,ScanNet and SemanticKITTI. Our experiments show that our method outperforms strong baselines,which we additionally propose for this task.",

keywords = "3D data, Frugal learning, GZSL, Generalized Zero Shot learning, Point cloud, Semantic segmentation, Transfer learning, ZSS, Zero Shot Semantic Segmentation, Zero-Shot learning",

author = "Bjorn Michele and Alexandre Boulch and Gilles Puy and Maxime Bucher and Renaud Marlet",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 9th International Conference on 3D Vision, 3DV 2021 ; Conference date: 01-12-2021 Through 03-12-2021",

year = "2021",

month = jan,

day = "1",

doi = "10.1109/3DV53792.2021.00107",

language = "English",

series = "Proceedings - 2021 International Conference on 3D Vision, 3DV 2021",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "992--1002",

booktitle = "Proceedings - 2021 International Conference on 3D Vision, 3DV 2021",

}

Michele, B, Boulch, A, Puy, G, Bucher, M & Marlet, R 2021, Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds. in Proceedings - 2021 International Conference on 3D Vision, 3DV 2021. Proceedings - 2021 International Conference on 3D Vision, 3DV 2021, Institute of Electrical and Electronics Engineers Inc., pp. 992-1002, 9th International Conference on 3D Vision, 3DV 2021, Virtual, Online, United Kingdom, 1/12/21. https://doi.org/10.1109/3DV53792.2021.00107

Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds. / Michele, Bjorn; Boulch, Alexandre; Puy, Gilles et al.
Proceedings - 2021 International Conference on 3D Vision, 3DV 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 992-1002 (Proceedings - 2021 International Conference on 3D Vision, 3DV 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds

AU - Michele, Bjorn

AU - Boulch, Alexandre

AU - Puy, Gilles

AU - Bucher, Maxime

AU - Marlet, Renaud

PY - 2021/1/1

Y1 - 2021/1/1

N2 - While there has been a number of studies on Zero-Shot Learning (ZSL) for 2D images,its application to 3D data is still recent and scarce,with just a few methods limited to classification. We present the first generative approach for both ZSL and Generalized ZSL (GZSL) on 3D data,that can handle both classification and,for the first time,semantic segmentation. We show that it reaches or outperforms the state of the art on ModelNet40 classification for both inductive ZSL and inductive GZSL. For semantic segmentation,we created three benchmarks for evaluating this new ZSL task,using S3DIS,ScanNet and SemanticKITTI. Our experiments show that our method outperforms strong baselines,which we additionally propose for this task.

AB - While there has been a number of studies on Zero-Shot Learning (ZSL) for 2D images,its application to 3D data is still recent and scarce,with just a few methods limited to classification. We present the first generative approach for both ZSL and Generalized ZSL (GZSL) on 3D data,that can handle both classification and,for the first time,semantic segmentation. We show that it reaches or outperforms the state of the art on ModelNet40 classification for both inductive ZSL and inductive GZSL. For semantic segmentation,we created three benchmarks for evaluating this new ZSL task,using S3DIS,ScanNet and SemanticKITTI. Our experiments show that our method outperforms strong baselines,which we additionally propose for this task.

KW - 3D data

KW - Frugal learning

KW - GZSL

KW - Generalized Zero Shot learning

KW - Point cloud

KW - Semantic segmentation

KW - Transfer learning

KW - ZSS

KW - Zero Shot Semantic Segmentation

KW - Zero-Shot learning

UR - https://www.scopus.com/pages/publications/85125018182

U2 - 10.1109/3DV53792.2021.00107

DO - 10.1109/3DV53792.2021.00107

M3 - Conference contribution

AN - SCOPUS:85125018182

T3 - Proceedings - 2021 International Conference on 3D Vision, 3DV 2021

SP - 992

EP - 1002

BT - Proceedings - 2021 International Conference on 3D Vision, 3DV 2021

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 9th International Conference on 3D Vision, 3DV 2021

Y2 - 1 December 2021 through 3 December 2021

ER -

Michele B, Boulch A, Puy G, Bucher M, Marlet R. Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds. In Proceedings - 2021 International Conference on 3D Vision, 3DV 2021. Institute of Electrical and Electronics Engineers Inc. 2021. p. 992-1002. (Proceedings - 2021 International Conference on 3D Vision, 3DV 2021). doi: 10.1109/3DV53792.2021.00107

Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this