Attention-based fusion for multi-source human image generation

Stephane Lathuiliere; Enver Sangineto; Aliaksandr Siarohin; Nicu Sebe

doi:10.1109/WACV45572.2020.9093602

Attention-based fusion for multi-source human image generation

Stephane Lathuiliere
, Enver Sangineto
, Aliaksandr Siarohin
, Nicu Sebe

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We present a generalization of the person-image generation task, in which a human image is generated conditioned on a target pose and a set X of source appearance images. In this way, we can exploit multiple, possibly complementary images of the same person which are usually available at training and at testing time. The solution we propose is mainly based on a local attention mechanism which selects relevant information from different source image regions, avoiding the necessity to build specific generators for each specific cardinality of X. The empirical evaluation of our method shows the practical interest of addressing the person-image generation problem in a multi-source setting.

Original language	English
Title of host publication	Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	428-437
Number of pages	10
ISBN (Electronic)	9781728165530
DOIs	https://doi.org/10.1109/WACV45572.2020.9093602
Publication status	Published - 1 Mar 2020
Event	2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020 - Snowmass Village, United States Duration: 1 Mar 2020 → 5 Mar 2020

Publication series

Name	Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020

Conference

Conference	2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020
Country/Territory	United States
City	Snowmass Village
Period	1/03/20 → 5/03/20

Access to Document

10.1109/WACV45572.2020.9093602

Cite this

Lathuiliere, S., Sangineto, E., Siarohin, A., & Sebe, N. (2020). Attention-based fusion for multi-source human image generation. In Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020 (pp. 428-437). Article 9093602 (Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV45572.2020.9093602

Lathuiliere, Stephane ; Sangineto, Enver ; Siarohin, Aliaksandr et al. / Attention-based fusion for multi-source human image generation. Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020. Institute of Electrical and Electronics Engineers Inc., 2020. pp. 428-437 (Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020).

@inproceedings{9ca00b8cfbe34963a9136d82805359e8,

title = "Attention-based fusion for multi-source human image generation",

abstract = "We present a generalization of the person-image generation task, in which a human image is generated conditioned on a target pose and a set X of source appearance images. In this way, we can exploit multiple, possibly complementary images of the same person which are usually available at training and at testing time. The solution we propose is mainly based on a local attention mechanism which selects relevant information from different source image regions, avoiding the necessity to build specific generators for each specific cardinality of X. The empirical evaluation of our method shows the practical interest of addressing the person-image generation problem in a multi-source setting.",

author = "Stephane Lathuiliere and Enver Sangineto and Aliaksandr Siarohin and Nicu Sebe",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020 ; Conference date: 01-03-2020 Through 05-03-2020",

year = "2020",

month = mar,

day = "1",

doi = "10.1109/WACV45572.2020.9093602",

language = "English",

series = "Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "428--437",

booktitle = "Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020",

}

Lathuiliere, S, Sangineto, E, Siarohin, A & Sebe, N 2020, Attention-based fusion for multi-source human image generation. in Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020., 9093602, Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020, Institute of Electrical and Electronics Engineers Inc., pp. 428-437, 2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020, Snowmass Village, United States, 1/03/20. https://doi.org/10.1109/WACV45572.2020.9093602

Attention-based fusion for multi-source human image generation. / Lathuiliere, Stephane; Sangineto, Enver; Siarohin, Aliaksandr et al.
Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020. Institute of Electrical and Electronics Engineers Inc., 2020. p. 428-437 9093602 (Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Attention-based fusion for multi-source human image generation

AU - Lathuiliere, Stephane

AU - Sangineto, Enver

AU - Siarohin, Aliaksandr

AU - Sebe, Nicu

PY - 2020/3/1

Y1 - 2020/3/1

N2 - We present a generalization of the person-image generation task, in which a human image is generated conditioned on a target pose and a set X of source appearance images. In this way, we can exploit multiple, possibly complementary images of the same person which are usually available at training and at testing time. The solution we propose is mainly based on a local attention mechanism which selects relevant information from different source image regions, avoiding the necessity to build specific generators for each specific cardinality of X. The empirical evaluation of our method shows the practical interest of addressing the person-image generation problem in a multi-source setting.

AB - We present a generalization of the person-image generation task, in which a human image is generated conditioned on a target pose and a set X of source appearance images. In this way, we can exploit multiple, possibly complementary images of the same person which are usually available at training and at testing time. The solution we propose is mainly based on a local attention mechanism which selects relevant information from different source image regions, avoiding the necessity to build specific generators for each specific cardinality of X. The empirical evaluation of our method shows the practical interest of addressing the person-image generation problem in a multi-source setting.

UR - https://www.scopus.com/pages/publications/85085514032

U2 - 10.1109/WACV45572.2020.9093602

DO - 10.1109/WACV45572.2020.9093602

M3 - Conference contribution

AN - SCOPUS:85085514032

T3 - Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020

SP - 428

EP - 437

BT - Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020

Y2 - 1 March 2020 through 5 March 2020

ER -

Lathuiliere S, Sangineto E, Siarohin A, Sebe N. Attention-based fusion for multi-source human image generation. In Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020. Institute of Electrical and Electronics Engineers Inc. 2020. p. 428-437. 9093602. (Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020). doi: 10.1109/WACV45572.2020.9093602

Attention-based fusion for multi-source human image generation

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this