Enhancing the accessibility of hearing impaired to video content through fully automatic dynamic captioning

Bogdan Mocanu; Ruxandra Tapu; Titus Zaharia

doi:10.1109/EHB47216.2019.8970038

Enhancing the accessibility of hearing impaired to video content through fully automatic dynamic captioning

Bogdan Mocanu
, Ruxandra Tapu
, Titus Zaharia

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In this paper we introduce an automatic subtitle positioning approach designed to enhance the video accessibility of deaf and hearing impaired people to multimedia documents. By using a dynamic subtitle and captioning approach, which exploits various computer vision techniques, including face detection, tracking and recognition, video temporal segmentation into shots and scenes and active speaker recognition, we are able to position each video subtitle segment in the near vicinity of the active speaker. The experimental evaluation performed on 30 video elements validates our approach with average F1-scores superior to 92%.

Original language	English
Title of host publication	2019 7th E-Health and Bioengineering Conference, EHB 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728126036
DOIs	https://doi.org/10.1109/EHB47216.2019.8970038
Publication status	Published - 1 Nov 2019
Externally published	Yes
Event	7th IEEE International Conference on E-Health and Bioengineering, EHB 2019 - Iasi, Romania Duration: 21 Nov 2019 → 23 Nov 2019

Publication series

Name	2019 7th E-Health and Bioengineering Conference, EHB 2019

Conference

Conference	7th IEEE International Conference on E-Health and Bioengineering, EHB 2019
Country/Territory	Romania
City	Iasi
Period	21/11/19 → 23/11/19

Keywords

Active speaker recognition
Deaf and hearing impaired people
Dynamic subtitle

Access to Document

10.1109/EHB47216.2019.8970038

Cite this

Mocanu, B., Tapu, R., & Zaharia, T. (2019). Enhancing the accessibility of hearing impaired to video content through fully automatic dynamic captioning. In 2019 7th E-Health and Bioengineering Conference, EHB 2019 Article 8970038 (2019 7th E-Health and Bioengineering Conference, EHB 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/EHB47216.2019.8970038

@inproceedings{64adea46c2184a9094ca2bdecebcfc44,

title = "Enhancing the accessibility of hearing impaired to video content through fully automatic dynamic captioning",

abstract = "In this paper we introduce an automatic subtitle positioning approach designed to enhance the video accessibility of deaf and hearing impaired people to multimedia documents. By using a dynamic subtitle and captioning approach, which exploits various computer vision techniques, including face detection, tracking and recognition, video temporal segmentation into shots and scenes and active speaker recognition, we are able to position each video subtitle segment in the near vicinity of the active speaker. The experimental evaluation performed on 30 video elements validates our approach with average F1-scores superior to 92\%.",

keywords = "Active speaker recognition, Deaf and hearing impaired people, Dynamic subtitle",

author = "Bogdan Mocanu and Ruxandra Tapu and Titus Zaharia",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 7th IEEE International Conference on E-Health and Bioengineering, EHB 2019 ; Conference date: 21-11-2019 Through 23-11-2019",

year = "2019",

month = nov,

day = "1",

doi = "10.1109/EHB47216.2019.8970038",

language = "English",

series = "2019 7th E-Health and Bioengineering Conference, EHB 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2019 7th E-Health and Bioengineering Conference, EHB 2019",

}

Mocanu, B, Tapu, R & Zaharia, T 2019, Enhancing the accessibility of hearing impaired to video content through fully automatic dynamic captioning. in 2019 7th E-Health and Bioengineering Conference, EHB 2019., 8970038, 2019 7th E-Health and Bioengineering Conference, EHB 2019, Institute of Electrical and Electronics Engineers Inc., 7th IEEE International Conference on E-Health and Bioengineering, EHB 2019, Iasi, Romania, 21/11/19. https://doi.org/10.1109/EHB47216.2019.8970038

Enhancing the accessibility of hearing impaired to video content through fully automatic dynamic captioning. / Mocanu, Bogdan; Tapu, Ruxandra ; Zaharia, Titus.
2019 7th E-Health and Bioengineering Conference, EHB 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8970038 (2019 7th E-Health and Bioengineering Conference, EHB 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Enhancing the accessibility of hearing impaired to video content through fully automatic dynamic captioning

AU - Mocanu, Bogdan

AU - Tapu, Ruxandra

AU - Zaharia, Titus

PY - 2019/11/1

Y1 - 2019/11/1

N2 - In this paper we introduce an automatic subtitle positioning approach designed to enhance the video accessibility of deaf and hearing impaired people to multimedia documents. By using a dynamic subtitle and captioning approach, which exploits various computer vision techniques, including face detection, tracking and recognition, video temporal segmentation into shots and scenes and active speaker recognition, we are able to position each video subtitle segment in the near vicinity of the active speaker. The experimental evaluation performed on 30 video elements validates our approach with average F1-scores superior to 92%.

AB - In this paper we introduce an automatic subtitle positioning approach designed to enhance the video accessibility of deaf and hearing impaired people to multimedia documents. By using a dynamic subtitle and captioning approach, which exploits various computer vision techniques, including face detection, tracking and recognition, video temporal segmentation into shots and scenes and active speaker recognition, we are able to position each video subtitle segment in the near vicinity of the active speaker. The experimental evaluation performed on 30 video elements validates our approach with average F1-scores superior to 92%.

KW - Active speaker recognition

KW - Deaf and hearing impaired people

KW - Dynamic subtitle

UR - https://www.scopus.com/pages/publications/85079356501

U2 - 10.1109/EHB47216.2019.8970038

DO - 10.1109/EHB47216.2019.8970038

M3 - Conference contribution

AN - SCOPUS:85079356501

T3 - 2019 7th E-Health and Bioengineering Conference, EHB 2019

BT - 2019 7th E-Health and Bioengineering Conference, EHB 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 7th IEEE International Conference on E-Health and Bioengineering, EHB 2019

Y2 - 21 November 2019 through 23 November 2019

ER -

Enhancing the accessibility of hearing impaired to video content through fully automatic dynamic captioning

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this