Skip to main navigation Skip to search Skip to main content

Learning and data selection in big datasets

  • Hossein S. Ghadikolaei
  • , Hadi Ghauch
  • , Carlo Fischione
  • , Mikael Skoglund
  • KTH Royal Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

10 Citations (Scopus)

Abstract

Finding a dataset of minimal cardinality to characterize the optimal parameters of a model is of paramount importance in machine learning and distributed optimization over a network. This paper investigates the compressibility of large datasets. More specifically, we propose a framework that jointly learns the input-output mapping as well as the most representative samples of the dataset (sufficient dataset). Our analytical results show that the cardinality of the sufficient dataset increases sub-linearly with respect to the original dataset size. Numerical evaluations of real datasets reveal a large compressibility, up to 95%, without a noticeable drop in the learnability performance, measured by the generalization error.

Original languageEnglish
Title of host publication36th International Conference on Machine Learning, ICML 2019
PublisherInternational Machine Learning Society (IMLS)
Pages3848-3857
Number of pages10
ISBN (Electronic)9781510886988
Publication statusPublished - 1 Jan 2019
Event36th International Conference on Machine Learning, ICML 2019 - Long Beach, United States
Duration: 9 Jun 2019 → 15 Jun 2019

Publication series

Name36th International Conference on Machine Learning, ICML 2019
Volume2019-June

Conference

Conference36th International Conference on Machine Learning, ICML 2019
Country/TerritoryUnited States
CityLong Beach
Period9/06/19 → 15/06/19

Fingerprint

Dive into the research topics of 'Learning and data selection in big datasets'. Together they form a unique fingerprint.

Cite this