A Storage-computation-communication tradeoff for distributed computing

Qifa Yan; Sheng Yang; Michele Wigger

doi:10.1109/ISWCS.2018.8491052

A Storage-computation-communication tradeoff for distributed computing

Qifa Yan, Sheng Yang, Michele Wigger

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper investigates distributed computing systems where computations are split into "Map" and "Reduce" functions. A new coded scheme, called distributed computing and coded communication (D3C), is proposed, and its communication load is analyzed as a function of the available storage space and the number of intermediate values (IVA) to be computed. D3C achieves the smallest possible communication load for a given storage space, while a smaller number of IVAs need to be computed compared to Li et al.'s coded distributed computing (CDC) scheme. More generally, our scheme can flexibly trade between storage space and the number of IVAs to be computed. Communication load is then analyzed for any given tradeoff.

Original language	English
Title of host publication	2018 15th International Symposium on Wireless Communication Systems, ISWCS 2018
Publisher	VDE Verlag GmbH
ISBN (Electronic)	9781538650059
DOIs	https://doi.org/10.1109/ISWCS.2018.8491052
Publication status	Published - 12 Oct 2018
Externally published	Yes
Event	15th International Symposium on Wireless Communication Systems, ISWCS 2018 - Lisbon, Portugal Duration: 28 Aug 2018 → 31 Aug 2018

Publication series

Name	Proceedings of the International Symposium on Wireless Communication Systems
Volume	2018-August
ISSN (Print)	2154-0217
ISSN (Electronic)	2154-0225

Conference

Conference	15th International Symposium on Wireless Communication Systems, ISWCS 2018
Country/Territory	Portugal
City	Lisbon
Period	28/08/18 → 31/08/18

Access to Document

10.1109/ISWCS.2018.8491052

Cite this

@inproceedings{568f7d12ea48444a8a36e112c0ffb206,

title = "A Storage-computation-communication tradeoff for distributed computing",

abstract = "This paper investigates distributed computing systems where computations are split into {"}Map{"} and {"}Reduce{"} functions. A new coded scheme, called distributed computing and coded communication (D3C), is proposed, and its communication load is analyzed as a function of the available storage space and the number of intermediate values (IVA) to be computed. D3C achieves the smallest possible communication load for a given storage space, while a smaller number of IVAs need to be computed compared to Li et al.'s coded distributed computing (CDC) scheme. More generally, our scheme can flexibly trade between storage space and the number of IVAs to be computed. Communication load is then analyzed for any given tradeoff.",

author = "Qifa Yan and Sheng Yang and Michele Wigger",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 15th International Symposium on Wireless Communication Systems, ISWCS 2018 ; Conference date: 28-08-2018 Through 31-08-2018",

year = "2018",

month = oct,

day = "12",

doi = "10.1109/ISWCS.2018.8491052",

language = "English",

series = "Proceedings of the International Symposium on Wireless Communication Systems",

publisher = "VDE Verlag GmbH",

booktitle = "2018 15th International Symposium on Wireless Communication Systems, ISWCS 2018",

}

Yan, Q, Yang, S & Wigger, M 2018, A Storage-computation-communication tradeoff for distributed computing. in 2018 15th International Symposium on Wireless Communication Systems, ISWCS 2018., 8491052, Proceedings of the International Symposium on Wireless Communication Systems, vol. 2018-August, VDE Verlag GmbH, 15th International Symposium on Wireless Communication Systems, ISWCS 2018, Lisbon, Portugal, 28/08/18. https://doi.org/10.1109/ISWCS.2018.8491052

A Storage-computation-communication tradeoff for distributed computing. / Yan, Qifa; Yang, Sheng; Wigger, Michele.
2018 15th International Symposium on Wireless Communication Systems, ISWCS 2018. VDE Verlag GmbH, 2018. 8491052 (Proceedings of the International Symposium on Wireless Communication Systems; Vol. 2018-August).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Storage-computation-communication tradeoff for distributed computing

AU - Yan, Qifa

AU - Yang, Sheng

AU - Wigger, Michele

PY - 2018/10/12

Y1 - 2018/10/12

N2 - This paper investigates distributed computing systems where computations are split into "Map" and "Reduce" functions. A new coded scheme, called distributed computing and coded communication (D3C), is proposed, and its communication load is analyzed as a function of the available storage space and the number of intermediate values (IVA) to be computed. D3C achieves the smallest possible communication load for a given storage space, while a smaller number of IVAs need to be computed compared to Li et al.'s coded distributed computing (CDC) scheme. More generally, our scheme can flexibly trade between storage space and the number of IVAs to be computed. Communication load is then analyzed for any given tradeoff.

AB - This paper investigates distributed computing systems where computations are split into "Map" and "Reduce" functions. A new coded scheme, called distributed computing and coded communication (D3C), is proposed, and its communication load is analyzed as a function of the available storage space and the number of intermediate values (IVA) to be computed. D3C achieves the smallest possible communication load for a given storage space, while a smaller number of IVAs need to be computed compared to Li et al.'s coded distributed computing (CDC) scheme. More generally, our scheme can flexibly trade between storage space and the number of IVAs to be computed. Communication load is then analyzed for any given tradeoff.

U2 - 10.1109/ISWCS.2018.8491052

DO - 10.1109/ISWCS.2018.8491052

M3 - Conference contribution

AN - SCOPUS:85056725958

T3 - Proceedings of the International Symposium on Wireless Communication Systems

BT - 2018 15th International Symposium on Wireless Communication Systems, ISWCS 2018

PB - VDE Verlag GmbH

T2 - 15th International Symposium on Wireless Communication Systems, ISWCS 2018

Y2 - 28 August 2018 through 31 August 2018

ER -

A Storage-computation-communication tradeoff for distributed computing

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this