WebGraph: The Next Generation (Is in Rust)

Tommaso Fontana, Sebastiano Vigna, Stefano Zacchiroli

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We report the results of a yearlong effort to port the WebGraph framework [4] from Java to Rust. For two decades WebGraph has been instrumental in the analysis and distribution of large graphs for the research community of TheWebConf, but the intrinsic limitations of the Java Virtual Machine had become a bottleneck for very large use cases, such as the Software Heritage Merkle graph [2] with its half a trillion arcs. As part of this clean-slate implementation of WebGraph in Rust, we developed a few ancillary projects bringing to the Rust ecosystem some missing features of independent interest, such as easy, consistent and zero-cost memory mapping of data structures. WebGraph in Rust offers impressive performance improvements over the previous implementation, enabling open-source graph analytics on very large datasets on top of a modern systems programming language.

Original languageEnglish
Title of host publicationWWW 2024 Companion - Companion Proceedings of the ACM Web Conference
PublisherAssociation for Computing Machinery, Inc
Pages686-689
Number of pages4
ISBN (Electronic)9798400701726
DOIs
Publication statusPublished - 13 May 2024
Event33rd Companion of the ACM World Wide Web Conference, WWW 2023 - Singapore, Singapore
Duration: 13 May 202417 May 2024

Publication series

NameWWW 2024 Companion - Companion Proceedings of the ACM Web Conference

Conference

Conference33rd Companion of the ACM World Wide Web Conference, WWW 2023
Country/TerritorySingapore
CitySingapore
Period13/05/2417/05/24

Keywords

  • Java
  • Rust
  • big data
  • compression
  • graphs
  • social networks
  • web graphs

Fingerprint

Dive into the research topics of 'WebGraph: The Next Generation (Is in Rust)'. Together they form a unique fingerprint.

Cite this