GitBulletin

Repository: farcasterxyz/hub-monorepo

Revolutionary Update Boosts Snapshot Sync Efficiency

2024-08-06 05:55:40 UTC

New features promise to cut download times in half, enhancing node reliability and performance in the hub ecosystem!

Commit Details:

feat: make snapshot sync retry and parallelize (#2236) ## Why is this change needed? Snapshot sync has become a very painful part of running a hub, to the point where it may take several attempts to start the node to get the snapshot to successfully download, on top of taking a long time to download all the chunks of the snapshot. This PR adds durability and parallelizes the download process (up to four simultaneous fetches, which appears to be the point where exceeding will become more unstable on fetch success). Snapshot sync now takes roughly 25-30 minutes to download. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR introduces retry and parallelization for snapshot synchronization in the `@farcaster/hubble` package. ### Detailed summary - Added retry mechanism for decompressing snapshot chunks - Implemented parallel downloading of snapshot chunks - Improved error handling during snapshot synchronization > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->

Search Similar