GitBulletin

Repository: farcasterxyz/hub-monorepo

**Revolutionary Snapshot Speed Boost: Keats Shatters 15-Minute Startup Barrier!**

2024-03-06 18:19:33 UTC

**Tech Wizards Unleash Multi-Threaded Gzip Magic, Transforming Snapshot Creation from Tedious to Turbocharged!**

Commit Details:

fix: When creating a snapshot, tar file first to speedup (#1785) ## Motivation `keats` is taking > 15mins to startup because it takes so long to create a snapshot. Fix this by: - Creating just the tar file first (no gzip), which is faster - gzip while uploading using multi-threaded gzp lib in rust ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [X] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [X] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [X] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. - [X] All [commits have been signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits) <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on improving snapshotting speed by tar-ing files before gzip. ### Detailed summary - Added `gzp` dependency for faster snapshotting - Created `rsCreateTarGzip` function for tar-gzip operation - Updated Dockerfile dependencies for compilation - Updated CI workflow to run on an ARM architecture - Modified file paths and exports in `hubble.ts` - Implemented `uploadToS3` method to upload tar.gz files - Added `dbCreateTarGzip` function for creating tar.gz files - Updated `RocksDB` methods for tarball creation - Added `create_tar_gzip` and `js_create_tar_gzip` functions in `rocksdb.rs` - Updated dependencies in `Cargo.lock` > The following files were skipped due to too many changes: `apps/hubble/src/addon/Cargo.lock` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->

Search Similar