Revolutionary Snapshot Overhaul: Data Storage Revolutionized!

New 4GB Chunk System Streamlines Memory Management and Enhances Efficiency in the Hub Application.

Commit Details:

fix: Split the snapshot into 4GB chunks (#1933) ## Motivation To avoid the snapshot .tar.gz from becoming too big, we split it into 4GB chunks ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [X] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [ ] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [ ] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. - [X] All [commits have been signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits) <!-- start pr-codex --> --- ## PR-Codex overview ### Focus: Implement snapshot chunking, improve snapshot download logging, and enhance error handling in the Hub application. ### Detailed summary: - Split snapshot into 4GB chunks for efficient storage - Update snapshot download log message - Implement chunked snapshot writing for backups - Improve error handling during snapshot upload - Modify snapshot metadata key to `keyBase` > The following files were skipped due to too many changes: `apps/hubble/src/utils/snapshot.ts`, `apps/hubble/src/hubble.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->