Revolutionizing Hub Sync: New S3 Snapshot Metadata Update Unleashes Power of Database Statistics!

Groundbreaking enhancements unveil the `snapshot-url` command, boosting efficiency and scaling capabilities like never before!

Commit Details:

feat(hubble): update s3 snapshot metadata to include database statistics (#1829) ## Motivation - As part of improvements to scaling Hub sync, we want to use snapshots more often - Determining whether to use s3 snapshot for catch up sync requires additional information in metadata JSON ## Change Summary - Update s3 snapshot metadata to include database statistics - Add `snapshot-url` command to print latest snapshot URL for convenience - Within `apps/hubble` run `yarn snapshot-url` or `node build/cli.js snapshot-url` ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [x] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. - [x] All [commits have been signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits) ## Additional Context Confirmed upload and download working ``` {"level":30,"time":1710985820939,"pid":5242,"hostname":"mbp.local","component":"Hub","progress":{"loaded":10485760000,"total":24114175697,"part":10,"Key":"snapshots/MAINNET/DB_SCHEMA_9/snapshot-2024-03-21-1710983283.tar.gz","Bucket":"test-bucket"},"msg":"Uploading snapshot to S3"} {"level":30,"time":1710985925021,"pid":5242,"hostname":"mbp.local","component":"Hub","progress":{"loaded":11534336000,"total":24114175697,"part":11,"Key":"snapshots/MAINNET/DB_SCHEMA_9/snapshot-2024-03-21-1710983283.tar.gz","Bucket":"test-bucket"},"msg":"Uploading snapshot to S3"} {"level":30,"time":1710986006718,"pid":5242,"hostname":"mbp.local","component":"Hub","progress":{"loaded":12582912000,"total":24114175697,"part":12,"Key":"snapshots/MAINNET/DB_SCHEMA_9/snapshot-2024-03-21-1710983283.tar.gz","Bucket":"test-bucket"},"msg":"Uploading snapshot to S3"} ``` <!-- start pr-codex --> --- ## PR-Codex overview This PR enhances the `hubble` app by adding a `snapshot-url` command to retrieve S3 snapshot metadata and statistics. It also includes improvements for snapshot handling and network synchronization. ### Detailed summary - Added `snapshot-url` command to retrieve S3 snapshot metadata - Updated snapshot handling logic with metadata and statistics - Improved network synchronization with Merkle Trie enhancements > The following files were skipped due to too many changes: `apps/hubble/src/utils/snapshot.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->