Bandwidth Breakthrough: Hubble Hub Scales Back to Save Data

New toggles introduced to curtail excessive bandwidth usage while enhancing network efficiency and scalability.

Commit Details:

fix(hubble): reduce hub bandwidth with floodsub toggle (#1851) ## Motivation - Hub bandwidth usage has increased to around 400-500 KiB per second, or 40-50 GiB per day - Needless bandwidth utilization can increase resource requirements to run Hub ## Context - The default settings for our libp2p library publish to floodsub and subscribe to floodsub topic - Floodsub is a p2p protocol where every message a node receives gets forwarded to all known peers in the network - While there are benefits to Floodsub, it has significant network amplification factors and hits scalability limits as the number of hubs in the network grows ## Change Summary - By default, disable `floodPublish` and `fallbackToFloodsub` - Expose environment variables `GOSSIPSUB_FALLBACK_TO_FLOODSUB` and `GOSSIPSUB_FLOOD_PUBLISH` to toggle the values for gossipsub p2p ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. - [x] All [commits have been signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits) ## Additional Context If this is a relatively large or complex change, provide more details here that will help reviewers <!-- start pr-codex --> --- ## PR-Codex overview This PR reduces hub bandwidth in the `hubble` app by adding toggles for `GOSSIPSUB_FALLBACK_TO_FLOODSUB` and `GOSSIPSUB_FLOOD_PUBLISH`. ### Detailed summary - Added toggles for `GOSSIPSUB_FALLBACK_TO_FLOODSUB` and `GOSSIPSUB_FLOOD_PUBLISH` in `gossipNode.ts` - Refactored callback handling in `server.ts` - Added environment variable checks for `fallbackToFloodsub` and `floodPublish` in `gossipNodeWorker.ts` - Added stats tracking for message sizes in `gossipNodeWorker.ts` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->