BREAKTHROUGH IN CONNECTIVITY: GOSSIP CONNECTION TIMEOUTS SLASHED FROM 30 SECONDS TO 2!
Revolutionary update promises to enhance hub performance and mitigate congestion, ensuring faster message validation across peer networks.
Commit Details:
fix: gossip connection timeout (#1747) ## Motivation - Hub nodes experienced significant delays from when a message was received and when it was validated in worker thread. - Large amounts of p2p connection attempts can cause backlog congestion as it may exhaust file descriptor or TCP connection limits - We observed large number of connection attempts that took 30 seconds, which seem to cause worker thread congestion ![image](https://github.com/farcasterxyz/hub-monorepo/assets/3589723/5d46383f-5c4e-485a-aedc-95c83b7db32f) ## Change Summary - Expose `LIBP2P_CONNECT_TIMEOUT_MS` environment variable to set dial timeout when connecting to peers - Set default connection timeout to `2 seconds` instead of `30 seconds` - Add statsd metrics for gossip worker method latency: `gossip.worker.${method}.latency_ms` - Fix small bug where signal handlers were initialized after hub started - this can cause ungraceful termination and corruption issues on first sync ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. - [x] All [commits have been signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits) ## Additional Context If this is a relatively large or complex change, provide more details here that will help reviewers <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on improving libp2p connection timeouts and exposing environment variables for configuration. ### Detailed summary - Set dial timeout for gossip node connections - Expose `LIBP2P_CONNECT_TIMEOUT_MS` environment variable - Update StatsD initialization in various files > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->