GitBulletin

Repository: farcasterxyz/hub-monorepo

**Database Overhaul: Prune Job Gets a Performance Throttle!**

2024-03-19 21:40:47 UTC

*New throttling measures implemented to prevent latency spikes and enhance system efficiency for seamless operation.*

Commit Details:

perf: Throttle prune job (#1827) ## Motivation The prune job is running too fast, causing spikes in merge latency. This PR throttles the job ## Change Summary // We prune at the rate of 1000 fids per second. If we are running ahead of schedule, we sleep to catch up // We do this so that: // 1. Each fid is pruned at the same time at every hub, reducing thrash // 2. Don't overload the DB or Sync Trie, causing spikes in latency ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [X] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [X] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [X] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. - [X] All [commits have been signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits) <!-- start pr-codex --> --- ## PR-Codex overview The focus of this PR is to improve the performance of the prune job in the `@farcaster/hubble` module. ### Detailed summary - Throttled the prune job to avoid overloading the system - Removed unnecessary parameter in `PruneMessagesJobScheduler` constructor - Added logic to sleep and catch up if pruning is ahead of schedule > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->

Search Similar