GitBulletin

Repository: farcasterxyz/hub-monorepo

Memory Crisis Averted: Sync Health Job Refined for Efficiency

2024-10-01 22:57:21 UTC

Innovative 10-Minute Chunks Revolutionize Message Processing, Alleviating Strain on Hubs!

Commit Details:

fix: Split sync health job into 10 min chunks to limit memory usage (#2345) ## Why is this change needed? Hubs might be running into memory pressure because divergingSyncIds is examining too many messages at once. Chunk sync health job to 10 minute intervals to reduce memory usage. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on optimizing the `syncHealthJob` by splitting the processing of sync health messages into 10-minute intervals to reduce memory usage. ### Detailed summary - Introduced a loop to process `syncHealthMessageStats` in 10-minute chunks. - Adjusted the handling of `syncHealthMessageStats` and error logging. - Updated the process for pushing diverging sync IDs and logging results. - Improved logging details for computed sync health stats. > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->

Search Similar