Tech Overhaul: Throttling Tactics to Revolutionize Storage Cache Performance!
New fix enhances efficiency by slowing down pre-population scans, paving the way for smarter data management and improved system responsiveness.
Commit Details:
fix: Throttle storageCache population (#1864) ## Motivation When a hub starts up, it attempts to pre-populate the storage cache. This is slow, since it has to basically scan every message. ## Change Summary - Slow down the pre-population - Don't do too many scans in parallel. ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [X] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [X] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [X] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [ ] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. - [X] All [commits have been signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits) ## Additional Context If this is a relatively large or complex change, provide more details here that will help reviewers <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on fixing storage cache prepopulation throttling and adding memory usage logging with optional heap dumps. ### Detailed summary - Fixed throttle issue in storage cache prepopulation - Added memory usage logging in LibP2PNode and Hub - Implemented optional heap dumps for memory analysis > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->