Revolutionizing Throughput: New Features Transform HubSubscriber Efficiency!
Major update enhances event streaming capabilities with customizable batch sizes and flush intervals to maximize performance and reduce Redis requests!
Commit Details:
feat(shuttle): Allow changing subscriber batch size or flush interval (#2147) ## Why is this change needed? The way we were writing events to the stream was resulting in a lot more separate requests to Redis since we weren't actually batching them together. Change this so we leverage the multi-argument version of `XADD` so that throughput can be increased at higher volumes. Accompanying this are the introduction of a few more configuration options that allow us to tweak the throughput of the `HubSubscriber`. While here, do a minor version bump since we're slightly changing how events are written to the stream. ## Merge Checklist - [x] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [x] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [x] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [x] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. <!-- start pr-codex --> --- ## PR-Codex overview This PR updates the `@farcaster/shuttle` package to version 0.5.0, introducing customization for event batch size and time between flushes. ### Detailed summary - Updated package version to 0.5.0 - Added customization for event batch size and time between flushes - Modified event handling logic in `EventStreamHubSubscriber` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex -->