GitBulletin

Repository: farcasterxyz/hub-monorepo

**Tech Titan Takes a Leap: Reactions Store and RocksDB Transformed by Rust Revolution!**

2024-03-04 18:20:40 UTC

**In a bold move to supercharge performance, developers relocate critical data stores to Rust, promising lightning-fast processing and crucial metrics enhancements!**

Commit Details:

Move Reactions Store and RocksDB to rust (#1748) ## Motivation Merging messages into the CRDT stores is slow, and to speed it up, we'll move the stores and the DB itself into rust. ## Change Summary - Move DB into Rust (instead of abstract-leveldown) - Move the reactions store into Rust - RocksDB is upgraded from 6.17 to 8.10 - Add statsd metrics to track merge latencies specific to rust vs nodejs ## Merge Checklist _Choose all relevant options below by adding an `x` now or at any time before submitting for review_ - [X] PR title adheres to the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) standard - [X] PR has a [changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets) - [X] PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore) - [X] PR includes [documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs) if necessary. - [X] All [commits have been signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits) ## Additional Context There's a README in `src/addon/` that describes the rust objects used <!-- start pr-codex --> --- ## PR-Codex overview This PR focuses on moving the DB and Reactions store to Rust for performance improvement. ### Detailed summary - Added Rust modules for DB operations - Updated storage configurations - Improved logging functionality - Implemented Rust functions for hashing - Refactored DB iterators - Added Rust client for statsd - Updated Cargo.toml dependencies and build.rs - Reorganized store modules > The following files were skipped due to too many changes: `apps/hubble/package.json`, `apps/hubble/src/network/sync/trieNode.ts`, `apps/hubble/src/storage/jobs/revokeMessagesBySignerJob.ts`, `apps/hubble/src/storage/stores/castStore.test.ts`, `Dockerfile.hubble`, `apps/hubble/src/storage/db/hubState.ts`, `apps/hubble/src/storage/db/migrations/2.fnameproof.ts`, `apps/hubble/src/storage/stores/usernameProofStore.ts`, `apps/hubble/src/storage/stores/store.test.ts`, `apps/hubble/src/storage/stores/castStore.ts`, `apps/hubble/src/storage/db/nameRegistryEvent.ts`, `apps/hubble/src/storage/db/migrations/1.usernameproof.ts`, `apps/hubble/src/storage/jobs/pruneMessagesJob.ts`, `apps/hubble/src/storage/db/migrations/5.fnameSyncIds.ts`, `.github/workflows/ci.yml`, `apps/hubble/src/network/sync/syncEngine.ts`, `apps/hubble/src/addon/src/statsd.rs`, `apps/hubble/src/storage/stores/storageCache.ts`, `apps/hubble/src/storage/stores/linkStore.ts`, `apps/hubble/src/storage/stores/onChainEventStore.ts`, `apps/hubble/src/storage/db/onChainEvent.ts`, `apps/hubble/src/storage/stores/reactionStore.test.ts`, `apps/hubble/src/storage/engine/index.ts`, `apps/hubble/src/network/sync/merkleTrie.ts`, `apps/hubble/src/profile/profile.ts`, `apps/hubble/src/rpc/server.ts`, `apps/hubble/src/rustfunctions.test.ts`, `apps/hubble/src/addon/README.md`, `apps/hubble/src/addon/src/lib.rs`, `apps/hubble/src/addon/src/store/store_event_handler.rs`, `apps/hubble/src/storage/jobs/validateOrRevokeMessagesJob.ts`, `apps/hubble/src/storage/stores/storeEventHandler.test.ts`, `apps/hubble/src/addon/src/logger.rs`, `apps/hubble/src/hubble.ts`, `yarn.lock`, `apps/hubble/src/storage/db/message.ts`, `apps/hubble/src/storage/stores/verificationStore.ts`, `apps/hubble/src/addon/src/store/utils.rs`, `apps/hubble/src/storage/stores/storeEventHandler.ts`, `apps/hubble/src/storage/stores/store.ts`, `apps/hubble/src/addon/src/store/message.rs`, `apps/hubble/src/rustfunctions.ts`, `apps/hubble/src/storage/db/rocksdb.ts`, `apps/hubble/src/addon/src/store/reaction_store.rs`, `apps/hubble/src/storage/stores/reactionStore.ts`, `apps/hubble/src/addon/src/db/rocksdb.rs`, `apps/hubble/src/storage/db/rocksdb.test.ts`, `apps/hubble/src/addon/src/store/store.rs`, `apps/hubble/src/addon/Cargo.lock`, `apps/hubble/src/addon/src/proto/protobufs.rs` > ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your question}` <!-- end pr-codex --> --------- Co-authored-by: Wasif Iqbal <[email protected]>

Search Similar