首页 > 解决方案 > How to ensure consistent reads relative to rocksdb backup state

问题描述

I have a RocksDB instance which I'm backing up using BackupEngine. This backup then goes to S3. I'd like to also write some metadata to S3, as a separate, light file that can be used to at a glance understand the contents of the backup.

This metadata is already saved in the database itself, so all I need to do is read the data from the db, and write it to S3. My concern is how to make sure my metadata is consistent with the state of the backup; if I read before backing up, my metadata might be stale. If I read after the backup, my metadata might be ahead. What's the right way to read "from the backup" (i.e. to read the same db state the backup reads), without literally restoring the backup into a new instance?

I've read the documentation on Checkpoints and also this question which says:

The process of copying the files from the database to the backup, is protected from modifications happening on the database while the call is active, by creating a rocksdb::Checkpoint [...]

I could make a Checkpoint, and read my metadata from it, and the Checkpoints documentation says you can use a Checkpoint to backup a database, but I don't see a way to do that via BackupEngine. So I'm looking for a clean solution which lets me guarantee my db backup and my metadata read from the db are reading exactly the same state.

标签: rocksdb

解决方案


推荐阅读