message-queue - How to backup all streams in JetStream server?

Question

Problem:

I have to backup all the streams of my JetStream server. I don't know what streams are there. I want to backup all the streams into a single snapshot. Currently, we can backup only a single stream using the following command.

$ nats stream backup <stream-name> /backup/dir/<stream-name>.tgz

What I have tried so far:

I have tried providing a wildcard instead of <stream-name>. It does not work.

$ nats stream backup * /backup/dir/backup.tgz
nats: error: "*" is not a valid stream name, try --help

Possible Workaround:

At first, I can list all the streams using nats str ls command. Then, I can loop through all the streams and backup them individualy.

However, this does not satisfy my requirement as I want to backup all the streams into a single snapshot. My snapshot should represent a complete state of the JetStream server just not a single stream.

标签： message-queuejetstreamnats.io

Answer 1

Yes creating a script to capture the list of all currently defined streams using nats stream ls and then backing them up in a loop using nats stream backup is the best you can do.

It is not possible to create a single 'frozen at a point in time' complete backup of all the streams at the same time because:

• NATS and JetStream continue to operate while the backup is happening
• NATS and JetStream are distributed systems: servers and streams are independent from each other

The only way to create such a backup would be to synchronously suspend all writes (publications or consumptions of messages) on all streams and their source subjects, perform the backup (which could take a fair amount of time), and then allow operations on the streams and subject to resume. This interruption in service would not be acceptable to many (most?) nats users.