Mass Insert Builder for redis-cli and valkey-cli --pipe

Turn CSV, JSON or plain commands into a file in the Redis protocol, ready for redis-cli --pipe or valkey-cli --pipe, the fast way to load a lot of data into Redis or Valkey. It also reads protocol bytes back as commands and replies. It runs in your browser.

Nothing you paste leaves this page. The tool doesn’t send anything anywhere, and the page’s security policy stops it from fetching or loading anything from another site, so your data stays on your machine.

On a wide screen you can open it full screen.

How to Use It

  1. Paste your data or load a file. CSV with any common delimiter, JSON as a list or one object per line, or commands written the way you’d type them in redis-cli.
  2. Write the key. Use ${field} where a field goes: user:${id}. For a cluster, a hash tag keeps related keys together: cart:{${user}}:items.
  3. Pick how each record is stored: a hash, a string, a list, a set or a sorted set. Add an expiry, a database number, or “delete each key first” so loading twice doesn’t double your lists.
  4. Download the file and load it:
redis-cli -h 10.0.0.4 -p 6379 --pipe < data.resp

The client prints errors: 0, replies: N when it’s done. --pipe talks to one server and doesn’t follow cluster redirects, so for a cluster, split the data by primary first. The Hash Slot Calculator shows which primary each key belongs to.

The second half of the page reads protocol bytes back as commands and replies. Paste raw bytes, a log line with \r\n written out, plain hex, or a hex dump from hexdump -C, xxd, od -t x1 or Wireshark’s Follow TCP Stream. From Follow TCP Stream it keeps both sides in order and reads the server’s lines as replies.

Why a Protocol File

A loop that sends one command and waits for the answer spends most of its time waiting. With half a millisecond per round trip, a million keys take more than eight minutes before the server does any real work. --pipe streams commands as fast as the connection takes them and counts the replies as they come back.

It wants its input in the protocol itself, where every value is preceded by its length in bytes. That’s what makes it safe for any data, line breaks and binary included, and also what makes hand-written files go wrong: count one length in characters instead of bytes and every command after it is garbage. The builder counts bytes.

How It Was Tested

Files built by the tool were loaded into Valkey 9.1.2 and Redis 8.10.2 with --pipe, and every key was read back and compared with the source data. On both servers, 2,000 CSV rows became hashes with expiries, 3,000 rows became 150 lists in the right order, 2,500 JSON lines became 300 sets, 1,887 scores became sorted-set entries, and 1,500 JSON records with 20-digit IDs were stored byte for byte. Every load ended with errors: 0. The reader was checked against real replies from both servers in RESP2 and RESP3, push messages and attributes included, and against dumps of a real session from hexdump -C, xxd, od -t x1 and Wireshark’s tshark.

The Code

The builder’s logic is one JavaScript file with no dependencies, open source under the Apache License 2.0. It runs as this page, as a command line in Node.js, and inside your own code:

node pipe/cli.js csv users.csv --key 'user:${id}' --type hash --ttl 3600 | redis-cli --pipe

The full manual and the tests are in the pipe folder on GitHub. The other tools work the same way.