Writes

Learn how Synnax writes data to the database.

This page walks you through the lifecycle of a write to a cluster. If you’d like a practical guide on reading data using a client library, look at the respective pages for Python and TypeScript.

Transactions

The transaction lifecycle is controlled by a writer. To begin a transaction, a caller opens a new writer and then begins writing data frames using a file like interface. After they’re done writing, the user commits the write to the database, making it available for reading. Synnax transactions are unique in that they can be committed multiple times, where each new commit extends the data written in the previous commit. After the final commit, the user closes the writer, at which point the cluster releases all write related resources and the domain can no longer be extended.

Open Phase

To start a new transaction, a user must specify two properties:

1. The channels they wish to write to.
2. The timestamp marking the start of the new domain.

In addition, the user may specify whether the writer is configured with auto-commit. Auto-commit is off by default.

Using this information, the database validates the following:

1. That all channels exist in the database.
2. That no existing domain overlaps with the provided timestamp. If any domains overlap, the new domain is invalid and cannot be committed.
3. That all channels share the same rate or index.

Write Phase

After all validation checks pass, the transaction enters the write phase. During this period, a user can write continuous frames of data to the domain. These frames must be vertical (i.e. all channels must have a single array specified), and all arrays must have the same number of samples. You can read more about frame alignment here.

These frames are assumed to be continuous by the database. To illustrate this, imagine we write the following frame:

After this write, our database looks exactly like our first frame. Then we write the next frame:

Now, our database looks like this:

Throughout the write phase, we can continue extending our database by adding frames.

Commit Phase

Once we’re ready to persist the transaction and make it available for reads, we commit our writer. During this phase, the cluster identifies the time range of the writer based on the provided starting timestamp and the samples written. The commit process succeeds and the write is now available for reads. After this phase completes, the transaction is moved back into the write phase. At this point, we can either write more frames to the cluster and commit again, or we can close the writer.

Alternatively, if the writer is configured with auto-commit on, the commit phase automatically follows the write phase. This renders the data immediately available as it is written. However, this may have a minor impact on performance.

Close Phase

Once done writing, the user can close the writer and release all of it’s resources. All uncomitted writes will be lost.