ArangoDB v4.x is under development and not released yet.

This documentation is not final and potentially incomplete.

Transactions

AQL queries can be transactional and you can execute supported operations as part of a client-controlled Stream Transaction

Transaction Types

ArangoDB offers different types of transactions:

AQL queries (with exceptions)
Stream Transactions

AQL Queries

AQL queries are principally transactional. If you insert 100 documents into a collection using a single AQL query and the first 99 writes are successful but the last one fails, the query is aborted. None of the documents are typically persisted and are not even visible temporarily to other operations.

FOR i IN 0..99
  INSERT { _key: TO_STRING(i % 99)  } INTO coll
  // Duplicate key ("0") on the 100th document write

There are limitations to the transactionality, however. The following conditions can make AQL queries non-transactional:

A query exceeds the specified size thresholds, causing the RocksDB storage engine to perform intermediate commits. The query’s operations carried out so far are committed and not rolled back in case of a later abort/rollback. See Known limitations for AQL queries.
A query runs in a cluster deployment and involves different shards, DB-Servers, or both. This can be a query using a collection with more than one shard or a multi-collection query. It is possible that write operations get committed for some shards but not others, and the system cannot automatically resolve this situation. The client sees some kind of error, but this can be a timeout or connection loss.

Stream Transactions

Stream Transactions allow you to perform multi-document transactions with individual begin and commit / abort commands. They work similar to the BEGIN, COMMIT, and ROLLBACK operations in relational database systems.

Only certain operations like document CRUD and AQL queries can be run as part of a Stream Transactions.

The client is responsible for making sure that transactions are committed or aborted when they are no longer needed, to avoid taking up resources.

Transactional Properties

Transactions in ArangoDB are atomic, consistent, isolated, and durable (ACID).

These ACID properties provide the following guarantees:

The atomicity principle makes transactions either complete in their entirety or have no effect at all.
The consistency principle ensures that no constraints or other invariants are violated during or after any transaction. A transaction never corrupts the database.
The isolation property hides the modifications of a transaction from other transactions until the transaction commits.
Finally, the durability proposition makes sure that operations from transactions that have committed are made persistent. The amount of transaction durability is configurable in ArangoDB, as is the durability on collection level.

The descriptions in this section only provide a general overview. The actual transactional guarantees depend on the deployment mode and usage pattern.

Also see:

Operation Atomicity for more details on atomicity guarantees.
Transactional Isolation for more details on isolation guarantees in the single server and OneShard database case.
Cluster Transaction Limitations for more details on the transactional behavior of multi-document transactions in cluster deployments.

Limitations of transactions

In general

Transactions in ArangoDB have been designed with particular use cases in mind. They are mainly for short and small data retrieval operations, modification operations, or both.

The implementation is not optimized for very long-running or very voluminous operations, and may not be usable for these cases.

One limitation is that transaction operations and transaction metadata must fit into main memory. The actual data modification operations of a transaction are only written to the write-ahead log on commit and therefore need to fit entirely into main memory.

Ongoing transactions also prevent the write-ahead logs from being fully garbage-collected. Information in the write-ahead log files cannot be written to collection data files or be discarded while transactions are ongoing.

To ensure progress of the write-ahead log garbage collection, transactions should be kept as small as possible, and big transactions should be split into multiple smaller transactions.

Transactions in ArangoDB cannot be nested, i.e. you cannot start another Stream Transaction inside of a Stream Transaction. It simply starts a separate Stream Transaction if you try to.

It is disallowed to execute user transaction on some of ArangoDB’s own system collections. This shouldn’t be a problem for regular usage as system collections don’t contain user data and there is no need to access them from within a user transaction.

Some operations are not allowed inside transactions in general:

Creation and deletion of databases
Creation and deletion of collections and Views
Creation and deletion of indexes
Other data definition operations

If you try to run such operations as part of a Stream Transaction, ArangoDB executes them independent of the transaction without warning.

Finally, all collections that may be modified during a transaction must be declared beforehand, i.e. using the collections attribute of the object passed when starting a Stream Transaction. If any attempt is made to carry out a data modification operation on a collection that was not declared in the collections attribute, the transaction aborts and ArangoDB throws error 1652 (unregistered collection used in transaction). It is possible to not declare collections you only read from, but this should be avoided if possible to reduce the probability of deadlocks and non-repeatable reads.

Transactions in cluster deployments

Using a single instance of ArangoDB (or a OneShard database in a cluster), multi-document / multi-collection queries are guaranteed to be fully ACID in the traditional sense . For more details see Operation Atomicity and Transactional Isolation. This is more than many other NoSQL database systems support. In cluster mode, single-document operations are also fully ACID.

Multi-document / multi-collection queries and transactions offer different guarantees. Understanding these differences is important when designing applications that need to be resilient against outages of individual servers.

Cluster transactions share the underlying characteristics of the storage engine that is used for the cluster deployment. A transaction started on a Coordinator translates to one transaction per involved DB-Server. The guarantees and characteristics of the given storage-engine apply additionally to the cluster specific information below. Please refer to Locking and isolation of transactions for more details on the storage-engines.

Atomicity

A transaction on one DB-Server is either committed completely or not at all.

ArangoDB transactions do currently not require any form of global consensus. This makes them relatively fast, but also vulnerable to unexpected server outages.

Should a transaction involve Leader Shards on multiple DB-Servers, the atomicity of the distributed transaction during the commit operation cannot be guaranteed. Should one of the involved DB-Servers fail during the commit the transaction is not rolled-back globally, sub-transactions may have been committed on some DB-Servers, but not on others. Should this case occur, the client application sees an error.

An improved failure handling issue might be introduced in future versions.

Consistency

ArangoDB provides consistency even in the cluster. A transaction never leaves the data in an incorrect or corrupt state.

In a cluster deployment, there is always exactly one DB-Server responsible for a given shard. The locking procedure in the RocksDB storage engine ensures that dependent transactions (in the sense that the transactions modify the same documents or unique index entries) are ordered sequentially. Therefore we can provide Causal-Consistency for your transactions.

From the applications point-of-view this also means that a given transaction can always read its own writes . Other concurrent operations don’t change the database state seen by a transaction.

Isolation

The ArangoDB Cluster provides Local Snapshot Isolation. This means that all operations and queries in the transactions see the same version, or snapshot, of the data on a given DB-Server. This snapshot is based on the state of the data at the moment in time when the transaction begins on that DB-Server.

Durability

It is guaranteed that successfully committed transactions are persistent. Using replication, waitForSync, or both, increases the durability (just as with the single server).

Size and time limits

Intermediate commits

Intermediate commits that would automatically split and commit parts of big transactions are disabled for Stream Transactions in the RocksDB storage engine, including AQL queries that run inside of such transactions.

Limits for Stream Transactions

Stream Transactions enforce a maximum lifetime and transaction size. See Stream Transactions for details.

Durability of transactions

Transactions are executed until there is either an abort or a commit.

The RocksDB storage engine applies operations of a transaction only in main memory until they are committed. In case of an abort, the entire transaction is just cleared, no extra rollback steps are required.

In the event of a server crash, the storage engine scans the write-ahead log to restore certain metadata like the number of documents in collection or the selectivity estimates of secondary indexes.

There is thus the potential risk of losing data between the commit of the transaction and the actual (delayed) disk synchronization. This is the same as writing into collections that have the waitForSync property set to false outside of a transaction. In case of a crash with waitForSync set to false, the operations performed in the transaction are either visible completely or not at all, depending on whether the delayed synchronization had kicked in or not.

To ensure durability of transactions on collections that have the waitForSync property set to false, you can set the waitForSync option to true when creating the Stream Transaction. This forces a synchronization of the transaction to disk even for collections that have waitForSync set to false:

var trx = db._createTransaction({
  collections: { write: "users" },
  waitForSync: true
});
// Perform operations like trx.save(), trx.update(), trx.query() ...
trx.commit();

An alternative is to perform an individual operation with an explicit waitForSync request (if supported) in a transaction. Example:

trx.collection("users").save({ _key: "1234" }, { waitForSync: true });

In this case, the waitForSync option makes the whole transaction be synchronized to disk at the commit.

In any case, ArangoDB gives you the choice of whether or not you want full durability for single collection transactions. Using the delayed synchronization (i.e. waitForSync with a value of false) potentially increases throughput and performance of transactions, but introduces the risk of losing the last committed transactions in the case of a crash.

When using db._createTransaction() with waitForSync set to true, the call to trx.commit() only returns after the data of all modified collections has been synchronized to disk and the transaction has been made fully durable. This not only reduces the risk of losing data in case of a crash but also ensures consistency after a restart.

Locking and isolation of transactions

Transactions need to specify from which collections they will read data and which collections they intend to modify. This can be done by setting the read, write, or exclusive attributes in the collections attribute when starting a Stream Transaction:

var trx = db._createTransaction({
  collections: {
    read: "users",
    write: ["test", "log"]
  }
});
trx.query(`FOR doc IN users RETURN doc`).toArray().forEach(function(doc) {
  trx.collection("log").insert({ value: "removed user: " + doc.name });
  trx.collection("test").remove(doc._key);
});
trx.commit();

The meaning of the attributes is the following:

read: Read-only access to the specified collection(s).
write: Write access to the collection(s), including any read accesses. The write access is shared, which means it can be interleaved with write accesses by other concurrent transactions.
exclusive: Exclusive write access to the collection(s). Write accesses of concurrent transactions cannot be interleaved if they involve the same collection(s).

Storage engine

The RocksDB storage engine does not lock any collections participating in a transaction for read. Read operations can run in parallel to other read or write operations on the same collections.

Locking

For all collections that are used in write mode, the RocksDB engine internally acquires a (shared) read lock. This means that many writers can modify data in the same collection in parallel (and also run in parallel to ongoing reads). However, if two concurrent transactions attempt to modify the same document or index entry, there is a write-write conflict, and one of the transactions aborts with error 1200 (conflict). It is then up to client applications to retry the failed transaction or accept the failure. Note that Stream Transactions are not aborted automatically if an operation fails that is part of the transaction. It is the client’s responsibility to react to the outcome of the individual operations and abort the transaction if necessary.

In order to guard long-running or complex transactions against concurrent operations on the same data, the RocksDB engine allows you to access collections in exclusive mode. Exclusive accesses internally acquire a write-lock on the collections, so that they are not executed in parallel with any other write operations. Read operations can still be carried out by other concurrent transactions.

Isolation

The RocksDB storage-engine provides snapshot isolation. This means that all operations and queries in the transactions see the same version, or snapshot, of the database. This snapshot is based on the state of the database at the moment in time when the transaction begins. No locks are acquired on the underlying data to keep this snapshot, which permits other transactions to execute without being blocked by an older uncompleted transaction (so long as they do not try to modify the same documents or unique index-entries concurrently). In the cluster, a snapshot is acquired on each DB-Server individually.

Lazily adding collections

There might be situations when declaring all collections a priori is not possible, for example, because further collections are determined by a dynamic AQL query inside the transaction, for example a query using AQL graph traversal.

In this case, it would be impossible to know beforehand which collection to lock, and thus it is legal to not declare collections that will be accessed in the transaction in read-only mode. Accessing a non-declared collection in read-only mode during a transaction adds the collection to the transaction lazily, and fetches data from the collection as usual. However, as the collection is added lazily, there is no isolation from other concurrent operations or transactions. Reads from such collections are potentially non-repeatable.

Examples:

var trx = db._createTransaction({
  collections: { read: "users" }
});
/* Execute an AQL query that traverses an anonymous graph starting at a "users" node.
   It is yet unknown into which other collections the query might traverse */
trx.query(`FOR v IN ANY "users/1234" connections RETURN v`).toArray().forEach(function (d) {
  /* ... */
});
trx.commit();

This automatic lazy addition of collections to a transaction also introduces the possibility of deadlocks. Deadlocks may occur if there are concurrent transactions that try to acquire locks on the same collections lazily.

In order to make operations fail when a non-declared collection is used inside a Stream Transaction for reading, set the allowImplicit option to false:

var trx = db._createTransaction({
  collections: {
    read: "users",
    allowImplicit: false
  }
});
/* The below query now fails because the collection "connections" has not
   been specified in the list of collections used by the transaction */
trx.query(`FOR v IN ANY "users/1234" connections RETURN v`).toArray().forEach(function (d) {
  /* ... */
});
trx.abort();

The default value for allowImplicit is true. Write-accessing collections that have not been declared in the collections array is never possible, regardless of the value of allowImplicit.

If users/1234 has an edge in connections, linking it to another document in the users collection, then the following explicit declaration works:

var trx = db._createTransaction({
  collections: {
    read: ["users", "connections"],
    allowImplicit: false
  }
});
// Perform operations like trx.save(), trx.update(), trx.query() ...
trx.commit();

However, if the edge points to a document in another collection, then the query fails, unless that other collection is added to the declaration as well.

Note that if a document handle is used as starting point for a traversal, e.g. FOR v IN ANY "users/not_linked" ... or FOR v IN ANY {_id: "users/not_linked"} ..., then no error is raised in the case of the start node not having any edges to follow, with allowImplicit set to false and the users collection not being declared for read access. AQL only sees a string and does not consider it a read access, unless there are edges connected to it. FOR v IN ANY DOCUMENT("users/not_linked") ... fails even without edges, as it is always considered to be a read access to the users collection.