Features and Improvements in ArangoDB 4.x

The following list shows in detail which features have been added or improved in ArangoDB 4.x. ArangoDB 4.x also contains several bug fixes that are not listed here.

ArangoSearch

Analyzers

Web interface

AQL

Spread operator for arrays and objects

You can use the spread syntax ..., inspired by JavaScript, to insert the elements of an array into an array literal and to copy the attributes of an object into an object literal:

LET arr = [2, 3]
LET obj = { b: 2, c: 9 }
RETURN {
  array: [1, ...arr, 4],         // [1, 2, 3, 4]
  object: { a: 1, ...obj, c: 3 } // { "a": 1, "b": 2, "c": 3 } (last "c" wins)
}

The spread syntax is a more concise and readable alternative to combining arrays with the PUSH() and APPEND() functions, and to merging objects with the MERGE() function. For details, see Array spread and Object spread.

This change also includes a change of behavior for duplicate attribute names in object literals. The last occurrence now wins instead of the first one. See Incompatible changes in ArangoDB 4.x.

String concatenation with the + operator

The + operator is now overloaded and can concatenate strings. If at least one of its two operands is a string, it concatenates the operands as strings, casting the other operand to a string if necessary. If both operands are non-string values, it performs arithmetic addition as before. This makes the operator behave similarly to the + operator in JavaScript.

RETURN "foo" + "bar" // "foobar"
RETURN "answer: " + 42 // "answer: 42"
RETURN 123 + "200" // "123200"
RETURN 1 + 2 // 3

Previously, you had to use the CONCAT() function for string concatenation and the + operator always performed arithmetic addition. Note that this is a potentially breaking change for queries that relied on the previous behavior, see Incompatible changes in ArangoDB 4.x.

For more information, see String operators.

PARTITION() string function

The new PARTITION() AQL function splits a string at a single occurrence of a separator and returns an array of exactly three strings: the part before the separator, the separator itself, and the part after the separator. An optional occurrence parameter lets you select which occurrence of the separator to split at, counted from the start (positive values) or from the end (negative values).

RETURN PARTITION("foo:bar:baz", ":", -1) // ["foo:bar", ":", "baz"]

UNION_DISTINCT_STABLE() function

The new UNION_DISTINCT_STABLE() function combines the unique values of an arbitrary number of arrays into a single array, like the existing UNION_DISTINCT() function, but retains the order of the elements. Each value appears at the position of its first occurrence across the arrays, processed from left to right:

RETURN UNION_DISTINCT_STABLE([1, 2, 3], [3, 2, 1], [4], [5, 6, 1])
// [1, 2, 3, 4, 5, 6]

Like UNION_DISTINCT(), the UNION_DISTINCT_STABLE() function cannot be used as an aggregation function in a COLLECT operation.

LIKE, NOT LIKE, =~, !~ as array comparison operators

You can now combine the LIKE and NOT LIKE operators for wildcard matching as well as the =~ and !~ operators for regular expression matching with the array comparison operators ALL, ANY, NONE, and AT LEAST (<expression>). This lets you match the elements of an array against a pattern, for example:

["foo", "bar"]  ANY LIKE  "b%"           // true
["foo", "bar"]  AT LEAST (2) LIKE  "_oo" // false

["foo", "bar"]  ALL =~  "[a-fro]{3}"  // true
["foo", "bar"]  ANY !~  "^mo+$"       // true

Previously, these operators were the only comparison operators that could not be combined with the array comparison operators. Internally, these constructs are transformed into approximately the following AQL expressions using the question mark operator:

["foo", "bar"][? ANY FILTER LIKE(CURRENT, "b%")]
["foo", "bar"][? AT LEAST(2) FILTER LIKE(CURRENT, "_oo")]

["foo", "bar"][? ALL FILTER REGEX_TEST(CURRENT, "[a-fro]{3}")]
["foo", "bar"][? ANY FILTER ! REGEX_TEST(CURRENT, "^mo+$")]

Sub-attributes as edge weights in graph queries

The weightAttribute option of graph queries is no longer limited to top-level edge attributes. In addition to a string, you can now specify an array of strings that describes the path to the attribute to read the edge weight from. This is supported by the following graph query types:

• Traversals with order: "weighted"
• SHORTEST_PATH
• K_SHORTEST_PATHS

For example, you can use the value of the nested sub attribute of the following edge document as the edge weight:

{ "attr": { "sub": 3 } }

FOR v IN OUTBOUND SHORTEST_PATH "nodes/A" TO "nodes/D" GRAPH "weightGraph"
  OPTIONS { weightAttribute: ["attr", "sub"] }
  RETURN v._key

A . in a string is still interpreted as a literal dot and thus selects a top-level attribute, for instance attr.sub of the following edge document:

{ "attr.sub": 5 }

FOR v IN OUTBOUND SHORTEST_PATH "nodes/A" TO "nodes/D" GRAPH "weightGraph"
  OPTIONS { weightAttribute: "attr.sub" }
  RETURN v._key

This change is backward compatible. Passing a string works like before and accesses a top-level attribute with exactly the specified name.

Improved joins in sharded clusters

A new upgrade-scatter-to-distribute optimizer rule has been added to utilize sharding information for join queries in cluster deployments.

In the execution plan, the optimization upgrades a ScatterNode to a DistributeNode where a join filter already determines the distribution. Only the respective DB-Server is involved if it’s known to have the documents when using a filter for a join like this:

FOR doc1 IN coll1
  FOR doc2 IN coll2
    FILTER doc2._key == doc1.attr
    RETURN [doc1, doc2]

If coll2 uses _key as sharding attribute, the only matching documents with doc1.attr == doc2._key reside on the DB-Server that serves the shard for doc2._key. Therefore, only doc1 / doc1.attr has to be distributed to that server, not scattered to all of them. This equally works for custom shardKeys if you filter by them in the join.

Improved joins for SmartGraphs

A new smart-join-smart-edge optimizer rule has been added to perform joins locally when joining edges on nodes that are part of a SmartGraph.

All incident edges are available locally for the join, so there is no need to contact other DB-Servers. For repeated joins, where the adjacent node might not reside on the same DB-Server, the query plan can still be optimized by only involving the DB-Server that has the node, using the SmartGraph attribute value that is encoded in the _from and _to attributes of the edge.

FOR n IN nodes
  FOR e IN edges
    FILTER e._from == n._id  // Local join, no other DB-Servers involved
    FOR m IN nodes
      FILTER e._to == m._id // Distributed to relevant DB-Server only
      RETURN [n, e, m]

Indexing

Upgrading vector indexes

Introduced in: v3.12.10

Newly created vector indexes use a new format version for writing data into RocksDB as well as a new format for the vector index metadata (the trained data produced by faiss) since v3.12.10.

To take advantage of the optimizations, you need to recreate the vector indexes after upgrading to v3.12.10 or later. Existing vector indexes are not automatically rewritten to the new format.

Server options

Miscellaneous changes

Client tools

Internal changes