Data access using Cursor
Query operations that fetch multiple documents from server do not immediately return all values.
The query can potentially match large sets of documents, and may need to be fetched in batches to
optimize network and memory usage. These queries return an object cursor
that facilitates data
consumption. Following are a few ways to access data from a cursor:
Asynchronous iteration
Cursor implements AsyncIterator
interface allowing iteration using for await...of
syntax.
const productsCursor = catalog.findMany();
for await (const product of productsCursor) {
console.log(product);
}
Streaming
A Cursor can be translated to Readable stream
of documents. This allows user to control the flow of data using stream.pause()
, stream.resume()
or stream.pipe()
and consume it as the application demands. A stream can be consumed in one of the
following two ways:
Streaming: Consume as events
Your event-driven application can consume stream by listening for following three events:
"data"
- Stream operates in object mode and emits document as typescript objects as "data" event"error"
- Any internal failure would be emitted as "error" event"end"
- "end" event is emitted when there is no more data to be consumend from the stream
const productsCursor = catalog.findMany();
const productsStream = productsCursor.stream();
productsStream.on("data", (document) => console.log(document));
productsStream.on("error", (err: Error) => console.log(err));
productsStream.on("end", () => console.log("No more data"));
Streaming: Asynchronous iteration
Streams support asynchronous iteration using for await...of
syntax
const productsCursor = catalog.findMany();
const productsStream = productsCursor.stream();
for await (const product of productsStream) {
console.log(product);
}
Array of documents
An array of documents can be requested from Cursor synchronously using toArray()
method, this
would entail reading all documents from database matching the query and returning it as an Array.
const productsCursor = catalog.findMany();
const products = await productsCursor.toArray();
console.log(products);
If a query matches large number of documents, the internal memory may not be enough to store the
Array. In such cases, toArray()
can cause performance issues or failures if dataset exceeds memory
constraints. It is recommended to use one of the iteration or streaming methods to read documents.
Utility methods
Reset
Although a Cursor object exposes multiple ways to consume data, it restricts usage to only a single
reading mode to preserve consistency. For example, if documents are being read from cursor iteratively,
building a stream out of it will throw TigrisCursorInUseError
. This is to ensure integrity in
consuming source.
const productsCursor = catalog.findMany();
// cursor is an iterator, calling next() would retrieve an item
productsCursor[Symbol.asyncIterator]().next();
// now consuming data via stream() or toArray() would throw error
try {
for await (const product of productsCursor.stream()) {
//...
}
} catch (err: TigrisCursorInUseError) {
// error is thrown
}
try {
productsCursor.toArray();
} catch (err: TigrisCursorInUseError) {
// error is thrown
}
reset()
method can be used to re-use the Cursor. This essentially sends a new query to server and
and allows Cursor to be re-used. A new query is sent regardless of current state, and the Cursor may
yield different results after reset.
const productsCursor = catalog.findMany();
// cursor is an iterator, calling next() would retrieve an item
productsCursor[Symbol.asyncIterator]().next();
productsCursor.reset();
console.log(productsCursor.toArray());
// no error