Real-time full-text search is a feature that enhances the user experience of web applications, particularly in online stores, social media platforms, documentation, and blogs. It enables users to search and instantly get up-to-date information returned to them. Combining that with an experience where search results update as a user types (without direct user query submission or a page reload) provides an even better UX that helps users get the information they need more efficiently.
In this tutorial, we'll walk you through converting a static Next.js e-commerce product listing into a database-driven site with real-time full-text search of all products using Tigris.
In the following section, we'll provide background info on real-time, full-text search, and how Tigris is an enabler of both. However, feel free to jump to the Tutorial to follow the step-by-step guide. Or, head to the real-time full-text search GitHub repo if you want to dive into the code.
Background
What Is Real-Time Search?
Real-time search returns search results based on an underlying search index that is instantaneously updated following any database operation. This allows users to find relevant and up-to-date information as quickly as possible.
There are a growing number of real-time search use cases, such as news aggregator websites that display the latest headlines as soon as they are published and social media platforms that show new posts from a user's network as soon as they are posted.
The increase in use cases is driven by user preference; as more and more information is generated online, real-time search gives users what they are looking for faster than before. As a result, many companies are incorporating real-time search capabilities to provide a better user experience on their online applications.
What is Full-Text Search?
Full-text search is a technique that allows users to search for information in a database or document collection by entering a query: typically a combination of keywords. The full-text search uses natural language processing and algorithms to understand the context and meaning of the query and returns results that are relevant even if they do not match the exact terms. This method is commonly used in various applications such as search engines, e-commerce sites, and enterprise searches to help employees find the information they need. Full-text search engines can provide features such as text indexing, parsing, tokenizing, stemming, creating an inverted index, ranking results, and providing an API or web interface for performing the search.
How does Tigris help with real-time full-text search?
Tigris is a Serverless NoSQL Database and Search Platform that integrates the database, search engine, and sync mechanism in a single, unified and fully managed platform, making it easy to build real-time search capabilities into applications. The database functionality is built on FoundationDB, an open-source, distributed, transactional key-value store. Tigris is designed to integrate with real-time web applications and serverless functions.
Real-time full-text search can present several challenges for developers, such as running two systems (database and search) in parallel, standing up and managing a sync mechanism to keep the database and search indexes in sync in real-time and remapping the search indexes as the database evolves.
Tigris addresses these challenges by providing an integrated database, search engine, and sync mechanism in a single, unified, and fully managed platform. This integration eliminates the complexity of running separate database and search systems, and the need to sync the database and search manually; Tigris does it all automatically. With Tigris, developers can implement real-time search functionality within their applications using a single NoSQL platform.
Already convinced? Then, sign up for Tigris Cloud. You should still read the rest of the tutorial, too, though!
Build a Next.js e-commerce product listing app
To follow along with this tutorial, you'll need to do the following:
- Install Node.js 18.13.0 or newer and NPM
- A Tigris Cloud account
Get the static e-commerce site
First, clone the Next.js e-commerce application created for the demonstrations in this tutorial:
git clone https://github.com/tigrisdata-community/real-time-full-text-search-nextjs
Then change the directory to the project folder, switch to the starter branch, and install the project dependencies:
cd real-time-full-text-search-nextjs
git checkout starter && npm install
You can now run the application and see the static listing of products:
npm run dev
The code and file structure will look familiar because the application was
created using the create-next-app command. The product information is defined
in db/products.json, which is then used in pages/index.tsx to display the
product listing.
Now, let's change the application to store and retrieve the products via a Tigris database.
Create a Tigris Project
Create a Tigris project for your e-commerce store in your Tigris Cloud dashboard. Click on the Create a new project button, enter store as your project name, and click on the Create button to continue.
After you select the Create button, a window will appear with a command to scaffold an application for your new Tigris Project. But we can ignore that.
As mentioned above, we do not need to execute the npx create-tigris-app
command. Click Go to Project.
Next, copy the Project Name, Client ID, and Client Secret from the
Application keys section of your new Tigris project. Then, open the project in
your preferred code editor, create a .env.local file in the root directory of
the project, and paste the credentials inside it:
TIGRIS_URI=api.preview.tigrisdata.cloud
TIGRIS_PROJECT=<REPLACE WITH PROJECT_NAME>
TIGRIS_CLIENT_ID=<REPLACE WITH CLIENT_ID>
TIGRIS_CLIENT_SECRET=<REPLACE WITH CLIENT_SECRET>
TIGRIS_DB_BRANCH="develop"
Install the Tigris TypeScript SDK
To integrate Tigris into your application, install the Tigris Typescript SDK:
npm install @tigrisdata/core
Next, update the tsconfig.json file with the configurations below to specify
the options the TypeScript compiler should use when transpiling the project's
code:
{
"compilerOptions": {
...
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
},
"ts-node": {
"compilerOptions": {
"module": "commonjs"
}
},
...
}
In the above configuration, we update compilerOptions object, which contains
several properties that control the TypeScript compiler's behavior, such as:
experimentalDecorators: Enables experimental support for TypeScript decorators, which is a way to add metadata to a class or its members.emitDecoratorMetadata: Enables emitting design-type metadata for decorated declarations in the source.ts-node: It's a configuration for ts-node package that allows you to run TypeScript files directly from the command line.moduleproperty is set tocommonjs, meaning that the module resolution is set to use the commonjs strategy.
Create Tigris Data Models
Each Tigris project has a database already set up, and it stores data records in the form of documents. These documents are similar to JSON objects and are organized into groups called Collections.
Let's create a Tigris model to define the collection for the e-commerce
application. Create a db/models folder in the root directory of your
e-commerce project. Inside that folder, create a products.ts file, and add the
code:
import {
Field,
PrimaryKey,
SearchField,
TigrisCollection,
TigrisDataTypes,
} from "@tigrisdata/core";
@TigrisCollection("products")
export class Product {
@PrimaryKey(TigrisDataTypes.INT32, { order: 1, autoGenerate: true })
id!: number;
@SearchField()
@Field()
name: string;
@Field()
price: number;
@SearchField({ sort: true })
@Field()
star: number;
@SearchField()
@Field()
tag: string;
@Field()
image: string;
}
The above code defines a Tigris model for the products collection in your
e-commerce application.
- It imports the
@TigrisCollectiondecorator, which defines that there should be a collection ofProductnamed “products” by passing in a name for the collection as an argument. - The
@PrimaryKeydecorator specifies the primary key for each record in the collection. - The
@Fielddecorator defines the fields in the collection - The
@SearchFielddecorator defines the fields that will be automatically synchronized in a search index so they can be used to search for records in the collection.
You can read more about the Tigris data model in our docs.
Instantiate the Tigris Client and get the database
Let's initialize the Tigris client and export it to other parts of the
application to make it available for use. Create a lib/tigris.ts file in the
project root folder and add the code snippets below:
import { DB, Tigris } from "@tigrisdata/core";
const tigrisClient = new Tigris();
const tigrisDB: DB = tigrisClient.getDatabase();
// export to share client and DB across modules
export { tigrisClient, tigrisDB };
The code imports the DB and Tigris classes from the @tigrisdata/core
package. It creates an instance of the Tigris class, which initializes the
Tigris client using the credentials stored in the environmental variables
defined in .env.local (note: these have to be loaded). Then creates a new
variable called tigrisDB and assigns it to the value returned by calling the
getDatabase method, which will get the pre-configured database of your project
from the Tigris console. Both the client and the database instances are exported
for shared use within the application.
Create Setup Scripts
Next, let's create a setup script that will create the collection for the data
model defined above. Go ahead to create a scripts/setup.ts file and add the
following code:
import { loadEnvConfig } from "@next/env";
loadEnvConfig(process.cwd());
import { Product } from "../db/models/products";
import { tigrisClient } from "../lib/tigris";
async function main() {
// ensure branch exists, create it if it needs to be created dynamically
await tigrisClient.getDatabase().initializeBranch();
// create collections
await tigrisClient.registerSchemas([Product]);
}
main()
.then(async () => {
console.log("Setup complete ...");
process.exit(0);
})
.catch(async (e) => {
console.error(e);
process.exit(1);
});
Here we import the loadEnvConfig function from the @next/env package, which
is used to load environment variables from .env.local. The tigrisClient is
imported from the shared utility library we just created, and we also import the
Product model. The loadEnvConfig must be called before importing the
tigrisClient as it relies on environmental variables having been loaded.
Within the main function, we initialize the Tigris client and create a
collection with the registerSchemas method using the provided model schema. We
call the initializeBranch() from getDatabase() method to create a
database branch
with the value supplied for the TIGRIS_DB_BRANCH variable in the .env.local
file.
Then update your package.json file to add the scripts setup, predev, and
postbuild, which all eventually run setup.ts:
"scripts": {
"predev": "npm run setup",
"dev": "next",
"build": "next build",
"postbuild": "npm run setup",
"setup": "npx ts-node ./scripts/setup.ts",
"start": "next start",
"typecheck": "tsc"
}
Now, run the scripts/setup.ts file to create a collection for the Product
model with the command:
npm run setup
You'll see output similar to the following telling you the schema registration was successful:
npm run setup
> my-app@0.1.0 setup
> npx ts-node ./scripts/setup.ts
Loaded env from /store/.env.local
info - Using reflection to infer type of Product#name
info - Using reflection to infer type of Product#name
info - Using reflection to infer type of Product#star
info - Using reflection to infer type of Product#star
info - Using reflection to infer type of Product#tag
info - Using reflection to infer type of Product#tag
info - Using reflection to infer type of Product#image
info - Using Tigris at: api.dev.tigrisdata.cloud:443
event - 'develop' branch already exists
info - Using database branch: 'develop'
event - Creating collection: 'products' in project: 'store'
Setup complete ...
Load Data to Tigris
With Tigris configured and the collection created for the application, it's time
to load some data into the products database collection. To do this, create a
scripts/loadData.ts file with the code below:
import { loadEnvConfig } from "@next/env";
loadEnvConfig(process.cwd());
import { Product } from "../db/models/products";
import { tigrisDB } from "../lib/tigris";
import productsJson from "../db/products.json";
async function main() {
const products: Array<Product> = productsJson as Array<Product>;
const productsCollection = tigrisDB.getCollection<Product>(Product);
const inserted = await productsCollection.insertMany(products);
console.log(inserted);
}
main()
.then(async () => {
console.log("Data loading complete ...");
process.exit(0);
})
.catch(async (e) => {
console.error(e);
process.exit(1);
});
As with the setup script, loadEnvConfig is imported and used to load the
environment variables. Then, it imports the Product model from
../db/models/products, which defines the structure of the products that will be
saved in the Tigris database. It also imports tigrisDB object from
../lib/tigris, which is responsible for handling the database operations.
The load data script reads the product data from the ../db/products.json file
and assigns it to the productsJson variable. Then, it defines an async
function main that performs an Array<Product>
type assertion
on productsJson. Then, it uses the tigrisDB object to access the “products”
collection and insert the data using the
insertMany
method. The function returns the result of the insertion and assigns it to the
inserted variable, which is logged to the console.
💡 Tigris also has an insertOne method to insert one or more documents into the collection.
Now, update the package.json file to add a script that allows you to load data
from the command line:
...
"scripts": {
...
"load-data": "npx ts-node ./scripts/loadData.ts"
},
...
Finally, use the command below to run the script that loads the product data in
the products.json file into the Tigris database:
npm run load-data
Upon executing the load-data command, the products are loaded into the
database and are visible on the data explorer page on the Tigris Console for
your Tigris Project.
Get All Products in the Next.js app
Now, let's add a function called fetchAll to the store API to an
api/store/index.ts file to get all the products in the database:
import { NextApiRequest, NextApiResponse } from "next";
import { Product } from "../../../db/models/products";
import { tigrisDB } from "../../../lib/tigris";
type FetchAllResponse = {
result?: Array<Product>;
error?: string;
};
async function fetchAll(res: NextApiResponse<FetchAllResponse>) {
try {
const productsCollection = tigrisDB.getCollection<Product>(Product);
const cursor = productsCollection.findMany();
const products = await cursor.toArray();
res.status(200).json({ result: products });
} catch (err) {
res.status(500).json({ error: err.message });
}
}
The code here will fetch all the data from the products collection using the
findMany() method (see
Query Documents
for more info). It will convert the cursor to an array using toArray()method
and store the data returned in the products variable.
Define the Next.js API handler
With the script to load and function to get products out of the way, add a handler function to the API endpoint to call the function you've just created earlier on a GET request:
export default async function handler(
req: NextApiRequest,
res: NextApiResponse<FetchAllResponse>
) {
switch (req.method) {
case "GET":
await fetchAll(res);
break;
default:
res.setHeader("Allow", ["GET"]);
res.status(405).end(`Method ${req.method} Not Allowed`);
}
}
Here we used a switch statement to determine which operation to perform based
on the HTTP method of the request, which is accessed using the req.method
property. If the request method is GET, the fetchAll function is called.
Update the UI to fetch the products from Tigris
Let's update the user interface of your e-commerce application to show the
products from the Tigris database and enable the real-time feature in the
application. First, update the Home Component in the pages/index.tsx as
follows:
//...
import { useEffect, useState } from "react";
import { Product } from "../db/models/store";
// import products from "../db/products.json";
export default function Home() {
const [products, setProducts] = useState<Product[]>([]);
const fetchProducts = async () => {
try {
const response = await fetch("/api/store");
const { result } = await response.json();
if (result) {
setProducts(result);
}
} catch (e) {
console.log(`Error: ${e}`);
}
};
useEffect(() => {
fetchProducts();
}, []);
//...
Here we import the useEffect and useState hooks from react and comment out
the loading of the products from the JSON file. We then create a state variable
named products with a function setProducts to update the value of
products. We also define a function fetchProducts which uses the
fetch API to
request the /api/store endpoint.
When the response is received within fetchProducts, it will be parsed as JSON,
and the result property of the JSON object is passed to the setProducts
function to update the products state variable. Then we use the useEffect
hook to call the fetchProduts function when the component renders for the
first time.
We've now converted our static product listing into a database-driven product listing.
Implement Real-time Search
At this point, you can get data from your collection by making an API request to
the store API (/api/store). But we need to add one more API endpoint to enable
the search experience for your users. So, let's create search.ts file in the
api/store directory to add real-time search to your application:
import { NextApiRequest, NextApiResponse } from "next";
import { Product } from "../../../db/models/products";
import { tigrisDB } from "../../../lib/tigris";
import { SearchQuery } from "@tigrisdata/core";
type Data = {
result?: Array<Product>;
error?: string;
};
export default async function handler(
req: NextApiRequest,
res: NextApiResponse<Data>
) {
const { query, page } = req.query;
try {
const productCollection = tigrisDB.getCollection<Product>(Product);
const searchRequest: SearchQuery<Product> = {
q: query as string,
sort: [
{
field: "star",
order: "$desc",
},
],
};
const results = await productCollection.search(
searchRequest,
Number(page) || 1
);
const products = new Array<Product>();
for (const hit of results.hits) {
products.push(hit.document);
}
res.status(200).json({ result: products });
} catch (err) {
res.status(500).json({ error: err.message });
}
}
In addition to the next package imports, we also import the Products
model, SearchQuery from the tigrisdata/core library and tigrisDB object.
We declare a Data type that defines the search response payload with
properties, results, and error of type Array<Products> and string,
respectively.
Then we destructure the query and page parameters from the incoming
request's query object and search for products in a collection using the Tigris
search method. The search query is defined as a SearchQuery object, with the
q property set to the incoming query parameter and sorting performed by the
field star. The page parameter is passed to the search method, with a
default value of 1.
Add search functionality to the UI
Begin by adding a state hook for the search query to the Home Component in
pages/index.tsx:
//...
export default function Home() {
const [searchInput, setSearchInput] = useState<string>();
//...
Next, create a function to make a GET request to the new search endpoint:
//...
const searchQuery = async () => {
const response = await fetch(`/api/store/search?query=${encodeURI(searchInput)}`);
const { result } = await response.json();
if (result) {
setProducts(result);
}
};
useEffect(() => {
//...
The searchQuery function sends a GET request using fetch to the
/api/store/search endpoint with a parameter name of query and the value set
to an encoded version of the user's text entry. The result is used to update the
products state variable.
Next, update the <form> tag in the Home Component to add event listeners to
call the the SearchQuery and SetSearchInput functions:
<form className="d-flex" role="search">
<input
className="form-control me-2"
type="search"
placeholder="Search"
aria-label="Search"
onKeyUp={searchQuery}
onChange={(e) => setSearchInput(e.target.value)}
/>
</form>
We bind the searchQuery function to the onKeyUp event so that the function
is called to perform a search every time the user releases a key.
That's it! Let's try out the application.
Test the Application
Now, let's test the application to see how the real-time search function works on your e-commerce application:
npm run dev
Your e-commerce application now lists the products from your Tigris database with the shiny real-time full-text search functionality.
Conclusion
In this tutorial, we explored the concept of real-time full-text search and how to implement it in a Next.js application using Tigris.
As a demonstration, we developed a Next.js e-commerce product listing app and
integrated the real-time search feature using Tigris.
We began by setting up the Tigris TypeScript SDK and created scripts to create
models for our product collection and import that product data into the
database. We then updated the application to retrieve the product details from
the Tigris database via a Next.js API endpoint. Finally, we added real-time
full-text search functionality for the products via a new API endpoint and by
updating the UI to listen for user keystroke events within a search form.
The complete code for this tutorial is available here on the main branch.
Next.js is a highly productive framework for building web apps. This tutorial shows just some of the benefits of combining Next.js with Tigris, a new all-in-one developer data platform that provides databases and automatic search indexing for real-time search. Check out the Tigris documentation to learn more. Happy coding!