Skip to Content
DocumentationGetting Started

Getting Started

Cloudburst is for developers using Postgres databases that need to synchronize their backend data with frontend apps. We connect to your existing postgres database and turn it into a realtime database. This allows you subscribe to a query from your frontend code and always have an up to date set of results for that query as the data in your database changes (with eventual consistency). The queries scale. We don’t hammer your database. It just works the way it’s supposed to.

We are currently in private alpha. Contact us to sign up and receive a database configuration. We will provide you with an ACCOUNT_ID, DATABASE_ID, and API_KEY to use with the rest of the set up. Stay tuned for our local development tools and performance metrics coming out shortly.

Example database

Cloudburst connects to your existing Postgres database via a logical replication slot. From your perspective as a developer, any time a change happens on the database that would affect the result of your query, you will receive the full refreshed query result set. Behind the scenes, Cloudburst tracks changes happening on the database, and delivers only the data needed to update the result set to the user’s device, so your query’s are refreshed in realtime without the overhead of rerunning full queries on your database.

For the rest of this tutorial, we’ll be working with a example database for a rudimentary chat application with one table, messages:

CREATE TABLE messages (
  group_name TEXT,
  sender_name TEXT, 
  message_text TEXT, 
  message_timestamp TIMESTAMP WITH TIME ZONE DEFAULT current_timestamp
  message_id uuid DEFAULT gen_random_uuid() PRIMARY KEY, 
);
 
CREATE INDEX recent_message_idx ON messages (group_name, message_timestamp);

Query Config

Before subscribing to a query, we need to provide Cloudburst with a query config specifying what queries clients are allowed to subscribe to. Let’s assume that our chat app already has a list of chat groups that a user is a part of loaded, and that we want to get an up to date list of recent messages for any given group. The query we want to run is:

SELECT * FROM messages WHERE group_name = $1 AND message_timestamp > $2

where $1 will be the name of the group we want messages for and $2 will be our timestamp cutoff for messages. To enable this query with Cloudburst, we specify the config

QUERY_CONFIG='
{
  "cbAccountId": <ACCOUNT_ID>,
  "sourceDatabaseId: <DATABASE_ID> 
  "activeTables": [{
    "tableName": "messages",
    "liveQueries": [
      {
        "conditions": [
          ["message_timestamp", ">"],
          ["group_name", "="]
        ]
      }
    ]
  }]
}'

In the config we include the ACCOUNT_ID and DATABASE_ID. Then we specify that we want to allow queries on the table messages and that we want to subscribe to queries on that table with a > filter on message_timestamp and an = filter on group_name. The filters specified in the conditions block for a query must exactly match the filters for your query.

We can then upload the config with

curl -X PUT -H "Content-Type: application/json" -H "x-api-key: $API_KEY" -d $QUERY_CONFIG "https://api.cloudburst.dev/database_config"

It is also necessary for queries to have an associated index on the database. This will be used when loading the initial result set for the query. For this query, we specified the index with the table specification above.

Client Side Subscription

Now that we have a query registered with Cloudburst, we can subscribe to it from our frontend. Cloudburst currently offers web client support through the library @cloudburst-dev/web.

First we create a Cloudburst client

import { Cloudburst } from "@cloudburst-dev/web";
 
const cloudburstClient = Cloudburst.create('https://api.cloudburst.dev', refreshAuthToken),

Note that we are providing a function refreshAuthToken to the client constructor. We create this function in the Client Auth section.

Ideally you will create one client when your application loads, and use it until your application closes. If you are creating and deleting multiple clients however, you can close a client the dispose function.

cloudburstClient.dispose();

With a cloudburst client in hand we can subscribe to our query. We will subscribe to all messages in the group party_chat that were sent in the last day.

  // set referenceDate to 24 hours ago
  const referenceDate = new Date();
  referenceDate.setDate(referenceDate.getDate() - 1);
 
  const unsubscribeQuery = client
    .selectFrom("messages")
    .where("message_timestamp", ">=", referenceDate)
    .where("group_name", "=", "party_chat")
    .onQueryResultChange(
      (row_values: readonly Record<string, unknown>[]) => {
        console.log("Up to date list of messages:", row_values);
      }, 
      () => {
        console.log("error");
      }
    );

After specifying the table and filters for the query, we subscribe to it with onQueryResultChange which takes two callbacks. The first callback is called whenever the query result set is updated and provides the updated list of rows for the query. The second callback is called if the subscription encounters an error. If this error callback is called, the subscription has entered an irrecoverable state (for example, due to an auth error) and has been canceled.

By default, Postgres puts tables in the public schema. If your table is in a different schema, you can specify the schema like so

  const unsubscribeQuery = client
    .selectFrom("messages")
    .schema("some_other_schema")

After subscribing to the query we are handed back an unsubscribeQuery function. This can be called to cancel the subscription without destroying the entire client.

unsubscribeQuery()

Queries currently support the >, <, >=, <=' and =filters. Currently you can use any number of=` filters, and only one inequality filter in a query.

We now have a subscription to our messages table. Before it works though, we need to set up auth.

Client Auth

Cloudburst does not come with an auth framework. We hook into your existing auth system, identifying clients by a jwt that you get from our api using your API_KEY. The authentication steps are

  1. You provide a function to the Cloudburst client that requests an auth token from your api. In this function, you include whatever auth credentials are necessary for you to verify the identity of a user.
  2. After authorizing the user requesting a token from your api, you request a token for that user from the Cloudburst api using your API_KEY.
  3. You return the requested token from your api and then from the client side auth function. The cloudburst client will use the token for subsequent requests to Cloudburst.

Here is an example client side auth function provided to cloudburst

const refreshAuthToken = async () => {
    const authTokenRequest = {
      userId: USER_ID,
    };
    const response = await fetch(<AUTH_API_ENDPOINT>, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify(authTokenRequest),
    });
 
    if (response.status !== 200) {
      throw new Error(
        `Failed to retrieve auth token from server: ${response.status}`,
      );
    }
    return await response.json();
  }

You can set a new function on the Cloudburst client to update the auth credentials used in the request.

  cloudburstClient.setAuthTokenRefreshFunction(refreshAuthToken);

Here is a corresponding Next.js endpoint that retrieves the auth token from Cloudburst

export async function POST(request: Request) {
  const body = await request.json();
  const { userId } = body;
  const token_request = {
    userId,
  };
 
  const response = await fetch("https://api.cloudburst.dev/user_auth_token", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-key": <API_KEY>,
    },
    body: JSON.stringify(token_request),
  });
 
  if (response.status !== 200) {
    throw new Error(
      `Failed to retrieve auth token from Cloudburst server: ${response.status}`
    );
  }
 
  return response;
}

Security rules

Wait! What stops a user from accsesing data they are not supposed to? This is handled by a security rule on your api. You provide us with a url to your security rule endpoint and we send a POST request to that endpoint for every row that will be delivered to a client. Every security request includes the body

{
  "userId":     string  // The id for the user who will be receiving the row. This id 
                        // matches the user id used in the client auth workflow.
  "sourceDbId": uuid    // The id of the database the row was read from.
  "schema":     string  // The postgres schema of the table the row is read from.
  "table":      string  // The name of the table the row is read from.
  "row":        object  // An object containing the retrieved row's data.
}

For every request, you return the object

{
  "delivery_authorized": bool // True if the user is allowed to receive the specified row. False otherwise.
}

If access for a row is denied, the corresponding subscription will be cancelled.

Last updated on