Building front-end applications that interact with an LLM poses two unique challenges:

1. Preventing privilege escalation

How do you ensure that the LLM is only able to access the data that the user has access to?

Take the example of a function that changes a user address.

client.default.register({
  name: "updateAddress",
  input: {
    schema: z.object({
      userId: z.string(),
      address: z.string(),
    }),
  },
  func: async (input) => {
    await db.exec("UPDATE users SET address = $1 WHERE id = $2", [
      input.address,
      input.userId,
    ]);
  },
});

You need to be able to ensure that this function was invoked at the behest of the user in question. If a nefarious user can “trick” the agent into calling this function with a different user’s ID, they can change that user’s address.

2. Preventing API key exposure

Since all data encoded in a front-end application is client side, the API key can not be kept secret.

​

Custom Authentication & Authorization

Therefore, we need a way to implement an out of band authentication flow:

1. That can’t be bypassed, or influenced by the agent.
2. Deterministically applied to the function call.

Inferable provides a way to implement this via a custom authentication strategy.

Custom authentication allows you to call any of the Run API endpoints using a token that the developer is responsible for validating.

This is in contrast to Cluster API Keys which are vaildated by Inferable.

​

Handling Custom Authentication

Assuming your application has some kind of authentication token for the user’s session, you can pass this token into calls to Inferable. If available, the default.handleCustomAuth will be used to validate the token (Otherwise the request will be rejected).

Custom auth tokens can be used:

1. When calling the API: The Authorization header and delimited by the custom scheme (e.g. Authorization: "custom MY_TOKEN")
2. When using the react-sdk: Helpers in the react-sdk
3. When using the assistant-ui runtime: Helpers in the assistant-ui runtime

​

handleCustomAuth

The JSON serialized object returned from default.handleCustomAuth will be passed to all subsequent calls in the run within the context object, allowing for authentication context (userId, etc) to be propagated. If the function throws an error, the request will not be permitted.

Subsequent function calls in the Run will receive the auth context as a parameter.

​

Example

The following example demonstrates how to implement a custom authentication strategy. Note that handleCustomAuth is a reserved name, and cannot be used for any other purpose.

import { handleCustomAuth } from "inferable";

client.default.register({
  name: "handleCustomAuth",
  func: (input) => {
    const user = await db.exec("SELECT * FROM users WHERE token = $1", [
      input.token,
    ]);

    if (!user) {
      throw new Error("Invalid token");
    }

    return {
      userId: user.id,
      role: "user",
    };
  },
  schema: {
    input: handleCustomAuth,
  },
});

Now, you can create a run from any insecure context (such as a web app) with the user’s token in the Authorization header.

curl -X POST https://api.inferable.ai/clusters/$YOUR_CLUSTER_ID/runs \
  -H "Authorization: custom $USER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"initialPrompt": "What's my user ID?"}'

Prior to this run being executed, the handleCustomAuth function will be called with the user’s token. The resulting context will be passed to all subsequent function calls in the run. For example, assuming that you have the following function registered:

client.default.register({
  name: "getUserID",
  func: (input, context) => {
    console.log(context);
    // {
    //   authContext: {
    //     userId: "abc-123",
    //     role: "user",
    //   }
    // }

    return context.authContext.userId; // "abc-123"
  },
});