Client & errors

Creating a client

The quick start reads everything from the environment with .environ_map = init.environ_map. You can also set every option explicitly:

var client: tai.Client = try .init(gpa, io, .{
    .api_key = api_key, // default: TYPESAFE_API_KEY from environ_map
    .base_url = "https://api.typesafe.ai", // default: TYPESAFE_BASE_URL, then this
    .model = "jev-latest", // default: TYPESAFE_DEFAULT_MODEL, then this
    .retry = .{ .max_retries = 3, .backoff_max_ms = 10_000 },
    .extra_headers = &.{.{ .name = "x-request-source", .value = "docs" }},
});
defer client.deinit();

Explicit options take precedence over environment variables, and environment variables take precedence over the defaults. Empty or whitespace-only environment values are ignored. init returns error.MissingApiKey if no key is found for TypeSafe’s own API.

Other System One servers

Any server that implements the System One API works through base_url, such as Kev running on your own machine. A key is optional for any server other than TypeSafe’s, and without one the client sends no Authorization header:

var client: tai.Client = try .init(gpa, io, .{
    .base_url = "http://127.0.0.1:8009",
    .model = "kev-latest",
});
defer client.deinit();

The client wraps a std.http.Client, so connections are pooled and reused. Requests can be issued from multiple threads.

Retries

Failed requests are retried with exponential backoff and jitter. The defaults match the official Python and JavaScript SDKs:

429 Too Many Requests and 529 Overloaded are retried by default. Set the policy on the client, as above, or per call with .retry (see per-call options). Use tai.RetryPolicy.none to disable retries.

Errors

Zig errors can’t carry a payload, so tai returns a precise error and puts the details in an optional Diagnostics value. This example is tested with an invalid API key:

var diag: tai.Client.Diagnostics = .{};
defer diag.deinit();

const outcome = client.ask("Is this spam?", .{ .spam = tai.noul("Is this spam?") }, .{ .diagnostics = &diag });
const problem: []const u8 = if (outcome) |_| "" else |err| switch (err) {
    // diag.status is 401; diag.body is the server's JSON explanation.
    error.Unauthorized => diag.body,
    error.RateLimited, error.Overloaded => "busy, try again later",
    else => return err,
};

Diagnostics records:

• status: the last HTTP status, if a response arrived.
• body: the last non-2xx body, usually JSON such as {"detail": {...}}.
• attempts: how many attempts were made, including the first.
• transport_error: the underlying error behind ConnectionFailed.

Call deinit on it when you’re done, since it owns a copy of the body.

Listing models

listModels calls GET /v1/models, which lists the aliases your account can use. Versioned ids such as jev-1.13.0 are accepted in model whether or not they’re listed.

var models = try client.listModels(.{});
defer models.deinit();

var has_latest = false;
for (models.value.models) |m| {
    if (std.mem.eql(u8, m.name, "jev-latest")) has_latest = true;
}

Timeouts

std.http.Client doesn’t support per-request timeouts yet. Because every call takes a std.Io, you can cancel it from outside. For example, run the call with io.concurrent and call cancel on the future after a deadline; the call then returns error.Canceled.