/ 
OpenAPI-Client-OpenAI-0.24
River stage one • 1 direct dependent • 1 total dependent
⭐ Starred 7 GitHub stars
/ OpenAPI::Client::OpenAI::Path::evals

NAME

OpenAPI::Client::OpenAI::Path::evals - Documentation for the /evals path.

DESCRIPTION

This document describes the API endpoint at /evals.

PATHS

GET /evals

List evals

List evaluations for a project.

Operation ID

listEvals

$client->listEvals( ... );

Parameters

  • after (in query) (Optional) - Identifier for the last eval from the previous pagination request.

    Type: string

  • limit (in query) (Optional) - Number of evals to retrieve.

    Type: integer

    Default: 20

  • order (in query) (Optional) - Sort order for evals by timestamp. Use `asc` for ascending order or `desc` for descending order.

    Type: string

    Allowed values: asc, desc

    Default: asc

  • order_by (in query) (Optional) - Evals can be ordered by creation time or last updated time. Use `created_at` for creation time or `updated_at` for last updated time.

    Type: string

    Allowed values: created_at, updated_at

    Default: created_at

Responses

Status Code: 200

A list of evals

Content Types:

  • application/json

    Example (See the OpenAI spec for more detail):

    {
  "object": "list",
  "data": [
    {
      "object": "eval",
      "id": "eval_67abd54d9b0081909a86353f6fb9317a",
      "data_source_config": {
        "type": "custom",
        "schema": {
          "type": "object",
          "properties": {
            "item": {
              "type": "object",
              "properties": {
                "input": {
                  "type": "string"
                },
                "ground_truth": {
                  "type": "string"
                }
              },
              "required": [
                "input",
                "ground_truth"
              ]
            }
          },
          "required": [
            "item"
          ]
        }
      },
      "testing_criteria": [
        {
          "name": "String check",
          "id": "String check-2eaf2d8d-d649-4335-8148-9535a7ca73c2",
          "type": "string_check",
          "input": "{{item.input}}",
          "reference": "{{item.ground_truth}}",
          "operation": "eq"
        }
      ],
      "name": "External Data Eval",
      "created_at": 1739314509,
      "metadata": {},
    }
  ],
  "first_id": "eval_67abd54d9b0081909a86353f6fb9317a",
  "last_id": "eval_67abd54d9b0081909a86353f6fb9317a",
  "has_more": true
}

POST /evals

Create eval

Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the Evals guide.

Operation ID

createEval

$client->createEval( ... );

Parameters

Request Body

Content Type: application/json

Example:

{
   "testing_criteria" : [
      null
   ]
}

Responses

Status Code: 201

OK

Content Types:

  • application/json

    Example (See the OpenAI spec for more detail):

    {
  "object": "eval",
  "id": "eval_67abd54d9b0081909a86353f6fb9317a",
  "data_source_config": {
    "type": "custom",
    "item_schema": {
      "type": "object",
      "properties": {
        "label": {"type": "string"},
      },
      "required": ["label"]
    },
    "include_sample_schema": true
  },
  "testing_criteria": [
    {
      "name": "My string check grader",
      "type": "string_check",
      "input": "{{sample.output_text}}",
      "reference": "{{item.label}}",
      "operation": "eq",
    }
  ],
  "name": "External Data Eval",
  "created_at": 1739314509,
  "metadata": {
    "test": "synthetics",
  }
}

SEE ALSO

OpenAPI::Client::OpenAI::Path

COPYRIGHT AND LICENSE

Copyright (C) 2023-2025 by Nelson Ferraz

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.14.0 or, at your option, any later version of Perl 5 you may have available.