POST
/
api
/
engine
/
chat
/
{provider}
curl --request POST \
  --url http://localhost:8000/api/engine/chat/{provider} \
  --header 'Content-Type: application/json' \
  --data '{
  "api_key": "<string>",
  "model": "<string>",
  "chat_input": "<string>",
  "parameters": "<any>",
  "is_stream": "<any>"
}'
{
  "id": "e2dc428b-bcee-43bf-844f-2a00d707bf49",
  "chat_input": "Write a haiku",
  "chat_output": " Here is a haiku I wrote for you:\n\nLeaves float gently down\nAutumn breeze whispers softly\nNature's beauty shines",
  "timestamp": 1702054550.185086,
  "provider": "anthropic",
  "model": "claude-2.1",
  "metrics": {
    "input_tokens": 4,
    "output_tokens": 30,
    "total_tokens": 34,
    "cost": 0.0007520000000000001,
    "latency": 3.2548670768737793,
    "time_to_first_token": 0.8070380687713623,
    "inter_token_latency": 0.09620759963989257,
    "tokens_per_second": 7.988037417789844
  },
  "parameters": {
    "temperature": 1,
    "max_tokens_to_sample": 256,
    "top_p": 1,
    "top_k": 5
  }
}
api_key
string
default: "None"required

API key to access the provider

model
string
required

API key to access the provider

chat_input
string
required

API key to access the provider

parameters
json
default: "None"

API key to access the provider

is_stream
json
default: "False"

API key to access the provider

List providers