aqueduct
Aqueduct is no longer being maintained. Aqueduct allows you to run LLM and ML workloads on any cloud infrastructure.
- 519
- Go
- Apache-2.0
- 3 y ago

A Python client to interact with Arize API
Repo Arize-ai/client_python, read from GitHub 4 h ago. Arize-ai has not joined Take My Repo.
Arize-ai/client_python is a public repo on GitHub by Arize-ai. A Python client to interact with Arize API. It declares the Apache-2.0 license on GitHub. It is written mainly in Python, has 57 stars and last changed on 2026-11-17. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-21.
In the maintainer's words. Shown from GitHub; Take My Repo does not keep a copy.
A helper package to interact with Arize AI APIs.
Arize is an AI engineering platform. It helps engineers develop, evaluate, and observe AI applications and agents.
Arize has both Enterprise and OSS products to support this goal:
We log over 1 trillion inferences and spans, 10 million evaluation runs, and 2 million OSS downloads every month.
Install the base package:
pip install arize
The following optional extras provide specialized functionality:
Note: The
otelextra installs thearize-otelpackage, which is also available as a standalone package. If you only need auto-instrumentation without the full SDK, installarize-oteldirectly.
| Extra | Install Command | What It Provides |
|---|---|---|
| otel | pip install arize[otel] |
OpenTelemetry auto-instrumentation package (arize-otel) for automatic tracing |
| embeddings | pip install arize[embeddings] |
Automatic embedding generation for NLP, CV, and structured data (Pillow, datasets, tokenizers, torch, transformers) |
| mimic | pip install arize[mimic] |
MIMIC explainer for model interpretability |
Install multiple extras:
pip install arize[otel,embeddings,mimic]
If you're upgrading from version 7, please refer to the Migration Guide for detailed migration steps and breaking changes.
Choose the region where your account is hosted. The public regions are US, EU, and CA. The default uses the normal US GCP deployment.
from arize import ArizeClient, Region
client = ArizeClient(api_key="your-api-key", region=Region.US)
You can also pass region="US" or set ARIZE_REGION=US. Friendly names are case-insensitive. The GCP identifiers and enum names remain valid aliases:
| Public region | Identifier | Enum alias |
|---|---|---|
US |
us-central-1a |
Region.US_CENTRAL_1A |
EU |
eu-west-1a |
Region.EU_WEST_1A |
CA |
ca-central-1a |
Region.CA_CENTRAL_1A |
The unfinished AWS US East deployment is not a supported public region. us-east-1b and us-east-1 are rejected. Use custom endpoints for self-hosted deployments.
Throughout the SDK, any parameter named space accepts either a space ID (e.g. "spc_abc123") or a space name (e.g. "my-space"). The SDK resolves the identifier automatically:
# Both of these are equivalent if your space is named "my-space"
client.datasets.list(space="U3BhY2U6OTA1MDoxSmtS")
client.datasets.list(space="my-space")
See arize-otel in PyPI:
from arize.otel import register
from openinference.instrumentation.openai import OpenAIInstrumentor
# Setup OpenTelemetry via our convenience function
tracer_provider = register(
space_id=SPACE_ID,
api_key=API_KEY,
project_name="agents-cookbook",
)
# Start instrumentation
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)
Use arize.spans to interact with spans: log spans into Arize, update the span's
evaluations, annotations and metadata in bulk.
from arize import ArizeClient
client = ArizeClient(api_key=API_KEY)
SPACE_ID = "<your-space-id>"
PROJECT_NAME = "<your-project-name>"
client.spans.log(
space_id=SPACE_ID,
project_name=PROJECT_NAME,
dataframe=spans_df,
# evals_df=evals_df, # Optionally pass the evaluations together with the spans
)
Use list to retrieve a page of spans. Set included_columns to return only
the columns you need, or set excluded_columns to omit large columns. Do not
set both. Fixed span fields are always returned. Set
truncate_string_length to shorten long string values inside each span's
attributes object. Strings longer than the limit are cut and end with …,
which counts toward the limit. Span IDs, names, status, timestamps,
annotations, evaluations, and events are not truncated.
from arize import ArizeClient
client = ArizeClient(api_key=API_KEY)
spans = client.spans.list(
project="<your-project-id>",
excluded_columns=["attributes.embedding.vectors"],
truncate_string_length=200,
)
for span in spans.spans:
print(span.name)
Use annotate to write human annotations to a batch of records via the REST API.
granularity selects what each record_id identifies: a span (SPAN, the
default), a trace's root span (TRACE), or a session (SESSION, written to
the root span of the session's earliest trace). Up to 1000 records may be
annotated per request for SPAN/TRACE; up to 100 for SESSION.
start_time/end_time bound the lookup window and are both optional: when
omitted, end_time defaults to now and start_time defaults to 31 days
before that (7 days for SESSION). If the record is older than the default
window, pass start_time explicitly or the record won't be found.
from datetime import datetime, timedelta, timezone
from arize import ArizeClient
from arize.spans.types import AnnotateRecordInput, AnnotationInput, RecordGranularity
client = ArizeClient(api_key=API_KEY)
end_time = datetime.now(timezone.utc)
start_time = end_time - timedelta(days=7)
client.spans.annotate(
project="<your-project-id>",
annotations=[
AnnotateRecordInput(
record_id="<session-id>",
values=[AnnotationInput(name="quality", label="good")],
),
],
granularity=RecordGranularity.SESSION, # optional; defaults to SPAN
start_time=start_time,
end_time=end_time,
)
from arize import ArizeClient
client = ArizeClient(api_key=API_KEY)
SPACE_ID = "<your-space-id>"
PROJECT_NAME = "<your-project-name>"
client.spans.update_evaluations(
space_id=SPACE_ID,
project_name=PROJECT_NAME,
dataframe=evals_df,
# force_http=... # Optionally pass force_http to update evaluations via HTTP instead of gRPC, defaults to False
)
client.spans.update_annotations(
space_id=SPACE_ID,
project_name=PROJECT_NAME,
dataframe=annotations_df,
)
client.spans.update_metadata(
space_id=SPACE_ID,
project_name=PROJECT_NAME,
dataframe=metadata_df,
)
Use the export_to_df or export_to_parquet to export large amounts of spans from Arize.
from arize import ArizeClient
from datetime import datetime
FMT = "%Y-%m-%d"
start_time = datetime.strptime("2024-01-01",FMT)
end_time = datetime.strptime("2026-01-01",FMT)
client = ArizeClient(api_key=API_KEY)
SPACE_ID = "<your-space-id>"
PROJECT_NAME = "<your-project-name>"
df = client.spans.export_to_df(
space_id=SPACE_ID,
project_name=PROJECT_NAME,
start_time=start_time,
end_time=end_time,
)
To export a random sample of traces instead of everything, pass sample_rate
(a fraction in [1e-6, 1]). Sampling happens server-side by trace id, so whole
traces are kept together and re-running the export returns the same sample:
df = client.spans.export_to_df(
space_id=SPACE_ID,
project_name=PROJECT_NAME,
start_time=start_time,
end_time=end_time,
sample_rate=0.1, # export ~10% of traces
)
Use client.traces to list traces for a project. Each returned trace carries
its full (flat) list of spans plus lightweight roll-up metadata, including
root_span_id and a spans_truncated flag. Traces are returned newest-first.
You can list traces for a project using client.traces.list(). Pass project
as a name (with space) or as an ID. Optional start_time/end_time,
filter, limit, and cursor (opaque pagination token) narrow the results.
The filter uses the same expression syntax as client.spans.list(), but the
semantics differ: a filter selects traces that contain at least one matching
span (the matching span is usually a child, not the root), rather than only
traces whose root span matches.
Note: This is a beta endpoint and may change without notice.
from arize import ArizeClient
client = ArizeClient(api_key=API_KEY)
SPACE = "<your-space-id-or-name>"
PROJECT = "<your-project-name-or-id>"
resp = client.traces.list(
project=PROJECT,
space=SPACE, # Optional when PROJECT is an ID
# start_time=..., # Optional, inclusive lower bound (defaults to 7 days ago)
# end_time=..., # Optional, exclusive upper bound (defaults to now)
# filter="status_code = 'ERROR'",
# limit=..., # Optional, defaults to 50
# cursor=..., # Optional, opaque pagination cursor
)
traces = resp.traces
for trace in traces:
print(trace.trace_id, trace.root_span_id, trace.spans_truncated, len(trace.spans))
# Paginate using the opaque cursor returned by the server
next_cursor = resp.pagination.next_cursor
Use arize.ml to interact with ML models: log ML data (training, validation, production)
into Arize, either streaming or in batches.
from arize import ArizeClient
from arize.ml.types import ModelTypes, Environments
client = ArizeClient(api_key=API_KEY)
SPACE_ID = "<your-space-id>"
MODEL_NAME = "<your-model-name>"
features=...
embedding_features=...
response = client.ml.log_stream(
space_id=SPACE_ID,
model_name=MODEL_NAME,
model_type=ModelTypes.SCORE_CATEGORICAL,
environment=Environments.PRODUCTION,
prediction_label=("not fraud",0.3),
actual_label=("fraud",1.0),
features=features,
embedding_features=embedding_features,
)
from arize import ArizeClient
from arize.ml.types import ModelTypes, Environments
client = ArizeClient(api_key=API_KEY)
SPACE_ID = "<your-space-id>"
MODEL_NAME = "<your-model-name>"
MODEL_VERSION = "1.0"
from arize.ml.types import Schema, EmbeddingColumnNames, ObjectDetectionColumnNames, ModelTypes, Environments
tags = ["drift_type"]
embedding_feature_column_names = {
"image_embedding": EmbeddingColumnNames(
vector_column_name="image_vector", link_to_data_column_name="url"
)
}
object_detection_prediction_column_names = ObjectDetectionColumnNames(
bounding_boxes_coordinates_column_name="prediction_bboxes",
categories_column_name="prediction_categories",
scores_column_name="prediction_scores",
)
object_detection_actual_column_names = ObjectDetectionColumnNames(
bounding_boxes_coordinates_column_name="actual_bboxes",
categories_column_name="actual_categories",
)
# Define a Schema() object for Arize to pick up data from the correct columns for logging
schema = Schema(
prediction_id_column_name="prediction_id",
timestamp_column_name="prediction_ts",
tag_column_names=tags,
embedding_feature_column_names=embedding_feature_column_names,
object_detection_prediction_column_names=object_detection_prediction_column_names,
object_detection_actual_column_names=object_detection_actual_column_names,
)
# Logging Production DataFrame
response = client.ml.log_batch(
space_id=SPACE_ID,
model_name=MODEL_NAME,
model_type=ModelTypes.OBJECT_DETECTION,
dataframe=prod_df,
schema=schema,
environment=Environments.PRODUCTION,
model_version = MODEL_VERSION, # Optionally pass a model version
)
Use the export_to_df or export_to_parquet to export large amounts of spans from Arize.
from arize import ArizeClient
from datetime import datetime
FMT = "%Y-%m-%d"
start_time = datetime.strptime("2024-01-01",FMT)
end_time = datetime.strptime("2026-01-01",FMT)
client = ArizeClient(api_key=API_KEY)
SPACE_ID = "<your-space-id>"
MODEL_NAME = "<your-model-name>"
MODEL_VERSION = "1.0"
df = client.ml.export_to_df(
space_id=SPACE_ID,
model_name=MODEL_NAME,
environment=Environments.TRAINING,
model_version=MODEL_VERSION,
start_time=start_time,
end_time=end_time,
)
import pandas as pd
from arize.embeddings import EmbeddingGenerator, UseCases
# You can check available models
print(EmbeddingGenerator.list_pretrained_models())
# Example dataframe
df = pd.DataFrame(
{
"text": [
"Hello world.",
"Artificial Intelligence is the future.",
"Spain won the FIFA World Cup on 2010.",
],
}
)
# Instantiate the generator for your usecase, selecting the base model
generator = EmbeddingGenerator.from_use_case(
use_case=UseCases.NLP.SEQUENCE_CLASSIFICATION,
model_name="distilbert-base-uncased",
tokenizer_max_length=512,
batch_size=100,
)
# Generate embeddings
df["text_vector"] = generator.generate_embeddings(text_col=df["text"])
You can list all datasets that the user has access to using client.datasets.list(). You can use the limit parameter to specify the maximum number of datasets desired in the response and you can specify space to filter by space. The space parameter accepts either a space ID (e.g. "U3BhY2U6OTA1MDoxSmtS") or a space name for substring filtering. You can also filter by dataset name.
resp = client.datasets.list(
name=... # Optional, case-insensitive substring filter on dataset name
space=... # Optional, space ID (e.g. "U3BhY2U6OTA1MDoxSmtS") or space name (e.g. "my-space")
limit=... # Optional, defaults to 100
)
The response is an object of type DatasetsList200Response, and you can access the list of datasets via its datasets attribute. In addition, you can transform the response object to a dictionary, to JSON format, or a pandas dataframe.
# Get the list of datasets from the response
dataset_list = resp.datasets
# Get the response as a dictionary
resp_dict = resp.to_dict()
# Get the response in JSON format
resp_json = resp.to_json()
# Get the response as a pandas dataframe
resp_df = resp.to_df()
You can create a dataset using client.datasets.create(). You must pass examples — we currently don't support creating an empty dataset. Examples can be provided as a list of dictionaries, a pandas dataframe, or a path to a Parquet or Arrow IPC file (or a directory of them). File paths are streamed via Arrow Flight one record batch at a time, so the dataset is never loaded into memory.
examples = [
{
"eval.Correctness Basic.explanation": "The query indicates that the user is having trouble accessing their account on their laptop, while access on their phone is still working. This suggests a potential issue with the login process on the laptop, which aligns with the 'Login Issues' queue. The mention of a possible change in the account could relate to login credentials or settings affecting the laptop specifically, but it still falls under the broader category of login issues.",
"eval.Correctness Basic.label": "correct",
"eval.Correctness Basic.score": 1,
"llm output": "Login Issues",
"query": "I can't get in on my laptop anymore, but my phone still works fine — could this be because I changed something in my account?"
},
{
"eval.Correctness Basic.explanation": "The query is about a user who signed up but is unable to log in because the system says no account is found. This issue is related to the login process, as the user is trying to access their account and is facing a problem with the login system recognizing their account. Therefore, assigning this query to the 'Login Issues' queue is appropriate.",
"eval.Correctness Basic.label": "correct",
"eval.Correctness Basic.score": 1,
"llm output": "Login Issues",
"query": "Signed up ages ago but never got around to logging in — now it says no account found. Do I start over?"
}
]
If the number of examples is too large, the client SDK will try to send the data via Arrow Flight via gRPC for better performance. If you want to force the data transfer to HTTP you can use the force_http flag. File paths always use Flight, so force_http is not allowed with them. The response is a Dataset object.
created_dataset = client.datasets.create(
space="<space-id-or-name>",
name="<your-dataset-name>", # Name must be unique within a space
examples=..., # List of dictionaries, pandas dataframe, or Parquet/Arrow path(s)
# force_http=... # Optionally pass force_http to create datasets via HTTP instead of gRPC, defaults to False
)
The Dataset object also has convenience methods:
# Get the response as a dictionary
dataset_dict = created_dataset.to_dict()
# Get the response in JSON format
dataset_json = created_dataset.to_json()
To get a dataset by its ID or name use client.datasets.get(). You can optionally pass space for disambiguation when looking up by name. The returned type is Dataset.
dataset = client.datasets.get(
dataset=... # The dataset ID or name
space=... # Optional, space ID or name (required when looking up by dataset name)
)
To delete a dataset by its ID or name use client.datasets.delete(). The call returns None if successful deletion took place, error otherwise.
client.datasets.delete(
dataset=... # The dataset ID or name
space=... # Optional, space ID or name (required when looking up by dataset name)
)
You can list the examples of a given dataset using client.datasets.list_examples(). You can specify the number of examples desired using the limit parameter. If you want all examples, use all=True, which exports data using Arrow Flight via gRPC for increased performance.
resp = client.datasets.list_examples(
dataset="<your-dataset-id-or-name>",
space=..., # Optional, space ID or name
dataset_version_id=..., # Optional, defaults to latest version
limit=... # number of desired examples. Defaults to 100
all=... # Whether or not to export all of the examples. Defaults to False
)
For more complex queries, pass an optional SQL-like filter to narrow results by example columns and annotation.<name>.* fields. filter is not supported when all=True.
resp = client.datasets.list_examples(
dataset="<your-dataset-id-or-name>",
filter="\"llm output\" = 'Login Issues'",
limit=50,
)
The response is an object of type DatasetsExamplesList200Response, and you can access the list of examples via its examples attribute. In addition, you can transform the response object to a dictionary, to JSON format, or a pandas dataframe.
# Get the list of examples from the response
examples_list = resp.examples
# Get the response as a dictionary
resp_dict = resp.to_dict()
# Get the response in JSON format
resp_json = resp.to_json()
# Get the response as a pandas dataframe
resp_df = resp.to_df()
To export all examples of a dataset to a Parquet file, use client.datasets.export_to_parquet(). Examples are streamed from Arize via Arrow Flight and written batch by batch, so datasets larger than available memory can be exported. The file holds the same rows and columns as list_examples(all=True), except that JSON-valued columns are stored as JSON strings. The file is written to <path>.partial and renamed when the export completes, so a failed export leaves any existing file at path unchanged.
client.datasets.export_to_parquet(
dataset="<your-dataset-id-or-name>",
space=..., # Optional, space ID or name (required when dataset is a name)
dataset_version_id=..., # Optional, defaults to the latest version
path="examples.parquet",
)
You can append examples to an existing dataset version using client.datasets.append_examples(). This creates a new dataset version with the appended data and returns the updated Dataset object.
updated_dataset = client.datasets.append_examples(
dataset="<your-dataset-id-or-name>",
space=..., # Optional, space ID or name
dataset_version_id="<version-id-to-append-to>",
examples=..., # List of dictionaries or pandas dataframe
)
You can update the content of existing dataset examples using client.datasets.update_examples(). Examples are matched by their existing id; an id not found in the targeted version is ignored (no error, no insert — use append_examples to add new examples). Pass new_version to snapshot the update as a new dataset version instead of updating in place.
updated_version = client.datasets.update_examples(
dataset="<your-dataset-id-or-name>",
space=..., # Optional, space ID or name
dataset_version_id="<version-id-to-update>", # Optional, defaults to latest version
examples=[
{"id": "<existing-example-id>", "question": "...", "answer": "..."},
],
new_version=..., # Optional, name for a new version; omit to update in place
)
You can delete a batch of examples from a dataset version using client.datasets.delete_examples(). Examples are removed in place from the given dataset_version_id (no new version is created). The delete is partial-tolerant and idempotent, so re-submitting already-deleted IDs is safe. Up to 1000 example IDs may be deleted per request, and they must not contain duplicate or empty values.
resp = client.datasets.delete_examples(
dataset="<your-dataset-id-or-name>",
space=..., # Optional, space ID or name
dataset_version_id="<version-id-to-delete-from>",
examples=..., # List of example IDs to delete (1-1000, no duplicates or empty values)
)
The response is an object of type DeleteDatasetExamplesResponse. Use completed to check whether the operation finished (retry the full request if False), deleted_example_ids for the IDs confirmed deleted, and not_deleted_example_ids for requested IDs that were not deleted (not found in the selected version, or not completed).
# Whether the operation finished and no retry is needed
completed = resp.completed
# IDs confirmed deleted in this request
deleted = resp.deleted_example_ids
# Requested IDs that were not deleted
not_deleted = resp.not_deleted_example_ids
You can list all experiments that the user has access to using client.experiments.list(). You can use the limit parameter to specify the maximum number of experiments desired in the response, and narrow the results by scope: pass dataset for only the experiments run on that dataset, or space for every experiment in that space — including those with no dataset. The two are mutually exclusive.
resp = client.experiments.list(
dataset=... # Optional, dataset ID or name
space=... # Optional, space ID or name
limit=... # Optional, defaults to 100
)
The response is an object of type ExperimentsList200Response, and you can access the list of experiments via its experiments attribute. In addition, you can transform the response object to a dictionary, to JSON format, or a pandas dataframe.
# Get the list of experiments from the response
experiment_list = resp.experiments
# Get the response as a dictionary
resp_dict = resp.to_dict()
# Get the response in JSON format
resp_json = resp.to_json()
# Get the response as a pandas dataframe
resp_df = resp.to_df()
You can run an experiment on a dataset using client.experiments.run() by defining a task, evaluators (optional), and passing the dataset id or name of the dataset you want to use, together with a name for the experiment. The function will download the entire dataset from Arize (unless cached, see caching section under "SDK Configuration"), execute the task to obtain an output, and perform evaluations (if evaluators were passed). The experiments will also be traced, and these traces will be visible in Arize. The experiment will be created and the data logged into Arize automatically. You can avoid logging to Arize by making dry_run=True. The function will return the Experiment object (or None if dry_run=True) together with the dataframe with the experiment data.
from arize.experiments import ExperimentMetadata
experiment, experiment_df = client.experiments.run(
name="<name-your-experiment>",
dataset="<your-dataset-id-or-name>",
space=..., # Optional, space ID or name
task=..., # The task to be performed in the experiment.
evaluators=..., # Optional: The evaluators to use in the experiment.
dry_run=..., # If True, the experiment result will not be uploaded to Arize. Defaults to False
dry_run_count=..., # Number of examples of the dataset to use in the dry run. Defaults to 10
concurrency=..., # The number of concurrent tasks to run. Defaults to 3.
set_global_tracer_provider=..., # If True, sets the global tracer provider for the experiment. Defaults to False
exit_on_error=..., # If True, the experiment will stop running on first occurrence of an error. Defaults to False
metadata=ExperimentMetadata(user_id="...", user_email="..."), # Optional: metadata attached to every experiment span
)
metadata accepts an ExperimentMetadata typed dict (for IDE autocomplete on known fields like user_id, user_name, user_email) or any plain dict[str, str] for custom fields. Dataset fields (dataset_id, dataset_name, dataset_version_id) are always populated automatically. Both can be combined via dict unpacking:
metadata={
**ExperimentMetadata(user_email="alice@example.com"),
"team": "ml-platform",
"git_sha": "abc123",
}
The Experiment object also has convenience methods:
# Get the response as a dictionary
experiment_dict = experiment.to_dict()
# Get the response in JSON format
experiment_json = experiment.to_json()
It is possible that you have run the experiment yourself without the above function, and hence you already have experiment data that you want to send to Arize. In this case, use the client.experiments.create() method by passing the runs data as a list of dictionaries, a pandas dataframe, or a path to a Parquet or Arrow IPC file (or a directory of them). File paths are streamed via Arrow Flight one record batch at a time and require dataset.
NOTE: If you don't have experiment data and want to run an experiment, see the
client.experiments.run()section above.
An experiment belongs to a space and may optionally be associated with a dataset. Pass exactly one of:
dataset — associates the experiment with a dataset, so its runs can reference that dataset's examples.space — creates the experiment directly in a space, with no dataset.In addition, you must specify which column holds the output using ExperimentTaskFieldNames. Also set example_id there when you pass dataset, so each run can be matched to the example it ran against; it isn't used for an experiment with no dataset. If you have evaluation data, indicate the evaluation columns using EvaluationResultFieldNames.
If the number of runs is too large, the client SDK will try to send the data via Arrow Flight via gRPC for better performance. If you want to force the data transfer to HTTP you can use the force_http flag. Experiments with no dataset always upload via HTTP, since the gRPC + Flight path only supports dataset-associated experiments. The response is an Experiment object.
from arize.experiments.types import ExperimentTaskFieldNames, EvaluationResultFieldNames
# Associated with a dataset: runs reference the dataset's examples.
created_experiment = client.experiments.create(
name="<your-experiment-name>", # Name must be unique within the dataset
dataset="<your-dataset-id-or-name>",
space=..., # Optional, space ID or name, used to resolve `dataset` by name
experiment_runs=..., # List of dictionaries, pandas dataframe, or Parquet/Arrow path(s)
task_fields=ExperimentTaskFieldNames(
output="<your-output-column>",
example_id="<your-example-id-column>",
),
evaluator_columns=... # Optional
# force_http=... # Optionally pass force_http to create experiments via HTTP instead of gRPC, defaults to False
)
# No dataset: created directly in a space, and runs need no example ID.
standalone_experiment = client.experiments.create(
name="<your-experiment-name>", # Name must be unique within the space
space="<your-space-id-or-name>",
experiment_runs=...,
task_fields=ExperimentTaskFieldNames(output="<your-output-column>"),
)
To get an experiment by its ID or name use client.experiments.get(). The returned type is Experiment (does not include experiment runs — use list_runs() for those).
experiment = client.experiments.get(
experiment=... # The experiment ID or name
dataset=... # Optional, dataset ID or name, to look up by name within a dataset
space=... # Optional, space ID or name; looks up by name within the space when
# `dataset` is omitted, which is the only option for an experiment
# with no dataset
)
To delete an experiment by its ID or name use client.experiments.delete(). The call returns None if successful deletion took place, error otherwise.
client.experiments.delete(
experiment=... # The experiment ID or name
dataset=... # Optional, dataset ID or name
space=... # Optional, space ID or name
)
You can list the runs of a given experiment using client.experiments.list_runs() and passing the experiment ID or name. You can specify the number of runs desired using the limit parameter. For more complex queries, pass an SQL-like filter expression: it supports unprefixed id, output, example_id, custom run columns, eval.<name>.score, eval.<name>.label, eval.<name>.explanation, eval.<name>.metadata.*, and annotation.<name>.* when present. Runs are returned in stable id ascending order; pass the cursor from a previous response's pagination.next_cursor to fetch the next page, keeping the filter unchanged across pages. If you want all runs, consider using the all=True parameter, which will make it so the SDK exports the data using Arrow Flight via gRPC, for increased performance (filter is not supported with all=True).
resp = client.experiments.list_runs(
experiment="<your-experiment-id-or-name>",
dataset=..., # Optional, dataset ID or name
space=..., # Optional, space ID or name
filter="output = '4'", # Optional SQL-like filter expression
limit=..., # number of desired runs. Defaults to 50; valid range 1-500
all=... # Whether or not to export all of the runs. Defaults to False
)
The response is an object of type ExperimentsRunsList200Response, and you can access the list of runs via its experiment_runs attribute. In addition, you can transform the response object to a dictionary, to JSON format, or a pandas dataframe.
# Get the list of runs from the response
run_list = resp.experiment_runs
# Get the response as a dictionary
resp_dict = resp.to_dict()
# Get the response in JSON format
resp_json = resp.to_json()
# Get the response as a pandas dataframe
resp_df = resp.to_df()
To export all runs of an experiment to a Parquet file, use client.experiments.export_to_parquet(). Runs are streamed from Arize via Arrow Flight and written batch by batch, so experiments larger than available memory can be exported. The file holds the same rows and columns as list_runs(all=True), except that JSON-valued columns are stored as JSON strings. The file is written to <path>.partial and renamed when the export completes, so a failed export leaves any existing file at path unchanged.
client.experiments.export_to_parquet(
experiment="<your-experiment-id-or-name>",
dataset=..., # Optional, dataset ID or name
space=..., # Optional, space ID or name (required when dataset is a name, or when experiment is a name and dataset is omitted)
path="runs.parquet",
)
Append between 1 and 1000 new runs to an existing experiment using client.experiments.append_runs(). Each run must include output; example_id (the ID of an example from the experiment's dataset) is required only when the target experiment is associated with a dataset. The response includes the updated experiment and the generated run IDs in input order (run_ids).
result = client.experiments.append_runs(
experiment="<your-experiment-id-or-name>",
dataset=..., # Optional, dataset ID or name (required when experiment is a name)
space=..., # Optional, space ID or name (required when dataset is a name)
experiment_runs=[
{"example_id": "ex-1", "output": "The answer is 42"},
{"example_id": "ex-2", "output": "Paris"},
],
# experiment_runs also accepts a pandas DataFrame
)
print(result.run_ids) # IDs of the appended runs, in input order
Use client.prompts to manage prompts and their versions in Arize's Prompt Hub.
resp = client.prompts.list(
name=..., # Optional, case-insensitive substring filter on prompt name
space=..., # Optional, space ID or name
limit=..., # Optional, defaults to 100
)
prompt_list = resp.prompts
Creating a prompt also creates its first version. You must specify the space, a name, and the prompt content.
from arize.prompts.types import InputVariableFormat, LlmProvider, LLMMessage, InvocationParams
prompt = client.prompts.create(
space="<space-id-or-name>",
name="<your-prompt-name>",
commit_message="Initial version",
input_variable_format=InputVariableFormat.F_STRING, # or MUSTACHE
provider=LlmProvider.OPEN_AI,
messages=[
LLMMessage(role="SYSTEM", content="You are a helpful assistant."),
LLMMessage(role="USER", content="Answer this question: {question}"),
],
description=..., # Optional
model=..., # Optional model name
invocation_params=..., # Optional
)
To get a prompt by its ID or name use client.prompts.get(). You can optionally request a specific version by version_id or by label. Defaults to the latest version. The returned type is PromptWithVersion.
prompt = client.prompts.get(
prompt="<your-prompt-id-or-name>",
space=..., # Optional, space ID or name
version_id=..., # Optional, get a specific version
label=..., # Optional, get the version with this label (e.g. "production")
)
updated_prompt = client.prompts.update(
prompt="<prompt-id-or-name>",
space=..., # Optional
description="Updated description",
)
client.prompts.delete(
prompt="<prompt-id-or-name>",
space=..., # Optional
)
You can list all versions of a prompt, create new versions, and retrieve a specific version.
# List versions
versions = client.prompts.list_versions(
prompt="<prompt-id-or-name>",
space=..., # Optional
limit=..., # Optional, defaults to 100
)
# Get a specific version
version = client.prompts.get_version(version_id="<version-id>")
# Create a new version
new_version = client.prompts.create_version(
prompt="<prompt-id-or-name>",
space=..., # Optional
commit_message="Update system prompt",
input_variable_format=InputVariableFormat.F_STRING,
provider=LlmProvider.OPEN_AI,
messages=[...],
model=..., # Optional
invocation_params=..., # Optional
)
Labels allow you to tag a specific prompt version with a named alias (e.g. "production", "staging").
# Get the version currently tagged with a label
version = client.prompts.get_version_by_label(
prompt="<prompt-id-or-name>",
space=..., # Optional
label_name="production",
)
# Set labels on a version (replaces all existing labels)
client.prompts.set_labels(
version_id="<version-id>",
labels=["production", "v2"],
)
# Delete a single label from a version
client.prompts.delete_label(
version_id="<version-id>",
label_name="staging",
)
Use client.evaluators to manage evaluators and their versions. Four evaluator
types are supported: template (LLM-based), code (managed built-in or custom Python),
remote (customer-hosted HTTP endpoint via an EVALUATOR integration), and
decision judge (typed questions answered by a Decision Judge model).
resp = client.evaluators.list(
name=..., # Optional, case-insensitive substring filter on evaluator name
space=..., # Optional, space ID or name
evaluator_type=..., # Optional, e.g. EvaluatorType.DECISION_JUDGE
limit=..., # Optional, defaults to 100
)
evaluator_list = resp.evaluators
Decision Judge evaluators are listed only when you pass
evaluator_type=EvaluatorType.DECISION_JUDGE; without a filter, every other
type is returned.
from arize.evaluators.types import TemplateConfigInput
evaluator = client.evaluators.create_template_evaluator(
name="<your-evaluator-name>",
space="<space-id-or-name>",
commit_message="Initial version",
template_config=TemplateConfigInput(...),
description=..., # Optional
)
from arize.evaluators.types import ManagedCodeConfigInput
evaluator = client.evaluators.create_code_evaluator(
name="<your-evaluator-name>",
space="<space-id-or-name>",
commit_message="Initial version",
code_config=ManagedCodeConfigInput(
name="hallucination",
managed_evaluator="hallucination",
variables=["input", "output", "context"],
),
description=..., # Optional
)
A remote evaluator calls a customer-hosted HTTP endpoint via an EVALUATOR integration. Requires an Enterprise account.
evaluator = client.evaluators.create_remote_evaluator(
name="<your-evaluator-name>",
space="<space-id-or-name>",
integration_id="<evaluator-integration-id>",
commit_message="Initial version",
description=..., # Optional
)
A Decision Judge evaluator asks a set of typed questions about each span and
writes one result per question, named eval.<evaluator name> <question name>.label
and .score. It runs on spans only, and Decision Judge must be enabled for the
account. The AI integration must use a Decision Judge provider (for example an
OpenAI integration with gpt-6-luna, or a TypeSafe AI, Perplexity, or
Cloudflare integration; see Create an LLM Integration).
Each question has a type that selects the shape of its criteria:
| Type | criteria |
Notes |
|---|---|---|
BOOLEAN |
Optional true_means and false_means |
A yes or no answer. Optional threshold between 0 and 1, exclusive: the answer is true when the model's probability of true is at least this value. Defaults to 0.5 |
CHOICE |
Required options: 2 to 255 labels, each mapped to when it applies |
One option from a fixed set |
SCORE |
Required levels: 2 to 10 descriptions, lowest to highest |
Optional direction, defaults to MAXIMIZE |
from arize.evaluators.types import (
DecisionJudgeBooleanCriteriaInput,
DecisionJudgeBooleanQuestionInput,
DecisionJudgeChoiceCriteriaInput,
DecisionJudgeChoiceQuestionInput,
DecisionJudgeScoreCriteriaInput,
DecisionJudgeScoreQuestionInput,
)
evaluator = client.evaluators.create_decision_judge_evaluator(
name="support reply",
space="<space-id-or-name>",
commit_message="Initial version",
decision_judge_config={
"ai_integration_id": "<ai-integration-id>",
"model_name": "gpt-6-luna",
"state_template": '{"input": "{input}", "output": "{output}"}',
"questions": [
DecisionJudgeBooleanQuestionInput(
name="resolved",
type="BOOLEAN",
instructions="Does the reply resolve the customer's issue?",
criteria=DecisionJudgeBooleanCriteriaInput(
true_means="The reply fully resolves the issue",
false_means="The issue is still open after the reply",
),
),
DecisionJudgeChoiceQuestionInput(
name="tone",
type="CHOICE",
instructions="What is the tone of the reply?",
criteria=DecisionJudgeChoiceCriteriaInput(
options={
"friendly": "Warm and polite",
"neutral": "Factual with no particular warmth",
"rude": "Dismissive or impolite",
}
),
),
DecisionJudgeScoreQuestionInput(
name="helpfulness",
type="SCORE",
instructions="How helpful is the reply?",
criteria=DecisionJudgeScoreCriteriaInput(
levels=["Not helpful", "Somewhat helpful", "Very helpful"]
),
direction="MAXIMIZE",
),
],
},
description="Checks support replies", # Optional
)
for question in evaluator.version.decision_judge_config.questions:
print(question.type, question.name)
decision_judge_config is a dict (as above), a DecisionJudgeConfigInput, or
the DecisionJudgeConfig read back from an existing version. A question can
also be a plain dict, such as
{"name": "resolved", "type": "BOOLEAN", "instructions": "..."}. On reads,
decision_judge_config.questions holds the concrete
DecisionJudgeBooleanQuestion, DecisionJudgeChoiceQuestion, or
DecisionJudgeScoreQuestion, so narrow on question.type. Read questions can
be passed straight back into a new version.
evaluator = client.evaluators.get(
evaluator="<evaluator-id-or-name>",
space=..., # Optional
version_id=..., # Optional, defaults to latest version
)
updated = client.evaluators.update(
evaluator="<evaluator-id-or-name>",
space=..., # Optional
name=..., # Optional
description=..., # Optional
)
client.evaluators.delete(
evaluator="<evaluator-id-or-name>",
space=..., # Optional
)
# List versions
versions = client.evaluators.list_versions(
evaluator="<evaluator-id-or-name>",
space=..., # Optional
limit=..., # Optional, defaults to 100
)
# Get a specific version
version = client.evaluators.get_version(version_id="<version-id>")
# Create a new template version
new_version = client.evaluators.create_template_version(
evaluator="<evaluator-id-or-name>",
space=..., # Optional
commit_message="Updated template",
template_config=TemplateConfigInput(...),
)
# Create a new remote version (switches to a different EVALUATOR integration)
new_remote_version = client.evaluators.create_remote_version(
evaluator="<evaluator-id-or-name>",
space=..., # Optional
commit_message="Switch endpoint",
integration_id="<evaluator-integration-id>",
)
A new Decision Judge version replaces the whole configuration, so pass every question the version should ask. The questions take the same shapes as in Create a Decision Judge Evaluator.
current = client.evaluators.get(
evaluator="<evaluator-id-or-name>",
space="<space-id-or-name>",
).version.decision_judge_config
# Keep the current questions and append one more
new_dj_version = client.evaluators.create_decision_judge_version(
evaluator="<evaluator-id-or-name>",
space=..., # Optional, required when evaluator is a name
commit_message="Add a conciseness question",
decision_judge_config={
"ai_integration_id": current.ai_integration_id,
"model_name": current.model_name,
"state_template": current.state_template,
"questions": [
*current.questions,
{
"name": "concise",
"type": "BOOLEAN",
"instructions": "Is the reply concise?",
},
],
},
)
resp = client.evaluators.delete_versions(
evaluator="<evaluator-id-or-name>",
space=..., # Optional
version_ids=..., # List of version IDs to delete (1-100)
)
Use client.tasks to manage online evaluation tasks that run continuously or on-demand against your production data.
resp = client.tasks.list(
name=..., # Optional, case-insensitive substring filter on task name
space=..., # Optional, space ID or name
project=..., # Optional, filter by project (name or ID)
dataset=..., # Optional, filter by dataset (name or ID)
task_type=..., # Optional, "TEMPLATE_EVALUATION", "CODE_EVALUATION", "RUN_EXPERIMENT" or "DECISION_JUDGE_EVALUATION"
limit=..., # Optional, defaults to 100
)
task_list = resp.tasks
Without task_type, the list leaves out RUN_EXPERIMENT and DECISION_JUDGE_EVALUATION tasks. Pass task_type="DECISION_JUDGE_EVALUATION" to list Decision Judge evaluation tasks. Name lookups in get, update, delete and trigger_run find them either way.
There are two task types for running evaluators, and one for server-side LLM experiments. Use the appropriate typed helper:
Template or code evaluation task — runs evaluators against spans from a project or experiments from a dataset.
from arize.tasks.types import TaskEvaluatorInput
# Project-based, continuous template evaluation task
task = client.tasks.create_evaluation_task(
name="Production Hallucination Check",
task_type="TEMPLATE_EVALUATION",
evaluators=[TaskEvaluatorInput(
evaluator_id="<evaluator-id>", # Required
column_mappings={"input": "attributes.input.value"}, # Optional
)],
project="<project-id-or-name>", # Required if not using dataset
space=..., # Optional, space ID or name
sampling_rate=1.0, # Optional, fraction of data to evaluate (0.0–1.0)
is_continuous=True, # Optional, run continuously on new data
query_filter=..., # Optional, filter expression for spans
)
# Dataset-based code evaluation task
task = client.tasks.create_evaluation_task(
name="My Code Evaluator Task",
task_type="CODE_EVALUATION",
evaluators=[TaskEvaluatorInput(evaluator_id="<evaluator-id>")],
dataset="<dataset-id-or-name>", # Required if not using project
experiment_ids=["<experiment-id>"], # Required when using dataset
)
Run experiment task — the server drives LLM calls using the configured AI integration. No local callable is required.
from arize.tasks.types import LlmGenerationRunConfig, RunConfiguration
task = client.tasks.create_run_experiment_task(
name="GPT-4o Baseline Task",
dataset="<dataset-id-or-name>",
run_configuration=RunConfiguration(
actual_instance=LlmGenerationRunConfig(
experiment_type="LLM_GENERATION",
ai_integration_id="<ai-integration-id>",
model_name="gpt-4o",
input_variable_format="F_STRING",
messages=[
{"role": "SYSTEM", "content": "You are a helpful assistant."},
{"role": "USER", "content": "Answer: {question}"},
],
)
),
space=..., # Optional, space ID or name
)
A Decision Judge evaluation task runs Decision Judge evaluators on the spans of a project. Every evaluator must be a Decision Judge evaluator (see Create a Decision Judge Evaluator). The task supports only the span shape, so datasets, experiments and query_filters are not accepted. The server returns 403 when Decision Judge evaluators are not enabled for the account.
from arize.tasks.types import SpanEvaluatorInput
task = client.tasks.create_decision_judge_evaluation_task(
name="Support Reply Judge",
project="<project-id-or-name>", # Required
space=..., # Optional, space ID or name, required when project is a name
evaluators=[
SpanEvaluatorInput(
evaluator_id="<decision-judge-evaluator-id>", # Required
column_mappings={"input": "attributes.input.value"}, # Optional
),
],
sampling_rate=0.5, # Optional
is_continuous=True, # Optional
query_filter="attributes.openinference.span.kind = 'LLM'", # Optional
)
run = client.tasks.trigger_run(
task=task.id,
data_start_time=..., # Required on the first run
data_end_time=...,
)
run = client.tasks.wait_for_run(run_id=run.id)
Each evaluator entry can also be a dict with the same fields. update, trigger_run and the run methods work on these tasks as on other evaluation tasks.
task = client.tasks.get(
task="<task-id-or-name>",
space=..., # Optional
)
You can trigger an on-demand run of a task. The returned TaskRun will initially have "pending" status. The method automatically dispatches based on the task type.
# Evaluation task (template_evaluation / code_evaluation)
run = client.tasks.trigger_run(
task="<task-id-or-name>",
space=..., # Optional
data_start_time=..., # Optional, start of data window
data_end_time=..., # Optional, end of data window
max_spans=..., # Optional, maximum spans to evaluate
override_evaluations=..., # Optional, re-evaluate already-evaluated spans
)
# Run experiment task — experiment_name is required
run = client.tasks.trigger_run(
task="<task-id-or-name>",
space=..., # Optional
experiment_name="GPT-4o Baseline v2", # Required for run_experiment tasks
max_examples=50, # Optional, limit number of dataset examples
tracing_metadata={"source": "api"}, # Optional, metadata for traces
)
# List runs for a task
runs_resp = client.tasks.list_runs(
task="<task-id-or-name>",
space=..., # Optional
status=..., # Optional, filter by "pending", "running", "completed", "failed", or "cancelled"
limit=..., # Optional, defaults to 100
)
# Get a specific run
run = client.tasks.get_run(run_id="<run-id>")
# Cancel a pending or running run
cancelled_run = client.tasks.cancel_run(run_id="<run-id>")
# Wait for a run to reach a terminal state (completed, failed, or cancelled)
finished_run = client.tasks.wait_for_run(
run_id="<run-id>",
poll_interval=5.0, # Optional, seconds between polls. Defaults to 5.0
timeout=600.0, # Optional, seconds before TimeoutError. Defaults to 600.0
)
Use client.projects to manage projects, which are namespaces for organizing tracing data.
resp = client.projects.list(
name=..., # Optional, case-insensitive substring filter on project name
space=..., # Optional, space ID or name
limit=..., # Optional, defaults to 100
)
project_list = resp.projects
project = client.projects.create(
name="<your-project-name>", # Must be unique within the space
space="<space-id-or-name>",
)
project = client.projects.get(
project="<project-id-or-name>",
space=..., # Optional
)
project = client.projects.update(
project="<project-id-or-name>",
space=..., # Optional, space ID or name (required when looking up by project name)
name="<new-project-name>", # Must be unique within the space
)
client.projects.delete(
project="<project-id-or-name>",
space=..., # Optional
)
Use client.organizations to manage organizations in the Arize platform.
resp = client.organizations.list(
name=..., # Optional, case-insensitive substring filter on organization name
limit=..., # Optional, defaults to 50 (max 100)
cursor=..., # Optional, pagination cursor from a previous response
)
org_list = resp.organizations
org = client.organizations.get(
organization="<organization-id-or-name>",
)
org = client.organizations.create(
name="<your-org-name>", # Must be unique within the account
description=..., # Optional
)
org = client.organizations.update(
organization="<organization-id-or-name>",
name=..., # Optional updated name
description=..., # Optional updated description (pass "" to clear)
)
Warning: This operation is irreversible. It permanently deletes the organization and all resources that belong to it, including all spaces and their contents.
client.organizations.delete(
organization="<organization-id-or-name>",
)
Add a user to an organization (or update their role if already a member). The user must already exist in the account.
from arize.organizations.types import CustomOrgRole, PredefinedOrgRole
# Predefined role (ADMIN, MEMBER, READ_ONLY, or ANNOTATOR)
membership = client.organizations.add_user(
organization="<organization-id-or-name>",
user_id="<user-id>",
role=PredefinedOrgRole(name="MEMBER"),
)
# Custom RBAC role
membership = client.organizations.add_user(
organization="<organization-id-or-name>",
user_id="<user-id>",
role=CustomOrgRole(id="<role-id>"),
)
Removes the user from the organization and all its child spaces.
client.organizations.remove_user(
organization="<organization-id-or-name>",
user_id="<user-id>",
)
Use client.spaces to create, retrieve, update, delete, and manage memberships
for Arize spaces.
Space responses from list, get, create, and update include organization_id,
the ID of the parent organization, when the server supports this field.
Older server versions may omit it.
resp = client.spaces.list(
organization_id=..., # Optional, filter to a specific organization
name=..., # Optional, case-insensitive substring filter
limit=..., # Optional, defaults to 50 (max 100)
cursor=..., # Optional, pagination cursor from a previous response
)
space_list = resp.spaces
Omit organization_id to list accessible spaces from all organizations. The SDK
sends this value as the REST query parameter org_id.
Accepts either a space name or ID.
space = client.spaces.get(space="<space-id-or-name>")
space = client.spaces.create(
name="my-space",
organization_id="<org-id>",
description="optional description",
is_private=True, # omit or set False for a public space (default)
)
Private spaces are visible only to their members and account/org/space admins.
A warning is logged when is_private=True to remind you to add members before
the space becomes inaccessible to other users.
Patch semantics: omitted fields preserve their current values.
space = client.spaces.update(
space="<space-id-or-name>",
name="new-name", # Optional
description="new desc", # Optional
is_private=False, # Optional; None preserves the current visibility
)
Irreversible — removes the space and all child resources (projects, datasets, monitors, dashboards, custom metrics, etc.).
client.spaces.delete(space="<space-id-or-name>")
Add a user to a space (or update their role if already a member). The user must already be a member of the space's parent organization.
from arize.spaces.types import CustomSpaceRole, PredefinedSpaceRole
# Predefined role (ADMIN, MEMBER, READ_ONLY, or ANNOTATOR)
membership = client.spaces.add_user(
space="<space-id-or-name>",
user_id="<user-id>",
role=PredefinedSpaceRole(name="MEMBER"),
)
# Custom RBAC role
membership = client.spaces.add_user(
space="<space-id-or-name>",
user_id="<user-id>",
role=CustomSpaceRole(id="<role-id>"),
)
client.spaces.remove_user(
space="<space-id-or-name>",
user_id="<user-id>",
)
Use client.users to manage users in the Arize platform.
Note: Unlike organizations, users are identified by opaque ID only — not by name. User display names are not unique within an account, so all methods require the user's ID.
resp = client.users.list(
email=..., # Optional, case-insensitive partial match on email
status=..., # Optional, list of statuses: "ACTIVE", "INVITED", "EXPIRED"
limit=..., # Optional, defaults to 50 (max 100)
cursor=..., # Optional, pagination cursor from a previous response
)
user_list = resp.users
Accepts either a user ID or an email address. When an email is provided, it
is resolved to the matching user (returns None if no match is found).
# by ID
user = client.users.get(user="<user-id>")
# by email
user = client.users.get(user="jane.smith@example.com")
from arize.users.types import PredefinedUserRole
user = client.users.create(
name="Jane Smith",
email="jane.smith@example.com",
role=PredefinedUserRole(name="MEMBER"), # ADMIN, MEMBER, ANNOTATOR
invite_mode="EMAIL_LINK", # NONE, EMAIL_LINK, TEMPORARY_PASSWORD
)
user = client.users.update(
user_id="<user-id>",
name=..., # Optional updated display name
is_developer=..., # Optional, grant or revoke developer permissions
)
Warning: This operation soft-deletes the user and cascades to organization memberships, space memberships, API keys, and role bindings.
client.users.delete(
user_id="<user-id>",
)
Note: This is a beta endpoint. Enable it via the pre-release opt-in.
Delete multiple users in a single call by ID, by email, or both. Returns a list of
BulkUserDeletionResult objects recording the outcome of each attempt.
from arize import ArizeClient, BulkUserDeletionResult
client = ArizeClient(api_key=API_KEY)
results = client.users.bulk_delete(
user_ids=["usr_abc123", "usr_def456"], # Optional
emails=["alice@example.com"], # Optional — resolved to IDs automatically
)
for result in results:
print(result.id, result.status, result.error)
Each BulkUserDeletionResult has:
| Field | Type | Description |
|---|---|---|
id |
str |
User ID, or the email if the user could not be resolved |
status |
DeletionStatus |
"deleted", "not_found", or "failed" |
error |
str | None |
Error message when status is "not_found" or "failed" |
client.users.resend_invitation(
user_id="<user-id>", # Must be in "INVITED" status
)
Triggers a password-reset email. The user must authenticate via password (not SSO/SAML) and must have already verified their account.
client.users.reset_password(
user_id="<user-id>",
)
Use client.api_keys to manage API keys in the Arize platform. Two key types are supported: user keys (authenticate as the creating user) and service keys (scoped to a specific space).
Note: The raw key value is only returned at creation time. Store it securely.
resp = client.api_keys.list(
key_type=..., # Optional, "USER" or "SERVICE"
status=..., # Optional, "ACTIVE" or "REVOKED"
space=..., # Optional, space ID or name (filters to service keys for that space)
user_id=..., # Optional, filter by user (admin only for user keys)
limit=..., # Optional, defaults to 50 (max 100)
cursor=..., # Optional, pagination cursor from a previous response
)
api_key_list = resp.api_keys
Creates a user-type key that authenticates as the creating user. For space-scoped service keys, use create_service_key() instead.
created = client.api_keys.create(
name="my-api-key",
description=..., # Optional
expires_at=..., # Optional, must be a future timestamp
)
raw_key = created.key # Store this securely — only returned once
Creates a service-type key scoped to a specific space, backed by a dedicated bot user. Role assignments default to space_role="MEMBER", org_role="READ_ONLY", account_role="MEMBER" when omitted. All roles must be at or below the caller's own privilege level.
created = client.api_keys.create_service_key(
name="my-service-key",
space="<space-id-or-name>",
description=..., # Optional
expires_at=..., # Optional
space_role=..., # Optional, "ADMIN" | "MEMBER" | "READ_ONLY"
org_role=..., # Optional, "ADMIN" | "MEMBER" | "READ_ONLY"
account_role=..., # Optional, "ADMIN" | "MEMBER"
)
raw_key = created.key # Store this securely — only returned once
Sets the key's status to revoked and deactivates it immediately. Revoking an already-revoked key is a no-op and still succeeds. Returns None on success.
client.api_keys.revoke(
api_key_id="<api-key-id>",
)
Atomically revokes the old key and issues a replacement with the same name, description, type, and scope.
Use grace_period_seconds to keep the old key valid briefly while your
services rotate to the new key.
refreshed = client.api_keys.refresh(
api_key_id="<api-key-id>",
expires_at=..., # Optional, new expiration for the replacement key
grace_period_seconds=..., # Optional, old key remains valid for N seconds
)
new_raw_key = refreshed.key # Store securely — only returned once
Common patterns:
# Immediate invalidation of old key (default)
refreshed = client.api_keys.refresh(api_key_id="<api-key-id>")
# Graceful rotation window (old key valid for 5 minutes)
refreshed = client.api_keys.refresh(
api_key_id="<api-key-id>",
grace_period_seconds=300,
)
# Graceful rotation + expiry on the replacement key
refreshed = client.api_keys.refresh(
api_key_id="<api-key-id>",
expires_at=...,
grace_period_seconds=300,
)
Notes:
expires_at applies to the replacement key.grace_period_seconds applies to the old key.Use client.annotation_configs to manage annotation configurations that define scoring schemas for human feedback.
resp = client.annotation_configs.list(
name=..., # Optional, case-insensitive substring filter
space=..., # Optional, space ID or name
limit=..., # Optional, defaults to 100
)
config_list = resp.annotation_configs
Three config types are supported: continuous (a numeric score in a range), categorical (a fixed set of labeled values), and freeform (open-ended text with no scale). Use whichever create_* method matches the config type you want.
from arize.annotation_configs.types import CategoricalAnnotationValue, OptimizationDirection
# Continuous (numeric) annotation config
config = client.annotation_configs.create_continuous(
name="quality-score",
space="<space-id-or-name>",
minimum_score=0.0,
maximum_score=1.0,
optimization_direction=OptimizationDirection.MAXIMIZE,
)
# Categorical annotation config
config = client.annotation_configs.create_categorical(
name="correctness",
space="<space-id-or-name>",
values=[
CategoricalAnnotationValue(label="correct", score=1),
CategoricalAnnotationValue(label="incorrect", score=0),
],
optimization_direction=OptimizationDirection.MAXIMIZE,
)
# Freeform (open-ended text) annotation config
config = client.annotation_configs.create_freeform(
name="reviewer-notes",
space="<space-id-or-name>",
)
config = client.annotation_configs.get(
annotation_config="<config-id-or-name>",
space=..., # Optional
)
Only the fields you pass are changed; omitted fields are left unchanged. Use the method matching the stored config's type, since the config type is immutable.
from arize.annotation_configs.types import CategoricalAnnotationValue, OptimizationDirection
# Rename a freeform config
config = client.annotation_configs.update_freeform(
annotation_config="<config-id-or-name>",
space=..., # Optional, required when passing a name
name="renamed-config",
)
# Replace the label set of a categorical config
config = client.annotation_configs.update_categorical(
annotation_config="<config-id-or-name>",
values=[
CategoricalAnnotationValue(label="good", score=1),
CategoricalAnnotationValue(label="bad", score=0),
],
optimization_direction=OptimizationDirection.MAXIMIZE,
)
# Adjust the score bounds of a continuous config
config = client.annotation_configs.update_continuous(
annotation_config="<config-id-or-name>",
minimum_score=0.0,
maximum_score=10.0,
)
client.annotation_configs.delete(
annotation_config="<config-id-or-name>",
space=..., # Optional
)
Use client.resource_restrictions to manage resource restrictions. Restricting a resource prevents roles bound at higher hierarchy levels (space, org, account) from granting access to it — access must be granted directly on the resource. Only space admins or users with the PROJECT_RESTRICT permission can manage restrictions. Currently only PROJECT resources are supported.
Note: Resource restrictions are a beta API and may change without notice.
You can list all resource restrictions that you are permitted to manage using client.resource_restrictions.list(). You can use the limit parameter to specify the maximum number of restrictions desired per page and you can specify resource_type to filter by resource type (currently only PROJECT is supported). You can use cursor for pagination, pass pagination.next_cursor from a previous response. Results are authorization-filtered after each page is fetched, so a page may contain fewer items than limit (or none) while pagination.has_more is still True; keep paging until pagination.has_more is False.
from arize.resource_restrictions.types import ResourceRestrictionType
resp = client.resource_restrictions.list(
resource_type=ResourceRestrictionType.PROJECT, # Optional filter
limit=..., # Optional, defaults to 50
cursor=..., # Optional, pagination cursor from a previous response
)
Idempotent. Restricting an already-restricted resource returns the existing restriction without error.
restriction = client.resource_restrictions.restrict(
resource_id="<project-id>",
)
client.resource_restrictions.unrestrict(
resource_id="<project-id>",
)
Use client.ai_integrations to manage external LLM provider integrations used in the Arize Playground and online evaluations.
resp = client.ai_integrations.list(
name=..., # Optional, case-insensitive substring filter
space=..., # Optional, space ID or name
provider=..., # Optional, e.g. AiIntegrationProvider.CLOUDFLARE
limit=..., # Optional, defaults to 100
)
integration_list = resp.ai_integrations
TYPE_SAFE_AI, PERPLEXITY, and CLOUDFLARE integrations are used only by
Decision Judge evaluators. They are listed only when provider names them.
from arize.ai_integrations.types import AiIntegrationProvider, AiIntegrationAuthType
integration = client.ai_integrations.create(
name="my-openai-integration",
provider=AiIntegrationProvider.OPEN_AI,
api_key="<your-provider-api-key>",
base_url=..., # Optional, custom base URL
model_names=..., # Optional, list of enabled model names
enable_default_models=True, # Optional
)
A Cloudflare integration (Decision Judge only) takes an API token and the account ID as provider metadata, and no base URL, models, or headers:
from arize.ai_integrations.types import CloudflareProviderMetadataRequest
integration = client.ai_integrations.create(
name="my-clou
Arize-ai/client_python is a public repo on GitHub by Arize-ai. A Python client to interact with Arize API.
It declares the Apache-2.0 license on GitHub.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
Its last change on GitHub was on 2026-11-17. Take My Repo does not judge from that whether it is maintained.
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.

Aqueduct is no longer being maintained. Aqueduct allows you to run LLM and ML workloads on any cloud infrastructure.
Free MLOps course from DataTalks.Club. Register here 👇🏼 to get notified about the next cohort
A free, powerful, multi-purpose tool that helps you monitor system resources, debug software and detect malware. Brought to you by Winsider Seminars & Solutions, Inc. @ https://windows-internals.com
[Not Actively Maintained] Whitebox is an open source E2E ML monitoring platform with edge capabilities that plays nicely with kubernetes

Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
OpenTelemetry Instrumentation for AI Observability
Agent skills for Arize — datasets, experiments, and traces via the ax CLI
Trace coding harnesses like claude-code, cursor, and codex to Arize AX or Phoenix
A specification for OpenInference, a semantic mapping of ML inferences