Signals

For instructions on how to authenticate to use this endpoint, see API overview.

Endpoints

`PATCH`
`DELETE`
`POST`
`POST`
`POST`
`POST`
`POST`
`GET`
`GET`
`POST`
`POST`
`GET`
`GET`
`GET`
`GET`
`POST`
`GET`
`GET`
`GET`
`GET`

← Previous pageNext page →

Update signals scout config

Tune one scout: change its schedule (rolling run_interval_minutes, or a cron run_cron_schedule that takes precedence when set), enabled, emit (dry-run) posture, network_access (trusted-domain allowlist vs full access for the scout's sandbox), or output destinations. skill_name is fixed. Enabling records enabled_by and is activity-logged since it drives spend.

Required API key scopes

signal_scout:write

Path parameters

id
string

Request parameters

display_name
string
enabled
boolean
emit
boolean
run_interval_minutes
integer
run_cron_schedule
string | null
output_destinations
structured_output_schema
object | null
network_access
model
string | null
auto_pause_exempt
boolean
tags
array
mcp_gateway_server_ids
array
repositories
array
write_scopes
array
precheck_query
string | null
precheck_disabled
boolean
suggestion_id
string
lifecycle_locked
boolean
allowed_mcp_tools
array | null
tool_preset

Response

Example request

PATCH /api/projects/:project_id/signals/scout/configs/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X PATCH \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/\
	-d display_name="string"
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.patch(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
	data={
		"display_name": "string"
	}
).json()

Example response

Status 200 Updated config.

RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "skill_name": "string",
  "description": "string",
  "display_name": "string",
  "scout_origin": "canonical",
  "scout_role": "specialist",
  "deprecation": {
    "phase": "announced",
    "reason": "string",
    "superseded_by": "string",
    "sunset_at": "2019-08-24T14:15:22Z"
  },
  "owners": [
    {
      "id": 0,
      "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
      "distinct_id": "string",
      "first_name": "string",
      "last_name": "string",
      "email": "user@example.com",
      "is_email_verified": true,
      "hedgehog_config": {},
      "role_at_organization": "engineering"
    }
  ],
  "enabled": true,
  "status": "active",
  "pause_reason": "no_output",
  "managed_by": "team",
  "emit": true,
  "run_interval_minutes": 30,
  "run_cron_schedule": "string",
  "output_destinations": {
    "slack": {
      "integration_id": 1,
      "channel": "string",
      "users": [
        "string"
      ],
      "thread_reports": true
    },
    "webhook": {
      "hog_function_id": "string"
    },
    "inbox": true
  },
  "structured_output_schema": {},
  "network_access": "trusted",
  "model": "string",
  "mcp_gateway_server_ids": [
    "497f6eca-6276-4993-bfeb-53cbbbba6f08"
  ],
  "repositories": [
    "string"
  ],
  "write_scopes": [
    "string"
  ],
  "lifecycle_locked": true,
  "allowed_mcp_tools": [
    "string"
  ],
  "tool_preset": "string",
  "precheck_query": "string",
  "precheck_disabled": true,
  "effective_precheck_query": "string",
  "precheck_query_source": "config",
  "last_precheck_at": "2019-08-24T14:15:22Z",
  "last_precheck_outcome": "run",
  "last_precheck_row_count": 0,
  "last_run_at": "2019-08-24T14:15:22Z",
  "consecutive_failure_count": 0,
  "status_changed_at": "2019-08-24T14:15:22Z",
  "status_changed_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {},
    "role_at_organization": "engineering"
  },
  "auto_pause_exempt": true,
  "tags": [
    "string"
  ],
  "source_product": "string",
  "source_id": "string",
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z"
}

Status 400 Invalid fields, or enabling would exceed the project's enabled-scouts maximum.

Status 404 Config not found for this project.

Delete signals scout config

Delete one scout config by its id, removing the per-(team, skill) schedule/emit row outright. The point is cleaning up an orphaned config whose skill was archived or deleted — it lingers in list with an empty description, never runs (the coordinator skips it and the skill can't load), but can't otherwise be removed over the API. Deletion is activity-logged. Note: auto-registration only scans live signals-scout-* skills, so a config deleted for one of those is back on the coordinator's next tick. A scout under any other name does not come back on its own: its config stays deleted until you re-register it, and its skill still reads as a scout meanwhile. To retire a live scout, archive its skill (or set enabled=false to make it inert) rather than deleting the config. A scout whose scout_role is operational cannot be deleted: it is part of the self-driving system rather than the project's own fleet.

Required API key scopes

signal_scout:write

Path parameters

id
string

Example request

DELETE /api/projects/:project_id/signals/scout/configs/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl  -X DELETE \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.delete(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
)

Example response

Status 204 Config deleted.

Status 400 This scout is operational and cannot be deleted.

Status 404 Config not found for this project.

Create signals scout config precheck

Run a scout's pre-check query once and return its rows, without starting a run and without saving anything. The query gets the same {since} and {now} values the next scheduled run would get, so the result says whether that run would start or skip. Pass precheck_query to try a query before you save it, or omit it to try the effective one: the saved query, or the default the scout's skill ships. A query error comes back in the error field with a 200, because a scheduled run treats it as a reason to run.

Required API key scopes

signal_scout:readquery:read

Path parameters

id
string

Request parameters

precheck_query
string | null

Response

Example request

POST /api/projects/:project_id/signals/scout/configs/:id/precheck_test
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/precheck_test/\
	-d precheck_query="string"
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.post(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/precheck_test/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
	data={
		"precheck_query": "string"
	}
).json()

Example response

Status 200 What the pre-check returns now, and whether a scheduled run would start.

RESPONSE
{
  "would_run": true,
  "reason": "rows",
  "since": "2019-08-24T14:15:22Z",
  "now": "2019-08-24T14:15:22Z",
  "interval_minutes": 0,
  "row_count": 0,
  "columns": [
    "string"
  ],
  "rows_text": "string",
  "error": "string",
  "acting_user": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {},
    "role_at_organization": "engineering"
  }
}

Status 400 The scout has no saved pre-check and the request gives no query.

Status 404 Config not found for this project.

Create signals scout config

Dispatch one on-demand run of this scout immediately, regardless of its schedule. Useful to test a scout right after authoring it, or to refresh its findings on demand. The run executes asynchronously on the worker and inherits every guard the scheduled path has: it is forbidden if scouts are not enabled for the project (403), and skipped if self-driving is paused at the project's pull request limit, or the project is over its daily report limit or daily run budget (429), or a run for this scout is already in progress (409). A manual run counts against the same daily run budget as scheduled runs, so repeated manual runs of the same scout can exhaust the project's daily allowance. A manual run does not change the scout's schedule or last_run_at. A disabled scout can still be run this way (to test before enabling). Pass an optional note to steer this one run without leaving a scout note that would steer every later run too. Returns immediately with the workflow id: poll the scout's runs for the result.

Required API key scopes

signal_scout:write

Path parameters

id
string

Request parameters

note
string

Response

Example request

POST /api/projects/:project_id/signals/scout/configs/:id/run
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/run/\
	-d note="string"
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.post(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/run/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
	data={
		"note": "string"
	}
).json()

Example response

Status 202 A run was dispatched. It executes asynchronously; poll the scout's runs for the result.

RESPONSE
{
  "skill_name": "string",
  "workflow_id": "string",
  "started": true
}

Status 400 The note is longer than the limit.

Status 403 Signals scouts are not enabled for this project, or the caller may not steer a run with a note.

Status 404 Config not found for this project (or the scout is withheld).

Status 409 A run for this scout is already in progress.

Status 429 Self-driving is paused at the project's pull request limit, or the project is over its daily report limit or its daily scout run budget; try again later.

Create signals scout config

Run a prompt, model, or effort variant against live data with private memory and report capture.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Request parameters

launch_id
string
context_id
string
variant
string
skill_body
string
model
string
reasoning_effort
string
note
string

Response

Example request

POST /api/projects/:project_id/signals/scout/configs/:id/trial
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial/\
	-d launch_id="string"
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.post(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
	data={
		"launch_id": "string"
	}
).json()

Example response

Status 202

RESPONSE
{
  "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
  "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
  "workflow_id": "string",
  "model": "string",
  "reasoning_effort": "string",
  "variant": "string"
}

Create signals scout config trial comparison

Freeze variants and the reviewed rubric, then run scouts and judge their results in the background.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Request parameters

comparison_id
string
baseline_variant_id
string
variants
array
note
string
expected_skill_version
integer

Response

Example request

POST /api/projects/:project_id/signals/scout/configs/:id/trial_comparison
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_comparison/\
	-d comparison_id="string",\
	-d baseline_variant_id="string",\
	-d variants="array"
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.post(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_comparison/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
	data={
		"comparison_id": "string",
		"baseline_variant_id": "string",
		"variants": "array"
	}
).json()

Example response

Status 202

RESPONSE
{
  "comparison_id": "3b27d395-1e6b-479b-baa5-81329cfca144",
  "config_id": "d1d31429-d888-4f1c-b9c1-4e842f9bce5b",
  "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
  "created_at": "2019-08-24T14:15:22Z",
  "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
  "rubric_revision": 0,
  "variants": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "label": "string",
      "launch_ids": [
        "497f6eca-6276-4993-bfeb-53cbbbba6f08"
      ],
      "model": "string",
      "reasoning_effort": "string",
      "skill_body_sha256": "string"
    }
  ],
  "status": "not_started",
  "archived": true,
  "error": "string",
  "evaluation": {
    "request": {
      "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
      "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
      "variants": [
        {
          "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
          "label": "string",
          "launch_ids": [
            "497f6eca-6276-4993-bfeb-53cbbbba6f08"
          ]
        }
      ],
      "rubric_source": "saved"
    },
    "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
    "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
    "status": "pending",
    "error": "string",
    "report": {
      "version": 1,
      "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
      "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
      "created_at": "2019-08-24T14:15:22Z",
      "completed_at": "2019-08-24T14:15:22Z",
      "summary": "string",
      "outcome": null,
      "rubric_source": "saved",
      "rubric_revision": 0,
      "rubric_reference_context": null,
      "rubric_reference_generation_id": null,
      "criteria": [
        {
          "id": "string",
          "title": "string",
          "description": "string",
          "pass_condition": "string",
          "applicability": "string"
        }
      ],
      "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
      "judge_model": "string",
      "judge_prompt_version": "string",
      "variants": [
        {
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "label": "string",
          "is_baseline": true,
          "total_runs": 0,
          "judged_runs": 0,
          "excluded_runs": 0,
          "judge_errors": 0,
          "score": 0,
          "coverage": 0,
          "baseline_delta": null,
          "criteria": [
            {
              "criterion_id": "string",
              "passed": 0,
              "failed": 0,
              "unknown": 0,
              "not_applicable": 0,
              "pass_rate": 0,
              "coverage": 0,
              "baseline_delta": null
            }
          ]
        }
      ],
      "runs": [
        {
          "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "status": "judged",
          "score": null,
          "coverage": null,
          "summary": "string",
          "criteria": [
            {
              "criterion_id": "string",
              "verdict": "pass",
              "reason": "string",
              "confidence": "low",
              "evidence": [
                {
                  "source_id": "string",
                  "quote": "string"
                }
              ]
            }
          ],
          "error": null,
          "input_tokens": null,
          "output_tokens": null
        }
      ],
      "evidence": [
        {
          "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec",
          "task_id": "736fde4d-9029-4915-8189-01353d6982cb",
          "task_run_id": "7f0ef9d7-e7b5-4d52-8247-ce8351652f8b",
          "execution_status": "string",
          "exclusion_reason": null,
          "model": "string",
          "runtime_adapter": "claude",
          "service_tier": null,
          "reasoning_effort": "string",
          "skill_body_sha256": "string",
          "input_tokens": null,
          "output_tokens": null,
          "files": [
            {
              "id": "string",
              "kind": "instructions",
              "filename": "string",
              "sha256": "string",
              "size_bytes": 0
            }
          ],
          "sources": [
            {
              "id": "string",
              "kind": "instructions",
              "text": "string"
            }
          ],
          "limitations": [
            "string"
          ]
        }
      ],
      "limitations": [
        "string"
      ]
    }
  }
}

Create signals scout config trial comparison

Hide a finished trial from history or restore it without deleting results or starting any work.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Request parameters

comparison_id
string
archived
boolean

Response

Example request

POST /api/projects/:project_id/signals/scout/configs/:id/trial_comparison_archive
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_comparison_archive/\
	-d comparison_id="string",\
	-d archived="boolean"
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.post(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_comparison_archive/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
	data={
		"comparison_id": "string",
		"archived": "boolean"
	}
).json()

Example response

Status 200

RESPONSE
{
  "comparison_id": "3b27d395-1e6b-479b-baa5-81329cfca144",
  "config_id": "d1d31429-d888-4f1c-b9c1-4e842f9bce5b",
  "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
  "created_at": "2019-08-24T14:15:22Z",
  "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
  "rubric_revision": 0,
  "variants": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "label": "string",
      "launch_ids": [
        "497f6eca-6276-4993-bfeb-53cbbbba6f08"
      ],
      "model": "string",
      "reasoning_effort": "string",
      "skill_body_sha256": "string"
    }
  ],
  "status": "not_started",
  "archived": true,
  "error": "string",
  "evaluation": {
    "request": {
      "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
      "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
      "variants": [
        {
          "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
          "label": "string",
          "launch_ids": [
            "497f6eca-6276-4993-bfeb-53cbbbba6f08"
          ]
        }
      ],
      "rubric_source": "saved"
    },
    "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
    "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
    "status": "pending",
    "error": "string",
    "report": {
      "version": 1,
      "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
      "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
      "created_at": "2019-08-24T14:15:22Z",
      "completed_at": "2019-08-24T14:15:22Z",
      "summary": "string",
      "outcome": null,
      "rubric_source": "saved",
      "rubric_revision": 0,
      "rubric_reference_context": null,
      "rubric_reference_generation_id": null,
      "criteria": [
        {
          "id": "string",
          "title": "string",
          "description": "string",
          "pass_condition": "string",
          "applicability": "string"
        }
      ],
      "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
      "judge_model": "string",
      "judge_prompt_version": "string",
      "variants": [
        {
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "label": "string",
          "is_baseline": true,
          "total_runs": 0,
          "judged_runs": 0,
          "excluded_runs": 0,
          "judge_errors": 0,
          "score": 0,
          "coverage": 0,
          "baseline_delta": null,
          "criteria": [
            {
              "criterion_id": "string",
              "passed": 0,
              "failed": 0,
              "unknown": 0,
              "not_applicable": 0,
              "pass_rate": 0,
              "coverage": 0,
              "baseline_delta": null
            }
          ]
        }
      ],
      "runs": [
        {
          "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "status": "judged",
          "score": null,
          "coverage": null,
          "summary": "string",
          "criteria": [
            {
              "criterion_id": "string",
              "verdict": "pass",
              "reason": "string",
              "confidence": "low",
              "evidence": [
                {
                  "source_id": "string",
                  "quote": "string"
                }
              ]
            }
          ],
          "error": null,
          "input_tokens": null,
          "output_tokens": null
        }
      ],
      "evidence": [
        {
          "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec",
          "task_id": "736fde4d-9029-4915-8189-01353d6982cb",
          "task_run_id": "7f0ef9d7-e7b5-4d52-8247-ce8351652f8b",
          "execution_status": "string",
          "exclusion_reason": null,
          "model": "string",
          "runtime_adapter": "claude",
          "service_tier": null,
          "reasoning_effort": "string",
          "skill_body_sha256": "string",
          "input_tokens": null,
          "output_tokens": null,
          "files": [
            {
              "id": "string",
              "kind": "instructions",
              "filename": "string",
              "sha256": "string",
              "size_bytes": 0
            }
          ],
          "sources": [
            {
              "id": "string",
              "kind": "instructions",
              "text": "string"
            }
          ],
          "limitations": [
            "string"
          ]
        }
      ],
      "limitations": [
        "string"
      ]
    }
  }
}

Retrieve signals scout config trial comparison

Read recent comparisons, including those saved before any scout run started.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Query parameters

cursor
string
include_archived
boolean
Default: false
limit
integer
Default: 30

Response

Example request

GET /api/projects/:project_id/signals/scout/configs/:id/trial_comparison_history
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_comparison_history/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_comparison_history/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200

RESPONSE
{
  "results": [
    {
      "comparison_id": "3b27d395-1e6b-479b-baa5-81329cfca144",
      "config_id": "d1d31429-d888-4f1c-b9c1-4e842f9bce5b",
      "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
      "created_at": "2019-08-24T14:15:22Z",
      "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
      "rubric_revision": 0,
      "variants": [
        {
          "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
          "label": "string",
          "launch_ids": [
            "497f6eca-6276-4993-bfeb-53cbbbba6f08"
          ],
          "model": "string",
          "reasoning_effort": "string",
          "skill_body_sha256": "string"
        }
      ],
      "status": "not_started",
      "archived": true,
      "error": "string",
      "evaluation": {
        "request": {
          "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
          "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
          "variants": [
            {
              "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
              "label": "string",
              "launch_ids": [
                "497f6eca-6276-4993-bfeb-53cbbbba6f08"
              ]
            }
          ],
          "rubric_source": "saved"
        },
        "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
        "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
        "status": "pending",
        "error": "string",
        "report": {
          "version": 1,
          "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
          "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
          "created_at": "2019-08-24T14:15:22Z",
          "completed_at": "2019-08-24T14:15:22Z",
          "summary": "string",
          "outcome": null,
          "rubric_source": "saved",
          "rubric_revision": 0,
          "rubric_reference_context": null,
          "rubric_reference_generation_id": null,
          "criteria": [
            {
              "id": "string",
              "title": "string",
              "description": "string",
              "pass_condition": "string",
              "applicability": "string"
            }
          ],
          "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
          "judge_model": "string",
          "judge_prompt_version": "string",
          "variants": [
            {
              "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
              "label": "string",
              "is_baseline": true,
              "total_runs": 0,
              "judged_runs": 0,
              "excluded_runs": 0,
              "judge_errors": 0,
              "score": 0,
              "coverage": 0,
              "baseline_delta": null,
              "criteria": [
                {
                  "criterion_id": "string",
                  "passed": 0,
                  "failed": 0,
                  "unknown": 0,
                  "not_applicable": 0,
                  "pass_rate": 0,
                  "coverage": 0,
                  "baseline_delta": null
                }
              ]
            }
          ],
          "runs": [
            {
              "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
              "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
              "status": "judged",
              "score": null,
              "coverage": null,
              "summary": "string",
              "criteria": [
                {
                  "criterion_id": "string",
                  "verdict": "pass",
                  "reason": "string",
                  "confidence": "low",
                  "evidence": [
                    {
                      "source_id": "string",
                      "quote": "string"
                    }
                  ]
                }
              ],
              "error": null,
              "input_tokens": null,
              "output_tokens": null
            }
          ],
          "evidence": [
            {
              "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
              "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
              "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec",
              "task_id": "736fde4d-9029-4915-8189-01353d6982cb",
              "task_run_id": "7f0ef9d7-e7b5-4d52-8247-ce8351652f8b",
              "execution_status": "string",
              "exclusion_reason": null,
              "model": "string",
              "runtime_adapter": "claude",
              "service_tier": null,
              "reasoning_effort": "string",
              "skill_body_sha256": "string",
              "input_tokens": null,
              "output_tokens": null,
              "files": [
                {
                  "id": "string",
                  "kind": "instructions",
                  "filename": "string",
                  "sha256": "string",
                  "size_bytes": 0
                }
              ],
              "sources": [
                {
                  "id": "string",
                  "kind": "instructions",
                  "text": "string"
                }
              ],
              "limitations": [
                "string"
              ]
            }
          ],
          "limitations": [
            "string"
          ]
        }
      }
    }
  ],
  "has_more": true,
  "next_cursor": "string"
}

Retrieve signals scout config trial comparison

Read comparison progress and its saved report without starting any scout or judge calls.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Query parameters

comparison_id
string

Response

Example request

GET /api/projects/:project_id/signals/scout/configs/:id/trial_comparison_result
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_comparison_result/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_comparison_result/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200

RESPONSE
{
  "comparison_id": "3b27d395-1e6b-479b-baa5-81329cfca144",
  "config_id": "d1d31429-d888-4f1c-b9c1-4e842f9bce5b",
  "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
  "created_at": "2019-08-24T14:15:22Z",
  "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
  "rubric_revision": 0,
  "variants": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "label": "string",
      "launch_ids": [
        "497f6eca-6276-4993-bfeb-53cbbbba6f08"
      ],
      "model": "string",
      "reasoning_effort": "string",
      "skill_body_sha256": "string"
    }
  ],
  "status": "not_started",
  "archived": true,
  "error": "string",
  "evaluation": {
    "request": {
      "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
      "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
      "variants": [
        {
          "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
          "label": "string",
          "launch_ids": [
            "497f6eca-6276-4993-bfeb-53cbbbba6f08"
          ]
        }
      ],
      "rubric_source": "saved"
    },
    "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
    "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
    "status": "pending",
    "error": "string",
    "report": {
      "version": 1,
      "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
      "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
      "created_at": "2019-08-24T14:15:22Z",
      "completed_at": "2019-08-24T14:15:22Z",
      "summary": "string",
      "outcome": null,
      "rubric_source": "saved",
      "rubric_revision": 0,
      "rubric_reference_context": null,
      "rubric_reference_generation_id": null,
      "criteria": [
        {
          "id": "string",
          "title": "string",
          "description": "string",
          "pass_condition": "string",
          "applicability": "string"
        }
      ],
      "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
      "judge_model": "string",
      "judge_prompt_version": "string",
      "variants": [
        {
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "label": "string",
          "is_baseline": true,
          "total_runs": 0,
          "judged_runs": 0,
          "excluded_runs": 0,
          "judge_errors": 0,
          "score": 0,
          "coverage": 0,
          "baseline_delta": null,
          "criteria": [
            {
              "criterion_id": "string",
              "passed": 0,
              "failed": 0,
              "unknown": 0,
              "not_applicable": 0,
              "pass_rate": 0,
              "coverage": 0,
              "baseline_delta": null
            }
          ]
        }
      ],
      "runs": [
        {
          "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "status": "judged",
          "score": null,
          "coverage": null,
          "summary": "string",
          "criteria": [
            {
              "criterion_id": "string",
              "verdict": "pass",
              "reason": "string",
              "confidence": "low",
              "evidence": [
                {
                  "source_id": "string",
                  "quote": "string"
                }
              ]
            }
          ],
          "error": null,
          "input_tokens": null,
          "output_tokens": null
        }
      ],
      "evidence": [
        {
          "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec",
          "task_id": "736fde4d-9029-4915-8189-01353d6982cb",
          "task_run_id": "7f0ef9d7-e7b5-4d52-8247-ce8351652f8b",
          "execution_status": "string",
          "exclusion_reason": null,
          "model": "string",
          "runtime_adapter": "claude",
          "service_tier": null,
          "reasoning_effort": "string",
          "skill_body_sha256": "string",
          "input_tokens": null,
          "output_tokens": null,
          "files": [
            {
              "id": "string",
              "kind": "instructions",
              "filename": "string",
              "sha256": "string",
              "size_bytes": 0
            }
          ],
          "sources": [
            {
              "id": "string",
              "kind": "instructions",
              "text": "string"
            }
          ],
          "limitations": [
            "string"
          ]
        }
      ],
      "limitations": [
        "string"
      ]
    }
  }
}

Create signals scout config trial comparison

Recover the same comparison without repeating saved scout runs or judge attempts.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Request parameters

comparison_id
string

Response

Example request

POST /api/projects/:project_id/signals/scout/configs/:id/trial_comparison_resume
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_comparison_resume/\
	-d comparison_id="string"
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.post(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_comparison_resume/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
	data={
		"comparison_id": "string"
	}
).json()

Example response

Status 202

RESPONSE
{
  "comparison_id": "3b27d395-1e6b-479b-baa5-81329cfca144",
  "config_id": "d1d31429-d888-4f1c-b9c1-4e842f9bce5b",
  "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
  "created_at": "2019-08-24T14:15:22Z",
  "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
  "rubric_revision": 0,
  "variants": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "label": "string",
      "launch_ids": [
        "497f6eca-6276-4993-bfeb-53cbbbba6f08"
      ],
      "model": "string",
      "reasoning_effort": "string",
      "skill_body_sha256": "string"
    }
  ],
  "status": "not_started",
  "archived": true,
  "error": "string",
  "evaluation": {
    "request": {
      "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
      "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
      "variants": [
        {
          "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
          "label": "string",
          "launch_ids": [
            "497f6eca-6276-4993-bfeb-53cbbbba6f08"
          ]
        }
      ],
      "rubric_source": "saved"
    },
    "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
    "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
    "status": "pending",
    "error": "string",
    "report": {
      "version": 1,
      "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
      "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
      "created_at": "2019-08-24T14:15:22Z",
      "completed_at": "2019-08-24T14:15:22Z",
      "summary": "string",
      "outcome": null,
      "rubric_source": "saved",
      "rubric_revision": 0,
      "rubric_reference_context": null,
      "rubric_reference_generation_id": null,
      "criteria": [
        {
          "id": "string",
          "title": "string",
          "description": "string",
          "pass_condition": "string",
          "applicability": "string"
        }
      ],
      "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
      "judge_model": "string",
      "judge_prompt_version": "string",
      "variants": [
        {
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "label": "string",
          "is_baseline": true,
          "total_runs": 0,
          "judged_runs": 0,
          "excluded_runs": 0,
          "judge_errors": 0,
          "score": 0,
          "coverage": 0,
          "baseline_delta": null,
          "criteria": [
            {
              "criterion_id": "string",
              "passed": 0,
              "failed": 0,
              "unknown": 0,
              "not_applicable": 0,
              "pass_rate": 0,
              "coverage": 0,
              "baseline_delta": null
            }
          ]
        }
      ],
      "runs": [
        {
          "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "status": "judged",
          "score": null,
          "coverage": null,
          "summary": "string",
          "criteria": [
            {
              "criterion_id": "string",
              "verdict": "pass",
              "reason": "string",
              "confidence": "low",
              "evidence": [
                {
                  "source_id": "string",
                  "quote": "string"
                }
              ]
            }
          ],
          "error": null,
          "input_tokens": null,
          "output_tokens": null
        }
      ],
      "evidence": [
        {
          "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
          "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
          "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec",
          "task_id": "736fde4d-9029-4915-8189-01353d6982cb",
          "task_run_id": "7f0ef9d7-e7b5-4d52-8247-ce8351652f8b",
          "execution_status": "string",
          "exclusion_reason": null,
          "model": "string",
          "runtime_adapter": "claude",
          "service_tier": null,
          "reasoning_effort": "string",
          "skill_body_sha256": "string",
          "input_tokens": null,
          "output_tokens": null,
          "files": [
            {
              "id": "string",
              "kind": "instructions",
              "filename": "string",
              "sha256": "string",
              "size_bytes": 0
            }
          ],
          "sources": [
            {
              "id": "string",
              "kind": "instructions",
              "text": "string"
            }
          ],
          "limitations": [
            "string"
          ]
        }
      ],
      "limitations": [
        "string"
      ]
    }
  }
}

Create signals scout config trial evaluation

Freeze rubric and evidence, then judge explicit variant groups without changing production scouts.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Request parameters

evaluation_id
string
baseline_variant_id
string
variants
array
rubric_source

Response

Example request

POST /api/projects/:project_id/signals/scout/configs/:id/trial_evaluation
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_evaluation/\
	-d evaluation_id="string",\
	-d baseline_variant_id="string",\
	-d variants="array",\
	-d rubric_source=undefined
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.post(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_evaluation/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
	data={
		"evaluation_id": "string",
		"baseline_variant_id": "string",
		"variants": "array"
	}
).json()

Example response

Status 202

RESPONSE
{
  "request": {
    "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
    "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
    "variants": [
      {
        "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
        "label": "string",
        "launch_ids": [
          "497f6eca-6276-4993-bfeb-53cbbbba6f08"
        ]
      }
    ],
    "rubric_source": "saved"
  },
  "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
  "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
  "status": "pending",
  "error": "string",
  "report": {
    "version": 1,
    "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
    "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
    "created_at": "2019-08-24T14:15:22Z",
    "completed_at": "2019-08-24T14:15:22Z",
    "summary": "string",
    "outcome": null,
    "rubric_source": "saved",
    "rubric_revision": 0,
    "rubric_reference_context": null,
    "rubric_reference_generation_id": null,
    "criteria": [
      {
        "id": "string",
        "title": "string",
        "description": "string",
        "pass_condition": "string",
        "applicability": "string"
      }
    ],
    "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
    "judge_model": "string",
    "judge_prompt_version": "string",
    "variants": [
      {
        "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
        "label": "string",
        "is_baseline": true,
        "total_runs": 0,
        "judged_runs": 0,
        "excluded_runs": 0,
        "judge_errors": 0,
        "score": 0,
        "coverage": 0,
        "baseline_delta": null,
        "criteria": [
          {
            "criterion_id": "string",
            "passed": 0,
            "failed": 0,
            "unknown": 0,
            "not_applicable": 0,
            "pass_rate": 0,
            "coverage": 0,
            "baseline_delta": null
          }
        ]
      }
    ],
    "runs": [
      {
        "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
        "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
        "status": "judged",
        "score": null,
        "coverage": null,
        "summary": "string",
        "criteria": [
          {
            "criterion_id": "string",
            "verdict": "pass",
            "reason": "string",
            "confidence": "low",
            "evidence": [
              {
                "source_id": "string",
                "quote": "string"
              }
            ]
          }
        ],
        "error": null,
        "input_tokens": null,
        "output_tokens": null
      }
    ],
    "evidence": [
      {
        "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
        "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
        "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec",
        "task_id": "736fde4d-9029-4915-8189-01353d6982cb",
        "task_run_id": "7f0ef9d7-e7b5-4d52-8247-ce8351652f8b",
        "execution_status": "string",
        "exclusion_reason": null,
        "model": "string",
        "runtime_adapter": "claude",
        "service_tier": null,
        "reasoning_effort": "string",
        "skill_body_sha256": "string",
        "input_tokens": null,
        "output_tokens": null,
        "files": [
          {
            "id": "string",
            "kind": "instructions",
            "filename": "string",
            "sha256": "string",
            "size_bytes": 0
          }
        ],
        "sources": [
          {
            "id": "string",
            "kind": "instructions",
            "text": "string"
          }
        ],
        "limitations": [
          "string"
        ]
      }
    ],
    "limitations": [
      "string"
    ]
  }
}

Retrieve signals scout config trial evaluation

Read saved scores, criterion evidence and baseline differences without starting model calls.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Query parameters

evaluation_id
string

Response

Example request

GET /api/projects/:project_id/signals/scout/configs/:id/trial_evaluation_result
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_evaluation_result/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_evaluation_result/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200

RESPONSE
{
  "request": {
    "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
    "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
    "variants": [
      {
        "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
        "label": "string",
        "launch_ids": [
          "497f6eca-6276-4993-bfeb-53cbbbba6f08"
        ]
      }
    ],
    "rubric_source": "saved"
  },
  "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
  "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
  "status": "pending",
  "error": "string",
  "report": {
    "version": 1,
    "evaluation_id": "4108ec8e-6efd-4b9b-bace-0bb4c296c324",
    "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
    "created_at": "2019-08-24T14:15:22Z",
    "completed_at": "2019-08-24T14:15:22Z",
    "summary": "string",
    "outcome": null,
    "rubric_source": "saved",
    "rubric_revision": 0,
    "rubric_reference_context": null,
    "rubric_reference_generation_id": null,
    "criteria": [
      {
        "id": "string",
        "title": "string",
        "description": "string",
        "pass_condition": "string",
        "applicability": "string"
      }
    ],
    "baseline_variant_id": "d2b87c97-777a-48f8-9766-d8e68e9a46cf",
    "judge_model": "string",
    "judge_prompt_version": "string",
    "variants": [
      {
        "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
        "label": "string",
        "is_baseline": true,
        "total_runs": 0,
        "judged_runs": 0,
        "excluded_runs": 0,
        "judge_errors": 0,
        "score": 0,
        "coverage": 0,
        "baseline_delta": null,
        "criteria": [
          {
            "criterion_id": "string",
            "passed": 0,
            "failed": 0,
            "unknown": 0,
            "not_applicable": 0,
            "pass_rate": 0,
            "coverage": 0,
            "baseline_delta": null
          }
        ]
      }
    ],
    "runs": [
      {
        "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
        "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
        "status": "judged",
        "score": null,
        "coverage": null,
        "summary": "string",
        "criteria": [
          {
            "criterion_id": "string",
            "verdict": "pass",
            "reason": "string",
            "confidence": "low",
            "evidence": [
              {
                "source_id": "string",
                "quote": "string"
              }
            ]
          }
        ],
        "error": null,
        "input_tokens": null,
        "output_tokens": null
      }
    ],
    "evidence": [
      {
        "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
        "variant_id": "8a2d415d-59aa-4132-bef3-011f97c4a0f6",
        "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec",
        "task_id": "736fde4d-9029-4915-8189-01353d6982cb",
        "task_run_id": "7f0ef9d7-e7b5-4d52-8247-ce8351652f8b",
        "execution_status": "string",
        "exclusion_reason": null,
        "model": "string",
        "runtime_adapter": "claude",
        "service_tier": null,
        "reasoning_effort": "string",
        "skill_body_sha256": "string",
        "input_tokens": null,
        "output_tokens": null,
        "files": [
          {
            "id": "string",
            "kind": "instructions",
            "filename": "string",
            "sha256": "string",
            "size_bytes": 0
          }
        ],
        "sources": [
          {
            "id": "string",
            "kind": "instructions",
            "text": "string"
          }
        ],
        "limitations": [
          "string"
        ]
      }
    ],
    "limitations": [
      "string"
    ]
  }
}

Retrieve signals scout config trial

Read recent private runs for the requesting operator in the internal comparison editor.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Query parameters

limit
integer
Default: 30

Response

Example request

GET /api/projects/:project_id/signals/scout/configs/:id/trial_history
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_history/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_history/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200

RESPONSE
{
  "results": [
    {
      "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
      "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
      "variant": "string",
      "model": "string",
      "reasoning_effort": "string",
      "status": "string",
      "started_at": "2019-08-24T14:15:22Z",
      "completed_at": "2019-08-24T14:15:22Z",
      "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec",
      "task_id": "736fde4d-9029-4915-8189-01353d6982cb",
      "task_run_id": "7f0ef9d7-e7b5-4d52-8247-ce8351652f8b"
    }
  ],
  "has_more": true
}

Retrieve signals scout config trial

Read a trial's existing run status and its privately captured reports and memory changes.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Query parameters

launch_id
string

Response

Example request

GET /api/projects/:project_id/signals/scout/configs/:id/trial_result
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_result/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_result/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200

RESPONSE
{
  "launch_id": "e84bcd27-699d-48e8-b842-1ce01401cf1f",
  "context_id": "f31d7249-b7b1-4729-b3a4-ec0ba07b4686",
  "model": "string",
  "reasoning_effort": "string",
  "skill_body_sha256": "string",
  "result_key": "string",
  "export_error": "string",
  "started_at": "2019-08-24T14:15:22Z",
  "completed_at": "2019-08-24T14:15:22Z",
  "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec",
  "task_id": "736fde4d-9029-4915-8189-01353d6982cb",
  "task_run_id": "7f0ef9d7-e7b5-4d52-8247-ce8351652f8b",
  "status": "string",
  "task_status": "string",
  "error": "string",
  "summary": "string",
  "invalid_reason": "string",
  "reports": [
    {
      "id": "string",
      "source_report_id": null,
      "document": {
        "property1": null,
        "property2": null
      },
      "payload": {
        "property1": null,
        "property2": null
      },
      "edits": [
        {
          "property1": null,
          "property2": null
        }
      ],
      "operator_metadata": {
        "property1": null,
        "property2": null
      },
      "evidence": [
        {
          "property1": null,
          "property2": null
        }
      ],
      "artefacts": [
        {
          "property1": null,
          "property2": null
        }
      ],
      "content_revision_count": 0,
      "corroboration_count": 0
    }
  ],
  "memory": {
    "property1": {
      "key": "string",
      "content": "string",
      "created_at": "string",
      "updated_at": "string",
      "expires_at": "string",
      "created_by_run_id": "string",
      "created_by_skill": "string",
      "created_by_run_url": "string"
    },
    "property2": {
      "key": "string",
      "content": "string",
      "created_at": "string",
      "updated_at": "string",
      "expires_at": "string",
      "created_by_run_id": "string",
      "created_by_skill": "string",
      "created_by_run_url": "string"
    }
  },
  "cost_usd": 0,
  "input_tokens": 0,
  "output_tokens": 0
}

Retrieve signals scout config trial

Read comparison readiness and source settings for the internal comparison editor.

Required API key scopes

signal_scout:writellm_skill:write

Path parameters

id
string

Query parameters

context_id
string

Response

Example request

GET /api/projects/:project_id/signals/scout/configs/:id/trial_setup
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/:id/trial_setup/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/{signals-4_id}/trial_setup/".format(
        project_id=project_id,
		signals-4_id="<the signals-4 id>"
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200

RESPONSE
{
  "config_id": "d1d31429-d888-4f1c-b9c1-4e842f9bce5b",
  "skill_name": "string",
  "skill_version": 0,
  "skill_body": "string",
  "ready": true,
  "blocked_reason": "string",
  "model": "string",
  "reasoning_effort": "string",
  "models": [
    {
      "model": "string",
      "reasoning_efforts": [
        "string"
      ]
    }
  ]
}

Create signals scout config

Materialize the scout fleet for this project on demand (idempotent): seed the canonical signals-scout-* skills, create a default-schedule config for any scout lacking one, retire the skills whose canonical scout no longer ships, and return all scout configs. Normally the Temporal coordinator does this on its next tick; this action exists so the scout UIs and setup flows (e.g. the wizard's self-driving program) can hand the user a tunable fleet immediately. With surface=onboarding, new configs stay disabled until explicitly activated.

Required API key scopes

signal_scout:write

Query parameters

surface
string
One of: "roster""desktop""wizard""onboarding"

Example request

POST /api/projects/:project_id/signals/scout/configs/sync
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/sync/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.post(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/sync/".format(
        project_id=project_id
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200 The team's full scout fleet after the sync, ordered by skill name.

RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "skill_name": "string",
  "description": "string",
  "display_name": "string",
  "scout_origin": "canonical",
  "scout_role": "specialist",
  "deprecation": {
    "phase": "announced",
    "reason": "string",
    "superseded_by": "string",
    "sunset_at": "2019-08-24T14:15:22Z"
  },
  "owners": [
    {
      "id": 0,
      "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
      "distinct_id": "string",
      "first_name": "string",
      "last_name": "string",
      "email": "user@example.com",
      "is_email_verified": true,
      "hedgehog_config": {},
      "role_at_organization": "engineering"
    }
  ],
  "enabled": true,
  "status": "active",
  "pause_reason": "no_output",
  "managed_by": "team",
  "emit": true,
  "run_interval_minutes": 30,
  "run_cron_schedule": "string",
  "output_destinations": {
    "slack": {
      "integration_id": 1,
      "channel": "string",
      "users": [
        "string"
      ],
      "thread_reports": true
    },
    "webhook": {
      "hog_function_id": "string"
    },
    "inbox": true
  },
  "structured_output_schema": {},
  "network_access": "trusted",
  "model": "string",
  "mcp_gateway_server_ids": [
    "497f6eca-6276-4993-bfeb-53cbbbba6f08"
  ],
  "repositories": [
    "string"
  ],
  "write_scopes": [
    "string"
  ],
  "lifecycle_locked": true,
  "allowed_mcp_tools": [
    "string"
  ],
  "tool_preset": "string",
  "precheck_query": "string",
  "precheck_disabled": true,
  "effective_precheck_query": "string",
  "precheck_query_source": "config",
  "last_precheck_at": "2019-08-24T14:15:22Z",
  "last_precheck_outcome": "run",
  "last_precheck_row_count": 0,
  "last_run_at": "2019-08-24T14:15:22Z",
  "consecutive_failure_count": 0,
  "status_changed_at": "2019-08-24T14:15:22Z",
  "status_changed_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {},
    "role_at_organization": "engineering"
  },
  "auto_pause_exempt": true,
  "tags": [
    "string"
  ],
  "source_product": "string",
  "source_id": "string",
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z"
}

Retrieve signals scout config tool

List every MCP tool a scout could be configured with. Each entry carries the tool's name, label, one-line summary, category, and required scopes, plus holdable: whether a scout run can hold every scope the tool needs, and missing_scopes: what it would have to be granted on top of the baseline preset. The response also returns the scout scope presets and the write scopes a person can grant to one scout, so a caller can show why a tool is out of reach. Tools a successor has replaced are left out. A tool can carry a feature_flag, which resolves per project: evaluate it for the project you are configuring before you offer the tool. Read-only, and the same for every project.

Required API key scopes

signal_scout:read

Response

Example request

GET /api/projects/:project_id/signals/scout/configs/tool_catalogue
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/configs/tool_catalogue/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/configs/tool_catalogue/".format(
        project_id=project_id
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200 Every catalogued MCP tool, with the scout scope postures to read it against.

RESPONSE
{
  "tools": [
    {
      "name": "string",
      "title": "string",
      "summary": "string",
      "category": "string",
      "feature": "string",
      "required_scopes": [
        "string"
      ],
      "is_read_only": true,
      "requires_ai_consent": true,
      "holdable": true,
      "missing_scopes": [
        "string"
      ],
      "feature_flag": "string",
      "feature_flag_behavior": "string",
      "feature_flag_variant": "string",
      "hidden_when_flag_on": "string",
      "feature_entitlement": "string"
    }
  ],
  "tool_presets": [
    {
      "name": "string",
      "label": "string",
      "tools": [
        "string"
      ]
    }
  ],
  "run_context_tools": [
    "string"
  ],
  "presets": [
    {
      "name": "string",
      "scopes": [
        "string"
      ]
    }
  ],
  "grantable_write_scopes": [
    "string"
  ]
}

List all signals scout members

Return the people who can review work on this project — one row per member with access to it, each with their user_uuid, email, first_name/last_name, resolved GitHub login (null when they have no linked GitHub identity), and the teams they're on. The cold-start reviewer-routing path: when a finding's owner can't be read off a fetched entity's created_by and there's no cached reviewer:<area> memory or inbox precedent, list members, match the owner by email/name, then put their resolved github_login in suggested_reviewers on emit-report / edit-report. Pass team to resolve a team slug to the people on it, maintainers first. Pass search to narrow a large roster; the result is capped at 200. Strictly team-scoped.

Required API key scopes

signal_scout_internal:read

Query parameters

search
string
team
string

Example request

GET /api/projects/:project_id/signals/scout/members
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/members/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/members/".format(
        project_id=project_id
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200 The project's members, each with their routing identity.

RESPONSE
{
  "user_uuid": "string",
  "email": "user@example.com",
  "first_name": "string",
  "last_name": "string",
  "github_login": "string",
  "teams": [
    {
      "provider": "string",
      "slug": "string",
      "name": "string",
      "is_maintainer": true
    }
  ]
}

Retrieve signals scout metadata

Return the project's scout metadata: whether it is enrolled, the current announcement banner (e.g. an alpha run-limit notice, or null when unset), and the enforced run limits with current usage. Limits reflect what the coordinator actually applies at dispatch, so a user can see the real throttle rather than what they assume they set. All values come from the signals-scout flag payload, so the banner and caps can change with no deploy.

Required API key scopes

signal_scout:read

Response

Example request

GET /api/projects/:project_id/signals/scout/metadata/current
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/metadata/current/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/metadata/current/".format(
        project_id=project_id
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200 This project's scout enrollment, announcement banner, and enforced run limits.

RESPONSE
{
  "enrolled": true,
  "banner_message": "string",
  "limits": {
    "max_runs_per_tick": 0,
    "max_runs_per_day": 0,
    "runs_today": 0,
    "runs_remaining_today": 0,
    "max_enabled_scouts": 0
  }
}

List all signals scout notes

Return the steering notes left for this project's scouts, newest first. Pass skill_name to get the notes addressed to one scout (or one pipeline audience, e.g. pipeline:report-research) plus the general (blank-target) fleet-wide notes — the shape a scout run reads at cold start. Omit skill_name to browse every note. Expired notes are excluded unless include_expired=true. date_from / date_to are a half-open window on created_at (>= date_from, < date_to); pass date_to (the created_at of the oldest note seen) to walk past the cap. Pass text to keep only the notes whose content contains it, case-insensitively. Results capped at 500.

Required API key scopes

signal_scout:read

Query parameters

content_max_chars
integer
date_from
string
date_to
string
include_expired
boolean
Default: false
include_general
boolean
Default: true
limit
integer
skill_name
string
text
string

Example request

GET /api/projects/:project_id/signals/scout/notes
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/signals/scout/notes/
api_key = "[your personal api key]"
project_id = "[your project id]"
response = requests.get(
    "<ph_app_host>/api/projects/{project_id}/signals/scout/notes/".format(
        project_id=project_id
    ),
    headers={"Authorization": "Bearer {}".format(api_key)},
).json()

Example response

Status 200 Matching notes newest-first.

RESPONSE
{
  "id": "string",
  "skill_name": "string",
  "content": "string",
  "created_at": "string",
  "expires_at": "string",
  "created_by_name": "string",
  "origin": "string"
}

← Previous pageNext page →