AWS Batch
BetaThis source is currently in beta. It works end to end, but you may still hit rough edges. Let us know if you do.
Let AI connect your sources for you
Skip the manual setup — run this in your project and the wizard auto-detects your databases and APIs and connects them to PostHog.
Connect AWS Batch to PostHog to sync your data into the PostHog data warehouse for analysis and modeling.
Sync AWS Batch jobs and resources from one region. Grant batch:DescribeJobQueues, batch:ListJobs, and batch:DescribeJobs to read jobs and job children. Grant batch:DescribeJobQueues and batch:ListServiceJobs to read service jobs. Grant batch:DescribeComputeEnvironments and batch:DescribeJobDefinitions to read those tables. Grant batch:ListSchedulingPolicies and batch:DescribeSchedulingPolicies to read scheduling policies. Grant batch:ListConsumableResources and batch:DescribeConsumableResource to read consumable resources. Enter an access key ID and secret access key. Add a session token for temporary credentials. All tables use full refresh so later job status changes remain visible. The jobs and service jobs tables include jobs that AWS still retains in existing queues. The job children table includes the children of array jobs and the nodes of multi-node parallel jobs.
Configuration
| Option | Description |
|---|---|
AWS access key IDType: text Required: True | |
AWS secret access keyType: password Required: True | |
AWS session tokenType: password Required: False | Required for temporary credentials. Replace these credentials when the session token expires. |
AWS regionType: text Required: True |
Linking AWS Batch to PostHog
- Go to the Data pipeline page in PostHog
- Click New source and select AWS Batch
- Fill in the required configuration fields
- Click Next, select the tables you want to sync, and then press Import
Supported tables
| Table | Description | Sync method | Incremental field | Primary key |
|---|---|---|---|---|
jobs | Jobs in each queue, with their status, attempts, and execution details. Array children and nodes are in the job_children table. | Full refresh | — | — |
job_children | Child jobs of array jobs and nodes of multi-node parallel jobs, with their execution details. | Full refresh | — | — |
service_jobs | Service jobs, such as SageMaker training jobs, in each service job queue. | Full refresh | — | — |
job_queues | Job queues, their priorities, and their associated compute environments. | Full refresh | — | — |
compute_environments | Compute environments, their capacity settings, and their state. | Full refresh | — | — |
job_definitions | Job definition revisions, including their resource requirements and container settings. | Full refresh | — | — |
scheduling_policies | Scheduling policies and their fair-share settings. | Full refresh | — | — |
consumable_resources | Consumable resources with their total, in-use, and available quantities. | Full refresh | — | — |