> ## Documentation Index
> Fetch the complete documentation index at: https://docs.synehq.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Install a team-managed dbt runner

> Connect a dedicated Linux machine to Syne for isolated dbt compilation with more time and capacity.

A team runner is a dedicated Linux machine that polls Syne for compilation jobs over outbound HTTPS. Teams own the machine, its capacity, and its GitHub read credential. Syne owns the project connection, job queue, source commit, artifact history, and review workflow. GitHub remains the source of truth.

Team-managed runners and all dbt project connection modes require Syne Pro or an eligible trial.

## Analyze an existing dbt project without a runner

If your models are already built in dbt Cloud or another platform, connect dbt Cloud metadata or import `manifest.json` in Syne. Neither analysis mode needs GitHub access or a runner. Syne reads the model definitions and queries the existing tables or views through your connected PostgreSQL warehouse. dbt Cloud supplies metadata, not model rows. Metadata refresh does not trigger a dbt job or certify data freshness. See [connect existing models](https://docs.synehq.com/features/dbt/connect-existing-models).

Use the installation below when a GitHub-backed project needs compilation on your own compute. The current integration is a PostgreSQL preview; support depends on the matching Syne app deployment.

## Shared Syne-managed capacity

SyneHQ can run a shared worker pool across teams from the main DevOps `docker/docker-compose.yml`. Pool workers automatically provision an isolated compiler container per leased job. They use an operator token and never register as a team runner. Teams choose the Syne-hosted runner in project settings; the app queues work across eligible teams with one active pool job per team and a global capacity cap.

See [shared pool deployment](https://github.com/SyneHQ/devops/blob/sudo/guides/dbt-pool.md). This requires the updated app and the runner image containing `pool_agent.py`. Capacity is set through worker replicas; VM provisioning and queue-driven autoscaling are not included.

## Choose a compilation target

| Target                        | Resources                                                | dbt time limit                                                 | Suitable for                                                 |
| ----------------------------- | -------------------------------------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------ |
| Syne-hosted legacy controller | 1 CPU, 512 MB                                            | 3 minutes 40 seconds                                           | Small PostgreSQL projects                                    |
| Shared Syne pool              | Default 2 CPU, 4 GB                                      | Default 15 minutes; operator-configurable up to 30 minutes     | Shared capacity across teams; hosted source limits apply     |
| Team-managed                  | Default 2 CPU, 4 GB; configurable up to 16 CPU and 16 GB | Default 15 minutes; configurable from 60 seconds to 30 minutes | Larger projects on a team's own compute                      |
| Imported manifest             | Your existing dbt environment                            | Determined by that environment                                 | Warehouse introspection, custom adapters, or existing builds |

Each registered runner executes one job at a time. Register separate identities on separate machines to run projects concurrently. One project can have one active metadata request; compilation, metadata refresh, and manifest import share a team quota of 20 requests per hour. Team-owned runners remain dedicated to a single team. Shared capacity uses the managed pool described above. Queue-driven autoscaling, production builds, scheduling, metered compute billing, and automatic failover are not included.

## Install

1. Provision a dedicated Linux amd64 host with Docker and gVisor `runsc`. Give it enough RAM and disk for the chosen sandbox limits plus the agent and operating system. Do not share its Docker daemon with databases or unrelated workloads. Docker socket access controls the host even when the socket mount is read-only.
2. In Syne, open **Database tools > dbt > Team-managed runners > Register runner** as a team admin. Save the one-time credential in your machine's secret manager. Syne stores only its SHA-256 hash. Each credential belongs to one team.
3. Grant the host read access to the private GHCR packages. Pre-pull a reviewed compiler image and matching runner image. Registry credentials stay in the host's deployment tooling; they are not mounted in the agent or sandbox. The repository and images are currently private: Syne must grant package access before customer onboarding.
4. Deploy `ghcr.io/synehq/syne-dbt-runner:sudo` using [the team runner Compose file](/downloads/dbt-runner/docker-compose.yml). Pin both image digests for production. The agent has no listening port; it needs outbound access to the Syne app origin, `api.github.com`, and `codeload.github.com`.
5. Set the variables below through the host's secret manager, start the agent, and confirm **Online** in Syne. If your team enforces an IP allowlist, add the machine's trusted egress address through the existing team policy.
6. Open the connected project's **Settings**, select the runner, save, and choose **Queue compilation**. Project configuration changes invalidate older jobs and artifacts. The app uses the team’s shared GitHub App installation (or a personal read token) to resolve the selected ref to an immutable commit; the runner separately needs local access to download that commit. Syne never sends its installation token to the runner.

| Variable                   | Required / default                                                                                       |
| -------------------------- | -------------------------------------------------------------------------------------------------------- |
| `SYNE_DBT_APP_URL`         | Required; Syne's HTTPS origin, without a path, query, or credentials                                     |
| `SYNE_DBT_RUNNER_TOKEN`    | Required; one-time credential from your team's registration dialog                                       |
| `SYNE_DBT_REPOSITORIES`    | Required; comma-separated `owner/repository` allowlist for this machine                                  |
| `SYNE_DBT_GITHUB_TOKEN`    | Local GitHub token with Contents read for allowed private repositories; optional for public repositories |
| `SYNE_DBT_TIMEOUT_SECONDS` | `900`; allowed range `60–1800`                                                                           |
| `SYNE_DBT_MEMORY_MB`       | `4096`; allowed range `512–16384`                                                                        |
| `SYNE_DBT_CPUS`            | `2`; allowed range `1–16`                                                                                |
| `DBT_SANDBOX_IMAGE`        | Matching pre-pulled compiler image; pin a digest                                                         |
| `DBT_DOCKER_GID`           | Group ID of the dedicated Docker socket, used by Compose                                                 |
| `DBT_DOCKER_SOCKET_PATH`   | Path of the dedicated Docker socket, used by Compose                                                     |
| `SYNE_DBT_RUNNER_IMAGE`    | Agent image, used by Compose; pin a digest                                                               |

### Start the agent

Download the Compose file linked above as `docker-compose.yml` on the dedicated host. Register `runsc` with that Docker daemon using the [gVisor installation instructions](https://gvisor.dev/docs/user_guide/install/). Confirm `runsc` appears in `docker info --format '{{json .Runtimes}}'` before proceeding.

Inject the required variables from the table through your secret manager. This non-secret example sets the capacity and pins the matching images published for the preview; supply the app origin, runner credential, repository allowlist, local GitHub token, and socket configuration separately:

```sh theme={"system"}
export SYNE_DBT_RUNNER_IMAGE=ghcr.io/synehq/syne-dbt-runner@sha256:a9717f647d22e255e7717b485fd57c63af895a51c07ab852828f11d74ae080c0
export DBT_SANDBOX_IMAGE=ghcr.io/synehq/syne-dbt-compiler@sha256:d8a5326582e1de2f9313b635d4a31732b31924a10ecb0f388b042e7cd0ee71e4
export SYNE_DBT_TIMEOUT_SECONDS=900
export SYNE_DBT_MEMORY_MB=4096
export SYNE_DBT_CPUS=2
docker login ghcr.io
docker pull "$DBT_SANDBOX_IMAGE"
docker compose -f docker-compose.yml up -d
docker compose -f docker-compose.yml ps
docker compose -f docker-compose.yml logs --tail=50 dbt-runner
```

Registry authentication needs package read access. Use `docker login` interactively or your deployment system's secret input, never a credential embedded in a command. Pull the compiler into the same daemon selected by `DBT_DOCKER_SOCKET_PATH`; set `DOCKER_HOST` appropriately in your deployment shell if it is not the default daemon. The Compose agent's internal `DOCKER_HOST` is already configured for its mounted socket.

Confirm **Online** in Syne before selecting the runner in project settings. If it stays offline, check outbound HTTPS, the app origin, the runner credential, the team's IP policy, and Docker socket permissions. If jobs fail before compilation, check GitHub Contents read access, the allowlist, the pre-pulled compiler image, and `runsc` registration. Stop the agent with `docker compose -f docker-compose.yml down`; pause new jobs in Syne first if you want a running job to finish.

The local GitHub token is never sent to Syne. Syne does not send its stored repository or database credentials to the runner. Only the source archive, target database/schema names, and numeric resource limits enter the compiler container. The agent's own environment contains credentials; operate it as a trusted host component. The compiler has no network, host mounts, credentials, capabilities, or writable root filesystem.

## Job lifecycle and controls

Syne pins the GitHub SHA before queueing. The runner checks its repository allowlist, downloads only that pinned archive, and advertises its locally configured capacity. A poll atomically claims one job with a fresh random lease. Heartbeats run every 20 seconds; a lease expires after 90 seconds. A failed heartbeat stops source retrieval or compilation. A job's absolute deadline is the advertised dbt timeout plus four minutes for source retrieval, startup, cleanup, and transfer.

Queued jobs expire after one hour. Expired jobs become failed when the queue or project is next inspected; they cannot be claimed or completed after expiry. Interrupted or failed jobs are never automatically retried or rerouted. Users request another compilation explicitly.

* **Pause new jobs** preserves the queue and lets the current job finish.
* **Cancel compilation** marks the job canceled immediately. The connected agent stops at its next heartbeat, normally within 20 seconds plus network latency. Syne rejects any subsequent result for that job.
* **Rotate credential** cancels queued/running jobs and invalidates the old token. Install the newly displayed token and restart the agent.
* **Revoke** cancels queued/running jobs and soft-deletes the runner. Projects keep their selection visible as unavailable until an admin chooses a replacement.
* Removing the requester from the team, deleting its connection, or changing project settings prevents the old job from submitting results. Disconnecting or suspending the shared GitHub App installation, removing repository access, or syncing a repository rename also invalidates the binding and cancels active jobs.

A timeout is a ceiling, not an artificial delay. The compiler process gets the configured duration; the host watchdog allows 40 additional seconds for startup/execution and then attempts forced removal, with a 15-second Docker cleanup limit and a bounded process wait. A broken Docker daemon still requires operator recovery. A failed compilation does not deploy models or write to the warehouse.

## Large-project bounds and compatibility

Team runners support up to 5,000 SQL/YAML/Markdown/CSV source files and 32 MB of decoded source, within a 64 MB encoded request. A downloaded archive is limited to 100 MB compressed, 256 MB expanded, and 20,000 entries. Source retrieval has a 120-second budget. Hidden files, profiles, logs, old targets, traversal paths, and selected symlinks are excluded or rejected. No archive is extracted onto the host.

The compiler manifest is capped at 32 MB and encoded result transport at 33 MB; Syne normalizes at most 10,000 resources. Sandbox limits include 256 processes and 512 MB temporary storage. Capacity is an upper bound; the machine must actually have those resources.

Compilation remains offline PostgreSQL dbt compilation with the pinned compiler version. Vendor packages in the repository. Warehouse introspection, dependency installation, Python models, custom adapters, `dbt run`, and `dbt build` are unsupported. Import a manifest from the existing environment when those are required. Model previews still use Syne's governed database bridge. A successful compilation is not proof of passing data tests or SQL execution.

Artifacts say **Compiled on team runner** and record its name. Team-controlled machines can alter their output; Syne does not attest that they executed an unmodified compiler. Reviewers should treat that provenance accordingly.

## Syne operator deployment

Apply the reviewed `20260911033525_dbt_team_runners` metadata migration after `20260910191632_dbt_governance`, generate Prisma, and deploy the matching app routes and UI. The durable queue lives in the metadata database and does not depend on a long-running app request.

The existing-model analysis modes additionally require `20260911041153_dbt_analysis_sources` and matching app and Kole versions. Apply reviewed migrations in order before enabling those routes. This guide's public copy and Compose download are generated in the docs repository with `npm run sync:dbt`; `npm run check:dbt-sync` detects drift against adjacent compiler and DevOps checkouts.

Allow these exact POST routes through browser-session/SSO middleware; each verifies its team runner bearer credential and team IP policy, and job routes additionally verify the lease:

* `/api/dbt/runner-agent/poll`
* `/api/dbt/runner-agent/jobs/<jobId>/heartbeat`
* `/api/dbt/runner-agent/jobs/<jobId>/complete`

Preserve the Authorization header and provide trusted client-IP headers from the edge. Do not exempt other dbt routes. Disable caching; allow at least 33,000,000 request-body bytes and a 60-second server budget on completion. A platform with a smaller fixed body limit requires a different artifact transport before large manifests are supported. The agent times out each HTTP request after 30 seconds and retries completion using the same lease; accepted completion is idempotent.

Keep the controller's shared secret separate from per-team runner credentials. Never enable `DBT_TRUSTED_LOCAL_TEST` or `SYNE_DBT_LOCAL_TEST` in deployment.