Skip to main content
A team runner is a dedicated Linux machine that polls Syne for compilation jobs over outbound HTTPS. Teams own the machine, its capacity, and its GitHub read credential. Syne owns the project connection, job queue, source commit, artifact history, and review workflow. GitHub remains the source of truth. Team-managed runners and all dbt project connection modes require Syne Pro or an eligible trial.

​
Analyze an existing dbt project without a runner

If your models are already built in dbt Cloud or another platform, connect dbt Cloud metadata or import manifest.json in Syne. Neither analysis mode needs GitHub access or a runner. Syne reads the model definitions and queries the existing tables or views through your connected PostgreSQL warehouse. dbt Cloud supplies metadata, not model rows. Metadata refresh does not trigger a dbt job or certify data freshness. See connect existing models. Use the installation below when a GitHub-backed project needs compilation on your own compute. The current integration is a PostgreSQL preview; support depends on the matching Syne app deployment.

​
Shared Syne-managed capacity

SyneHQ can run a shared worker pool across teams from the main DevOps docker/docker-compose.yml. Pool workers automatically provision an isolated compiler container per leased job. They use an operator token and never register as a team runner. Teams choose the Syne-hosted runner in project settings; the app queues work across eligible teams with one active pool job per team and a global capacity cap. See shared pool deployment. This requires the updated app and the runner image containing pool_agent.py. Capacity is set through worker replicas; VM provisioning and queue-driven autoscaling are not included.

​
Choose a compilation target

Each registered runner executes one job at a time. Register separate identities on separate machines to run projects concurrently. One project can have one active metadata request; compilation, metadata refresh, and manifest import share a team quota of 20 requests per hour. Team-owned runners remain dedicated to a single team. Shared capacity uses the managed pool described above. Queue-driven autoscaling, production builds, scheduling, metered compute billing, and automatic failover are not included.

​
Install

  1. Provision a dedicated Linux amd64 host with Docker and gVisor runsc. Give it enough RAM and disk for the chosen sandbox limits plus the agent and operating system. Do not share its Docker daemon with databases or unrelated workloads. Docker socket access controls the host even when the socket mount is read-only.
  2. In Syne, open Database tools > dbt > Team-managed runners > Register runner as a team admin. Save the one-time credential in your machine’s secret manager. Syne stores only its SHA-256 hash. Each credential belongs to one team.
  3. Grant the host read access to the private GHCR packages. Pre-pull a reviewed compiler image and matching runner image. Registry credentials stay in the host’s deployment tooling; they are not mounted in the agent or sandbox. The repository and images are currently private: Syne must grant package access before customer onboarding.
  4. Deploy ghcr.io/synehq/syne-dbt-runner:sudo using the team runner Compose file. Pin both image digests for production. The agent has no listening port; it needs outbound access to the Syne app origin, api.github.com, and codeload.github.com.
  5. Set the variables below through the host’s secret manager, start the agent, and confirm Online in Syne. If your team enforces an IP allowlist, add the machine’s trusted egress address through the existing team policy.
  6. Open the connected project’s Settings, select the runner, save, and choose Queue compilation. Project configuration changes invalidate older jobs and artifacts. The app uses the team’s shared GitHub App installation (or a personal read token) to resolve the selected ref to an immutable commit; the runner separately needs local access to download that commit. Syne never sends its installation token to the runner.

​
Start the agent

Download the Compose file linked above as docker-compose.yml on the dedicated host. Register runsc with that Docker daemon using the gVisor installation instructions. Confirm runsc appears in docker info --format '{{json .Runtimes}}' before proceeding. Inject the required variables from the table through your secret manager. This non-secret example sets the capacity and pins the matching images published for the preview; supply the app origin, runner credential, repository allowlist, local GitHub token, and socket configuration separately:
Registry authentication needs package read access. Use docker login interactively or your deployment system’s secret input, never a credential embedded in a command. Pull the compiler into the same daemon selected by DBT_DOCKER_SOCKET_PATH; set DOCKER_HOST appropriately in your deployment shell if it is not the default daemon. The Compose agent’s internal DOCKER_HOST is already configured for its mounted socket. Confirm Online in Syne before selecting the runner in project settings. If it stays offline, check outbound HTTPS, the app origin, the runner credential, the team’s IP policy, and Docker socket permissions. If jobs fail before compilation, check GitHub Contents read access, the allowlist, the pre-pulled compiler image, and runsc registration. Stop the agent with docker compose -f docker-compose.yml down; pause new jobs in Syne first if you want a running job to finish. The local GitHub token is never sent to Syne. Syne does not send its stored repository or database credentials to the runner. Only the source archive, target database/schema names, and numeric resource limits enter the compiler container. The agent’s own environment contains credentials; operate it as a trusted host component. The compiler has no network, host mounts, credentials, capabilities, or writable root filesystem.

​
Job lifecycle and controls

Syne pins the GitHub SHA before queueing. The runner checks its repository allowlist, downloads only that pinned archive, and advertises its locally configured capacity. A poll atomically claims one job with a fresh random lease. Heartbeats run every 20 seconds; a lease expires after 90 seconds. A failed heartbeat stops source retrieval or compilation. A job’s absolute deadline is the advertised dbt timeout plus four minutes for source retrieval, startup, cleanup, and transfer. Queued jobs expire after one hour. Expired jobs become failed when the queue or project is next inspected; they cannot be claimed or completed after expiry. Interrupted or failed jobs are never automatically retried or rerouted. Users request another compilation explicitly.
  • Pause new jobs preserves the queue and lets the current job finish.
  • Cancel compilation marks the job canceled immediately. The connected agent stops at its next heartbeat, normally within 20 seconds plus network latency. Syne rejects any subsequent result for that job.
  • Rotate credential cancels queued/running jobs and invalidates the old token. Install the newly displayed token and restart the agent.
  • Revoke cancels queued/running jobs and soft-deletes the runner. Projects keep their selection visible as unavailable until an admin chooses a replacement.
  • Removing the requester from the team, deleting its connection, or changing project settings prevents the old job from submitting results. Disconnecting or suspending the shared GitHub App installation, removing repository access, or syncing a repository rename also invalidates the binding and cancels active jobs.
A timeout is a ceiling, not an artificial delay. The compiler process gets the configured duration; the host watchdog allows 40 additional seconds for startup/execution and then attempts forced removal, with a 15-second Docker cleanup limit and a bounded process wait. A broken Docker daemon still requires operator recovery. A failed compilation does not deploy models or write to the warehouse.

​
Large-project bounds and compatibility

Team runners support up to 5,000 SQL/YAML/Markdown/CSV source files and 32 MB of decoded source, within a 64 MB encoded request. A downloaded archive is limited to 100 MB compressed, 256 MB expanded, and 20,000 entries. Source retrieval has a 120-second budget. Hidden files, profiles, logs, old targets, traversal paths, and selected symlinks are excluded or rejected. No archive is extracted onto the host. The compiler manifest is capped at 32 MB and encoded result transport at 33 MB; Syne normalizes at most 10,000 resources. Sandbox limits include 256 processes and 512 MB temporary storage. Capacity is an upper bound; the machine must actually have those resources. Compilation remains offline PostgreSQL dbt compilation with the pinned compiler version. Vendor packages in the repository. Warehouse introspection, dependency installation, Python models, custom adapters, dbt run, and dbt build are unsupported. Import a manifest from the existing environment when those are required. Model previews still use Syne’s governed database bridge. A successful compilation is not proof of passing data tests or SQL execution. Artifacts say Compiled on team runner and record its name. Team-controlled machines can alter their output; Syne does not attest that they executed an unmodified compiler. Reviewers should treat that provenance accordingly.

​
Syne operator deployment

Apply the reviewed 20260911033525_dbt_team_runners metadata migration after 20260910191632_dbt_governance, generate Prisma, and deploy the matching app routes and UI. The durable queue lives in the metadata database and does not depend on a long-running app request. The existing-model analysis modes additionally require 20260911041153_dbt_analysis_sources and matching app and Kole versions. Apply reviewed migrations in order before enabling those routes. This guide’s public copy and Compose download are generated in the docs repository with npm run sync:dbt; npm run check:dbt-sync detects drift against adjacent compiler and DevOps checkouts. Allow these exact POST routes through browser-session/SSO middleware; each verifies its team runner bearer credential and team IP policy, and job routes additionally verify the lease:
  • /api/dbt/runner-agent/poll
  • /api/dbt/runner-agent/jobs/<jobId>/heartbeat
  • /api/dbt/runner-agent/jobs/<jobId>/complete
Preserve the Authorization header and provide trusted client-IP headers from the edge. Do not exempt other dbt routes. Disable caching; allow at least 33,000,000 request-body bytes and a 60-second server budget on completion. A platform with a smaller fixed body limit requires a different artifact transport before large manifests are supported. The agent times out each HTTP request after 30 seconds and retries completion using the same lease; accepted completion is idempotent. Keep the controller’s shared secret separate from per-team runner credentials. Never enable DBT_TRUSTED_LOCAL_TEST or SYNE_DBT_LOCAL_TEST in deployment.