Skip to main content
Agent environments define interaction and deployment. Sandbox pools govern shared execution capacity. Tasksets define the work. These resources have different lifecycles and separate pages. A pool has a Docker or E2B backend, total concurrent trial slots, a per-job limit, a lease timeout, and an allowed-user list. One lease reserves one trial slot; it is not a measurement of physical containers, CPUs or provider billing. A trial may involve additional verifier services, so provision backend capacity accordingly.

​
Provision and register

Provision the provider account or Docker runtime first, then create a pool in Assets → Sandbox pools as an administrator. Use a single pool for capacity shared by the same jobs; duplicate pools pointing at the same provider do not share a budget. The allowed-user field accepts comma-separated usernames, or * for all platform users. Creating a pool does not deploy machines, grant Docker access, store provider credentials or create warm sandboxes. Operators see authorized pools and their own leases. Administrators manage limits, see all leases, drain pools and confirm cleanup. The page shows available, occupied and cleanup-required slots, with unreleased leases first and pagination.

​
Use from Agent evaluations

Select a matching active pool in Agent evaluations. The prepared configuration records its ID. Run it through the harbor-eval experiment in tuneplane-examples. The worker obtains one lease before each trial, waits when capacity is full, renews while running, and releases after explicit provider cleanup succeeds. The configured per-job concurrency must not exceed the pool’s per-job limit. The pool applies to jobs using its lease API. A job without a selected pool keeps only its local concurrency limit. Existing OpenEnv runner registration does not automatically acquire pool leases. Other training harnesses must integrate the same acquire/renew/release contract in their user project or environment plugin. The control plane does not execute agents or call provider SDKs.

​
Drain, revoke and recover

  • Drain stops new allocation. Existing leases can renew and finish normally.
  • Revoke lease marks its slot as requiring cleanup. A cooperating worker stops when its next renewal is refused, then attempts provider cleanup.
  • Lease timeout marks a lost worker’s slot as requiring cleanup. It does not prove that the remote sandbox was deleted, so it does not restore capacity.
  • Confirm provider cleanup and release is an administrator action. First delete or verify absence of the provider resources, then record evidence in the page. This action releases accounting capacity; it does not call the provider.
A worker that fails cleanup reports it immediately. A killed worker requires provider TTLs or operator reconciliation. Cleanup uncertainty keeps capacity occupied deliberately. Reducing capacity below occupied slots is refused, and old attempts still count toward a run’s limit until their resources are cleaned.

​
Worker API

Use an attempt-bound job ingest token, not a user’s console token: An acquisition key identifies one trial within one job attempt and is idempotent while its lease is active. Never reuse it after completion or expiration. Capacity exhaustion returns 429; retry with backoff. Draining, expiration or a superseded attempt returns 409. Retry/resume tokens cannot renew earlier attempts. An old attempt can confirm cleanup of its own lease while its job record still exists.

​
Managed E2B instances

With an E2B pool selected, the updated harbor-eval adapter registers a creation intent before calling E2B. Each intent grants creation once; an uncertain request is not retried automatically. Provider metadata carries the pool, lease and instance identities so a separate cleanup process can discover a sandbox whose creation response was lost. Main and separate verifier environments are tracked. The Provider instances table shows provider IDs, job attempts, last reported states and cleanup errors. A created state is a receipt, not a live health probe. The worker renews a five-minute provider TTL at most every minute. It reports successful deletion before releasing the lease; deletion errors retain capacity. TTL expiration alone is not a platform cleanup receipt. Deploy the independent cleanup process from tuneplane-examples on a host with the E2B Python SDK, outside the evaluation job’s lifetime:
Run it from the example project root with automatic process restart. The console token must belong to an administrator. Use the same E2B_API_KEY, E2B_DOMAIN and E2B_API_URL settings as the workers. The platform stores a credential fingerprint, never the provider key. A different fingerprint blocks automated cleanup; credential rotation requires the original credential or manual recovery. Every 30 seconds by default, the process scans expired or revoked leases, checks provider ownership, deletes remaining instances and confirms absence. Failed cleanup backs off up to one hour. A known ID absent under the matching credential can be confirmed deleted. An intent without an ID and without a discoverable sandbox stays unconfirmed: absence cannot prove that an uncertain creation never completed. Stop its creator, verify provider resources, then use the existing administrator cleanup action with evidence. All registered instances must be confirmed deleted before their trial slot is released. Deploying the console does not start this process. Update and archive the runtime source before launching evaluations from it; previously archived jobs retain their previous adapter. Pin a compatible Harbor/E2B image. The adapter uses Harbor’s environment import_path extension and E2B lifecycle APIs; see the Harbor E2B implementation and E2B Python SDK. Docker pools retain the existing cleanup adapter; OpenEnv runners do not gain this integration automatically. Local tests cover lifecycle and failure handling with provider and Harbor doubles. Real cloud creation/deletion, runtime-version compatibility and large-scale throughput still require deployment validation.