Deploy CosmicAC

Deploy the CosmicAC Docker Compose stack on your host machine. After deployment, CosmicAC connects to your GPU Kubernetes cluster. For the services in the stack and how they interact, see Deployment architecture.

Prerequisites

The recommended host is Ubuntu 22.04 or 24.04 on an x86_64 CPU. Before you start, make sure that you have the following.

Docker Engine and Docker Compose v2.
Task.
Node.js.
The jq and kubectl command-line tools.
Access to the private CosmicAC deployment repository, which contains the Compose files and deployment scripts. To get access, contact the CosmicAC team.
GitHub Container Registry (GHCR) credentials for the private CosmicAC images in ghcr.io/tetherto.
- Your GitHub username.
- A classic GitHub personal access token with the read:packages scope. If the tetherto organization enforces single sign-on (SSO), approve the token for the organization. See Managing your personal access tokens.
A GPU Kubernetes cluster that meets the requirements. CosmicAC connects to the cluster but doesn't create it.
A kubeconfig for the cluster.

Steps

Verify the prerequisites

Enable Docker, and check that each required tool is installed.

sudo systemctl enable --now docker
docker compose version
task --version
jq --version
node --version
kubectl version --client

Set up the deployment

Clone the deployment repository and change to its directory.

git clone <deployment-repo-url>
cd <deployment-repo>

Create the .env file from the example file.

cp .env.example .env

In .env, set the variables that are required before the first deployment. For every supported variable, see Deployment configuration.

If GITHUB_PAT and GITHUB_USER aren't set in .env, bootstrap prompts for them before it pulls the private images from GHCR.

Add the kubeconfig

Get the kubeconfig from your cluster administrator. The kubeconfig must meet the kubeconfig requirements.

Create a file on the host, paste the kubeconfig into it, and save the file.

nano ~/kubeconfig

Get the absolute path of the file.

realpath ~/kubeconfig

In .env, set KUBECONFIG_SRC to that path.

KUBECONFIG_SRC=/home/<user>/kubeconfig

Check that the file isn't empty and that the kubeconfig reaches your cluster.

test -s ~/kubeconfig && echo ok
kubectl --kubeconfig ~/kubeconfig config current-context
kubectl --kubeconfig ~/kubeconfig cluster-info

Run the bootstrap

The task bootstrap command deploys the whole stack with the TAG value in .env. For what bootstrap runs, see Task deployment commands.

Get the path of the deployment directory, and copy it.

pwd

Open a root shell.

sudo -i

A root shell starts in the root home directory. Change back to the deployment directory.

cd <deployment-directory>

Run the bootstrap.

task bootstrap

Run later commands as root

Run all later task commands as root. Bootstrap and the running services create the deployment's configuration and state files as root, so commands such as task backup and task update fail for other users.

Verify the deployment

To check the services and the web interface, run the following commands.

task ps
curl -s4 -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/

The services are running when task ps shows every service as Up and the web interface returns HTTP 200.

Create the administrator account

Bootstrap doesn't create a user account. To log in to CosmicAC, create the first administrator account.

In .env, replace the placeholder values of the following variables.

BOOTSTRAP_ADMIN_EMAIL — the email address that you log in with.
BOOTSTRAP_ADMIN_PASSWORD — a strong password for the account.

To create the account, run the following command.

task seed-admin

The command prints Seeded bootstrap admin user and the email address. For more information about the account's roles, see Administrator account.

Log in to the web interface

In your browser, go to http://<server-ip>:5173. Replace <server-ip> with the IP address of your host machine. If you changed UI_PORT, use that port instead of 5173.

In the Email and Password fields, enter the values of BOOTSTRAP_ADMIN_EMAIL and BOOTSTRAP_ADMIN_PASSWORD from .env. Then click Log In.

Help and troubleshooting

Bootstrap fails with rm: cannot remove ...: Permission denied

Bootstrap resets the HyperMQ stores in ./services/*/store before it deploys. The stack's containers create those files as root, so a non-root user can't delete them.

To fix the error, run bootstrap again from a root shell in the deployment directory.

sudo -i
cd <deployment-directory>
task bootstrap

Bootstrap fails with credentials still cannot access or error from registry: denied

The token in .env can't read the CosmicAC images. Before bootstrap pulls the images, it logs in again with the values in .env, so you don't need to remove earlier Docker credentials. For more information, see task ensure-login.

To fix the error, do the following as root in the deployment directory.

In .env, check that GITHUB_PAT is a classic personal access token that meets the following conditions.
- It has the read:packages scope.
- It hasn't expired.
- It's authorized for the tetherto organization, if the organization enforces single sign-on (SSO).
In .env, check that TAG names a release that exists. A misspelled tag fails with the same error.
Run bootstrap again.
```
task bootstrap
```
Bootstrap logs in with the corrected values and pulls the images.

Bootstrap fails with docker login failed for ghcr.io

GHCR rejected the GITHUB_USER or GITHUB_PAT value in .env.

To fix the error, do the following as root in the deployment directory.

In .env, correct GITHUB_USER and GITHUB_PAT.
Run bootstrap again.
```
task bootstrap
```
Bootstrap logs in with the corrected values and pulls the images.

If the login still fails with correct values, open the Docker configuration file, ~/.docker/config.json by default, and look for a credsStore or credHelpers entry. Either entry makes Docker store credentials in a separate helper program, and a missing or broken helper makes every login fail.

Next steps

After you deploy CosmicAC, add a recommended model configuration for each model that you want to serve. Then install the CLI and create your first job.

Deploy CosmicAC

On this page