Rules that will bite you

Five things that are not obvious and have each cost someone time.

  1. Public hostnames, never internal DNS. From a workspace, https://litellm.k8s.semanticscience.org — not litellm.litellm-prod.svc.cluster.local. Internal names are blocked by policy. It looks like the service is down; it is not.
  1. GPU workspaces: keep cpu ≤ 6. g1 has 32 cores for three cards, and every VM costs one more. The template does not enforce this yet; the scheduler does, by leaving an oversized request Pending with no message. If a GPU workspace will not start, lower cpu first; if it still will not, it is waiting on a *card* — check who holds the three.
  1. Stop GPU workspaces you are not using. Running holds the card, idle or not.
  1. /mnt is ephemeral. Only /home/coder survives a rebuild.
  1. No scale-to-zero on models. A served model occupies its hardware until removed. New models are swaps.

Two smaller ones: