Servers and pools

Servers and pools brings physical machines into Frazer: servers you already run at a provider, in a colocation rack or in your office, and servers you rent from Frazer. They live in your organization’s infrastructure project, join through an agent that only dials out, and take no work until you, or a rule you set, accept them. Pools then share their cores, memory and GPUs with the projects you choose, where clusters, VMs, devboxes and databases run on them.

Product overview: Servers and pools.

How it works

Every organization gets an infrastructure project when it is created: acme-infra for Acme. Sites, servers, pools, join tokens and provider accounts live there, so a server outlives the projects that run on it. A lien keeps the infrastructure project from being deleted while it has servers or shares.

Whichever way a server arrives, it follows the same path:

1. A join token lets the agent on the machine ask to join, and nothing more. The machine makes its own key, prints a fingerprint and a 6-character code, and appears as pending.
2. You compare the code and accept the server (frazer servers add does both for you, over your own SSH), or a pool rule accepts it. Accepting issues a 24-hour certificate, renewed while the server stays healthy.
3. Frazer inspects the hardware, binds the server’s identity and installs the OS when Frazer manages it. Burn-in runs when the pool asks for it, and always on servers you rent.
4. The server is available in its pool. The projects the pool is shared with place clusters, VMs, devboxes and databases on it.

Key concepts:

• Site: where servers are: one provider data center and its private network, one rack, one office. A site is one failure domain, under 2 ms between servers, and a cluster lives in one site.
• Region: where Frazer runs the control plane and the Frazer Link gateways your sites dial out to. The first region is Virginia, USA (us-virginia1). A site’s name carries the region that serves it: projects/acme-infra/locations/us-virginia1/sites/ovh-vint-hill.
• Server pool: a named set of servers chosen by labels, with a role, a wipe policy, accept rules, a maintenance window and share settings.
• Substrate host pool: its servers form one pool substrate (Kubernetes and Frazer’s VM layer on bare metal) that Frazer runs. Projects get vCPU, memory and GPUs from it as VMs. Your containers never run on the bare host.
• Dedicated node pool: hands whole servers to one cluster, for large databases, GPUs or full NIC speed. Shares of it count whole servers.
• Share settings: which projects of the organization use a pool, with a guaranteed amount and a limit for each.
• frazer-nodelet: the agent on each server. One signed binary under systemd, open source under Apache-2.0. It pulls signed, typed operations and runs nothing else.
• Frazer Link: the WireGuard gateways in each region that carry console, kubectl, log and metric traffic from your sites. No inbound port is ever opened at a site.

Server states

State	What it means
provisioning	Frazer installs the OS through a provider’s API, or fulfills an order.
registering	Frazer checks BMC credentials through a Site Gateway.
pending	The agent presented a valid join token. Nothing is trusted yet.
inspecting	Hardware and firmware inventory; identity bound; OS installed if needed.
burn-in	Stress tests and a benchmark score.
available	Healthy, unallocated and schedulable.
allocating, allocated	Joining, then a member of a pool substrate or a dedicated cluster.
draining, releasing, wiping	Leaving: cordon and evict, leave the cluster, then wipe by the pool’s policy.
maintenance	Held by a person or by an automatic repair. Not schedulable.
lost	No heartbeat for 5 minutes. Back to its previous state when it reconnects; failed after 24 hours.
failed	A step failed: inspection, burn-in, provisioning or wipe. The server page says which.
detached	You removed Frazer from the server. No credentials remain; the record stays for audit.
retired	End of life. Read-only.

Add a server

There are six ways in. Each one ends in the same list and the same lifecycle. For a Linux server you already run, frazer servers add is the main one.

Way in	You need	What happens	Operating system
frazer servers add	SSH as root (or sudo) to Ubuntu 22.04 or 24.04, Debian 12 or 13, Rocky Linux or RHEL 9 or 10	The CLI installs the agent over SSH and accepts the server as you, in one command. Nothing is wiped; the server becomes a dedicated node.	Yours; you patch it
Run the install command	The same Linux, plus a join token from the console or the CLI	For automation and cloud-init: the agent installs and the server waits as pending for a person or a pool rule.	Yours; you patch it
Connect a provider account	API keys limited to your servers at OVHcloud or Hetzner	Frazer reinstalls the servers you pick through the provider’s API. Each one joins and is accepted by itself.	Ubuntu LTS, managed by Frazer
Boot our image	Any console: KVM, IPMI, your provider’s KVM or a USB stick	An ISO, USB image, iPXE script or qcow2 with the token inside.	Ubuntu LTS, managed by Frazer
Server with a BMC	A BMC that a Site Gateway in your site can reach	Zero-touch through Redfish: inspection, firmware, RAID, reinstall and power.	Frazer OS
Buy from Frazer	A card on file or an invoice account	A server Frazer owns, dedicated to your organization for the term and burned in first.	Frazer OS

Before you start, you need:

• A site. Create one in Infrastructure › Sites › Create with its provider, network mode and the IP pool that load-balancer addresses come from. See Regions and sites.
• Egress from the site on TCP 443 and UDP 51820 (or TCP 443 only), NTP and DNS. An HTTP(S) proxy works.
• Hardware that meets the minimum requirements.
• The Servers admin role on the infrastructure project.

Add a server with the CLI

Use this for a Linux server you already run, or a Linux instance in one of your cloud accounts. One command from your laptop installs the agent over SSH, checks the server’s code in the same session and accepts it as you.

1. Install the frazer CLI on your laptop and sign in (frazer login). See CLI.

   curl -fsSL https://get.frazer.ai | sh

2. Check that you can reach the server: ssh root@203.0.113.10 works, or a user with sudo. The CLI runs your own ssh client, so your keys, agent, ~/.ssh/config aliases and jump hosts work as they do now, and the server’s host key is checked against your known_hosts.

3. Add the server, naming the site, the pool and any labels. Leave out --site or --pool and the CLI lists the ones you may use and asks:

   frazer servers add root@203.0.113.10 --site sf-office --pool sf-office-default --label rack=r2

   The CLI creates a join token for this one run (single use, short-lived, bound to the site and pool), then runs the signed install script on the server over SSH. Your Frazer credentials stay on your laptop: the server receives only the join token. The agent runs preflight, makes its machine key on the server and prints its code, which the CLI reads over the same session, so no one has to compare codes by hand:

   Output

   ✓ SSH          root@203.0.113.10 · Ubuntu 24.04 · host key known
   ✓ Join token   sf-office · sf-office-default · single use · for this run only
   ✓ Agent        installed from get.frazer.ai/node · signature verified
   ✓ Preflight    kernel · cgroups v2 · clock · swap · disk · 443
   ✓ Machine key  generated on the server · TPM-sealed
   → Fingerprint  TPM EK 9f3a…c41e
   ✓ Code M3T-8QX read over SSH · accepted by lena@acme.example · audit entry
     inspecting … available

4. The command waits until the server is available, or stops at the step that failed and prints what to fix.

The same command runs on the server itself: install the CLI there, sign in with frazer login --no-browser, then run sudo frazer servers add --site sf-office --pool sf-office-default with no destination. Add --no-accept to leave the server pending for someone else to accept in the console.

Accepting needs the Servers admin role on the infrastructure project; without it the server stays pending with its code, for an admin to accept. Agents use the same command with their own identity and role, and every step is in the audit log.

Run the install command

Use this for automation: cloud-init user data, configuration management or a golden image, where no one runs a command by hand. The server joins and waits as pending until a person, or a pool rule, accepts it.

1. Create a join token for the site and pool: open Infrastructure › Servers › Add server › Run install command and choose the site, the pool and labels, or run frazer servers join-tokens create --site sf-office --pool sf-office-default. You get one line with the token:

   curl -fsSL https://get.frazer.ai/node | sudo sh -s -- --token <join-token> --site sf-office

   The token expires in 24 hours by default, 30 days at most. It has a use limit and you can revoke it. Keep it in your automation’s secret store, not in an image or a repository.

2. Run the line on the server, or put it in the instance’s cloud-init runcmd. The script and the agent are signed, and the script prints the checksum it verified. The agent runs preflight (kernel, cgroups v2, time sync, swap, free disk, outbound ports), then prints its fingerprint and a code:

   Output

   ✓ Script and agent signed · checksum verified
   ✓ Preflight   kernel · cgroups v2 · clock · swap · disk · 443
   ✓ Machine key generated here · TPM-sealed when present
   → Fingerprint TPM EK 9f3a…c41e
   → Code        M3T-8QX   match it in the console
     pending · no credentials until someone accepts

3. In the console, the server appears under Waiting for servers as pending, with the same code. Compare the codes, then choose Accept, or let a pool rule accept it. If the codes differ, choose Reject: Frazer deletes the record and blocks the fingerprint.

4. Watch the server move from inspecting to available.

A server joined with frazer servers add or the install command keeps your operating system. You patch it, and the agent reports its patch level. It takes the dedicated node role only, because Frazer runs no VMs on an OS it does not control. To host VMs, reinstall it with Convert to Frazer-managed OS, which wipes the server and needs an approval.

If a machine Frazer already knows joins again, the console offers to re-attach it to its existing record instead of creating a second one.

Connect a provider account

Frazer reinstalls servers you rent at OVHcloud or Hetzner through the provider’s API, with the agent and a token bound to each server. Details: Provider accounts.

1. Open Infrastructure › Provider accounts › Add and choose OVHcloud US. Frazer shows the rights it needs and links to a page at OVHcloud that creates a key limited to them: list dedicated servers (GET /dedicated/server/*), reinstall (POST …/reinstall), reboot (POST …/reboot) and IP moves. Paste the keys. Frazer tests them, refuses a key with broader rights, stores it in Frazer Secrets in your infrastructure project and shows only its fingerprint afterwards.

2. Pick the servers to import and the pool, and read the plan.

   CLI

   frazer servers import ovh --account acme-ovh \
       --servers ns5012345,ns5012346,ns5012347 \
       --pool vint-hill-default --dry-run

   The plan names every server and what it erases:

   Output

   Plan          p-0031 · 3 servers · expires in 24 h or when acme-ovh changes
   Reinstall     erases every disk on 3 servers
   + Server      Atlas, Borealis, Cobalt · Ubuntu 26.04 · bound join token
   + ServerPool  vint-hill-default · substrate host · wipe crypto-erase
   + Substrate   acme-vint-hill · forms at 3 hosts · Acme only
   Cost          96 physical cores · per-core fee while accepted

   Console

   Open the account and choose Import servers. Frazer lists the account’s dedicated servers with model, data center and IPs. Tick the servers, then pick or create the site, the pool and labels. The plan shows the same lines as the CLI.

3. Apply the plan. A reinstall erases every disk on the servers you picked, so the console asks you to type the confirmation, for example erase 3 servers. From the CLI, run the command again without --dry-run.

4. Frazer reinstalls each server with the agent and a join token in its user data. The token is single use, valid 3 hours and bound to that server. Each server moves provisioning → pending → inspecting → available with no accept click: the pool accepts the join only when the token, the server’s service name and its IP all match. Any other join waits for a person.

On Hetzner, which has no reinstall-with-user-data call, Frazer boots the rescue system with a one-time SSH key, writes the image from rescue and reboots. The token binding is the same.

The provider’s private network (OVHcloud vRack, Hetzner vSwitch) becomes the site’s network, and the additional IP blocks routed to it become the site’s load-balancer addresses. You keep paying your provider for each server; Frazer bills its per-core fee on top. A server at any other provider joins with frazer servers add, the install command or the boot image.

Boot our image

1. Open Infrastructure › Servers › Add server › Boot our image. Choose the site, the pool, labels, the token’s lifetime and its number of uses. Download an ISO, a USB image, an iPXE script URL or a qcow2. The token is inside.

2. Boot the image on the server from any console: KVM, IPMI, your provider’s KVM or a USB stick. The image runs the agent in install mode and shows a QR code and the 6-character code on screen. The server appears as pending.

3. Match the code, choose Accept and pick the install disk. The image installs the OS and the agent, reboots, and the server moves from inspecting to available.

Add servers with a BMC

A Site Gateway lets Frazer install, configure and power-cycle servers with nobody on site.

1. Open your site in Infrastructure › Sites, then Site Gateway › Install. Install it on one small VM or server in the site. It runs bare-metal provisioning, a DHCP proxy, iPXE, an image cache and its own outbound agent.

2. Add BMC addresses: one, a CSV, or a scan of a subnet for Redfish endpoints. Add the credentials and choose Keep on site (recommended: they never leave the Site Gateway) or Store in Frazer’s vault.

3. Each server moves registering → inspecting (Redfish out of band, plus the agent in band) → optional firmware settings → OS install → optional burn-in → available.

Rent a server from Frazer

1. Open Infrastructure › Servers › Add server › Buy from Frazer. Pick the region, the server type, the quantity, the term (monthly, 12 months or 36 months) and the pool. Each type shows whether it is in stock or its delivery date.

2. Frazer creates an order with a quote. An organization owner approves it. Nothing is ordered before that.

3. Frazer installs Frazer OS with a token bound to the order. Each server moves provisioning → inspecting → mandatory burn-in → available in your pool, owned by Frazer and dedicated to your organization.

A rented server runs Frazer OS only, so it can prove what it booted. Frazer replaces failed parts. When the term ends, the server is released, fully wiped with a certificate in your audit log, checked with a burn-in and returned to stock. Server types, terms and the wipe: Buy from Frazer.

Instances in your cloud accounts

A Linux instance in one of your cloud accounts joins with frazer servers add, or with the install command in its cloud-init, like any server, as a dedicated node. Frazer needs no access key to your cloud account. Clusters you already run elsewhere attach through an outbound agent instead: see Your cloud accounts and Kubernetes.

Share a pool with projects

A pool’s share settings decide which projects of your organization place work on it. Sharing never crosses organizations: a server you bring runs only your organization’s workloads.

• Who can use it: LOCAL (only the infrastructure project), SPECIFIC_PROJECTS (the projects you list) or ORGANIZATION (every current and future project; a default entry applies to the ones you do not list).
• Guaranteed: vCPU, GiB and GPUs per class (whole servers on a dedicated node pool) held for the project and never lent to others.
• Limit: how far the project may burst into capacity no project holds. At least the guarantee; empty means the same as the guarantee.
• Placement: SHARED_SERVERS (the default: VMs of several projects share a server, each in its own project network) or EXCLUSIVE_SERVERS (the project gets whole servers, for production and staging separation or compliance). Dedicated node pools are always exclusive.
• Cost: show pool cost back to each project by usage, by guarantee, or not at all.
• Unshare grace period: how long a removed project keeps running before its VMs stop. 7 days by default.

CLI

frazer servers server-pools share vint-hill-default \
    --project acme-infra --location us-virginia1 \
    --share-type specific-projects \
    --consumer 'acme-production=guaranteed:vcpus=96,memory-gib=384;limit:vcpus=120,memory-gib=480' \
    --dry-run

Quote the --consumer value so the shell does not read its ;. The plan shows the capacity math and who gains access:

Output

Plan        ~ pool vint-hill-default · revision 8 → 9
            + acme-production · guaranteed 96 vCPU, 384 GiB · limit 120, 480
Capacity    112 of 136 vCPU guaranteed (144 minus one surge VM) · 24 free to burst
Who         members of acme-production with compute.vms.create can place VMs here
Affected    none · a new share

Run the command again without --dry-run to apply it.

Console

1. Open Infrastructure › Pools and sharing, then the pool, then the Sharing tab, and choose Share.
2. Who can use it: only this project, specific projects, or everyone in Acme. The picker lists projects of your organization only.
3. Per project: set guaranteed and limit. The console shows the math live, for example “112 of 136 vCPU guaranteed (144 minus one surge VM); 24 free for bursting”.
4. Isolation: projects may share servers, or each project gets whole servers.
5. Cost: show back by usage, by guarantee, or not at all.
6. Advanced: require the pool-user permission and pick principals; set the unshare grace period.
7. Plan: capacity before and after, affected resources, the showback preview and who gains access. Choose Apply.

YAML

vint-hill-default.yaml

apiVersion: servers.frazer.ai/v1
kind: ServerPool
name: projects/acme-infra/locations/us-virginia1/serverPools/vint-hill-default
spec:
  site: ovh-vint-hill
  role: substrate-host # or dedicated-node
  wipePolicy: crypto-erase
  shareSettings:
    shareType: SPECIFIC_PROJECTS # LOCAL, SPECIFIC_PROJECTS or ORGANIZATION
    projectMap:
      acme-production:
        capacity: { guaranteed: { vcpus: 96, memoryGib: 384 }, limit: { vcpus: 120, memoryGib: 480 } }
      acme-staging:
        capacity: { guaranteed: { vcpus: 16, memoryGib: 64 }, limit: { vcpus: 32, memoryGib: 128 } }
    unshareGracePeriod: 168h # stop, never delete
    costAllocation: USAGE # show back to each project by usage

frazer apply -f vint-hill-default.yaml --dry-run

Terraform

main.tf

resource "frazer_servers_server_pool" "vint_hill" {
  project  = "acme-infra"
  location = "us-virginia1"
  name     = "vint-hill-default"
  share_settings {
    share_type = "SPECIFIC_PROJECTS"
    project_map {
      id = "acme-production"
      capacity {
        guaranteed { vcpus = 96  memory_gib = 384 }
        limit      { vcpus = 120 memory_gib = 480 }
        placement  = "SHARED_SERVERS"
      }
    }
    unshare_grace_period = "168h"
    cost_allocation      = "USAGE"
  }
}

terraform plan shows the same quota impact as --dry-run. Removing a project_map entry plans a drain and lists the affected resources; it never deletes them.

How capacity is counted

• Physical, after a reserve. Each server keeps 8 cores and 32 GiB for the host and the substrate. The rest is the pool’s capacity.
• One surge VM stays free. The largest VM shape on the pool is held back for upgrades and repairs, so guarantees add up to at most the capacity minus that VM.
• CPU sharing is stated, memory is never overcommitted. Control-plane VMs count their full CPU. Workers share CPU 1:1 in production, up to 2:1 in staging and up to 4:1 for previews; the ratio turns requested vCPU into physical vCPU before it counts.
• Bursts are never preempted. A project grows to its limit, or to its guarantee plus what no project holds, whichever is smaller.
• A full pool refuses; it never evicts. The plan then offers to request more guaranteed capacity, which goes to the pool’s admins as an approval, or to place on the Frazer fleet where that is allowed.
• Quotas still apply. A project’s own quotas, such as clusters per project or public IPs, apply on top of its share.
• Three servers for production. One server is a working substrate for small clusters. A production cluster puts its three control-plane VMs on three different servers. A plan that leaves no room to recreate a lost server’s VMs (N+1) asks you to add a server or acknowledge it.

See shared capacity in a project

frazer servers server-pools list --project acme-staging --include-shared

In the console, Project › Settings › Capacity shows each pool shared with the project: guaranteed, used and limit, drains in progress with their deadline, and Request more. When you create a cluster, VM, devbox or database, Where it runs lists the project’s own pools first, then pools shared with it, then the Frazer fleet. A project sees only its own entry, never other projects.

The infrastructure project pays the server fees. Compute on a shared pool costs the consuming project nothing and is shown back to it as “allocated from acme-infra, not invoiced”. The consuming project still pays its own storage, egress and Frazer fleet usage.

Keep servers healthy

• Health conditions. The agent reports a heartbeat and inventory every 30 seconds, with conditions for the agent connection, disk health, ECC errors, NIC link and clock. A failing critical condition takes the server out of scheduling until it recovers.
• Automatic repair. For an allocated server: alert, cordon and drain, reboot through the agent, power cycle through the BMC or the provider’s API, then maintenance with a ticket, then failed. Never more than one server, or 10 % of a pool, at a time.
• Maintenance window. Set it on the pool. OS, Kubernetes and agent updates are staged inside it. Kernel updates reboot only in the monthly window, one host per site at a time, with drains coordinated. A critical kernel vulnerability may cause an announced emergency reboot outside the window. Firmware updates run as operations in the pool’s window.
• Notices. Maintenance on a shared server notifies every project with VMs on it, in that project’s own maintenance calendar.
• Agent upgrades. Each pool follows a channel, stable or fast. An upgrade goes to one server, then 10 %, then all; an agent that misses heartbeats after an upgrade rolls back. Agent and OS releases must carry the signature of an offline release key pinned at enrollment.
• GPUs. Inspection finds GPUs before any driver loads: model, NVLink, PCIe topology, IOMMU groups, ECC and MIG support. GPUs pass whole into VMs; MIG and time-slicing stay inside one project’s VM. Requirements and burn-in: Real hardware.

Release, detach or move a server

Action	Who	What happens	Result
Release	An admin of the consuming or the infrastructure project	Drain, leave the cluster, wipe by the pool’s policy.	available in its pool
Detach	An organization owner in the console, or locally on the server	Optional drain; Frazer’s components and credentials removed; you keep the disks, reset state or crypto-erase.	detached; billing stops
Move	An admin of both projects	An available server moves to another project of the same organization and joins a pool there, under that pool’s wipe policy.	In the other project
End rental	An organization owner, or the end of the term	Release, full wipe, certificate, burn-in check.	Back to Frazer’s stock
Retire	An organization owner	Wipe, then notes such as an RMA or disposal.	retired

To detach from the server itself, which works even when Frazer is unreachable:

sudo frazer-nodelet detach

Output

  Works even when Frazer is unreachable
✓ Drained · left the cluster
✓ Frazer components and credentials removed
? Disks: keep · reset state · crypto-erase › keep
  detached · billing stopped · record kept for audit

A wipe, properly. Every data disk Frazer formats is encrypted with LUKS2 and a per-server key sealed to the TPM. A wipe destroys the key, sanitizes each disk (NVMe sanitize or secure format, or ATA secure erase), discards every block, verifies with sample reads and writes a wipe certificate: who, when, the method, disk models and serials, and the result. The certificate is stored with the audit log and you can download it. If the agent is unreachable, Frazer cannot wipe: the server is marked lost and needs BMC or physical recovery first.

Unshare or shrink a pool

frazer servers server-pools unshare vint-hill-default \
    --project acme-infra --location us-virginia1 \
    --consumer acme-staging --grace-period 168h --dry-run

The plan lists every cluster, VM, devbox and database the change affects. When you apply it:

1. The project’s entry goes to DRAINING. It takes no new placements; running workloads keep running. The project’s owners get notices 7 days, 1 day and 1 hour before the deadline.
2. At the deadline, its VMs are stopped, never deleted, and its clusters on the pool are suspended with their nodes at zero. Disks stay on the servers for 30 days.
3. The project snapshots its disks to Frazer storage, or moves to another pool or the Frazer fleet.

Unsharing a production project needs a second person’s approval. Shrinking a guarantee or a limit below what a project uses is refused, naming the resources; even with the safety checks overridden, nothing is evicted, and the project cannot grow until it is back under. A pool with consumers or servers cannot be deleted.

Access and audit

Who	What they may do
Servers admin on the infrastructure project	Create join tokens, accept or reject servers, create and edit sites and pools, set accept rules.
Holders of servers.serverPools.share	Edit a pool’s share settings. Sharing is its own permission, so people who edit capacity cannot widen who uses it.
Organization admin	Confirm a wipe or reinstall. Turn on a remote shell for one server, for a set time.
Organization owner	Approve orders, detach or retire servers, end a rental.
Members of a consuming project	Place clusters, VMs, devboxes and databases on a shared pool with the create permission they already have, such as kubernetes.clusters.create or compute.vms.create. They see only their own project’s share.
roles/servers.poolUser	Needed on a pool only when it requires the pool-user permission.

Moving a server needs servers.servers.move on both projects. Organization policies can limit which projects may own servers (constraints/servers.allowedHostProjects), which share types are allowed (constraints/servers.allowedShareTypes) and which pools a project may consume (constraints/servers.restrictPoolConsumption).

What Frazer may do to a server you bring:

Action	Through	By default	Guard
Read inventory, health and metrics	Agent	On	None needed
Install or upgrade the OS, Kubernetes and the agent	Agent	On, staged, in your maintenance window	Release signature checked against the pinned key
Join or leave a cluster, drain	Agent, cluster controller	On	Audit log
Reboot	Agent	On, in the window	Rate limit per pool
Wipe or reinstall	Agent, provider API, BMC	Needs an organization admin’s confirmation	Approval and audit
BMC power, virtual media, firmware	Site Gateway	Same as a wipe	BMC credentials can stay on site
Remote shell, serial console	Agent through Frazer Link	Off. An organization admin turns it on per server, time-boxed; you are notified	Session recording
Read your workload data	None	Never. There is no path	Disk keys sealed per server

The agent enforces its own rules, so they hold even if Frazer’s control plane is wrong: no exec, only typed operations; every command signed by Frazer; no join for another organization, or for a project without a signed share; no listening port except on localhost.

Pin your own approval key

Pin a key your organization holds, a passkey-backed signing key held by organization admins or a key in your own KMS. Wipe, reinstall, BMC power, firmware and shell commands then need its signature, so a compromised Frazer could only propose them. The pin changes only with a command signed by the old key, or with frazer-nodelet reset-trust at the server’s console.

What is logged:

• Join tokens, and every accept or reject with the fingerprint and the token used.
• Every reinstall, wipe, firmware update and shell session, with its approval and result.
• Share changes, and every placement on a pool: the project, the resource name and its size, never its data. Both the infrastructure project and the consuming project see the actions that touch them.
• Wipe certificates, stored with the audit log.
• A hash-chained local log on each server, which you can forward.

Limits and quotas

Item	Default or limit
Join token	24 hours by default, 30 days at most; a use limit; revocable
Join token from a provider import	Single use, 3 hours, bound to one server
Machine certificate	24 hours, renewed automatically
Heartbeat	Every 30 seconds; a revoked server disconnects within one heartbeat
Lost and failed	lost after 5 minutes without a heartbeat; failed after 24 hours
Reserve per server	8 cores and 32 GiB for the host and the substrate
Surge	One VM of the largest shape per pool, held for upgrades and repairs
Automatic repairs	One server, or 10 % of a pool, at a time
Site	Under 2 ms between servers; a cluster stays in one site
Unshare grace period	7 days (168 h) by default; 1 hour minimum, 30 days maximum
Disks after a drain deadline	Kept on the servers for 30 days
Projects in one share	Soft limit of 1,000

Minimum requirements

The site page shows these as a checklist the agent fills in.

Item	Dedicated node	Substrate host (runs VMs)	GPU node
CPU	x86-64-v2, 8 cores	x86-64-v3, VT-x or AMD-V and an IOMMU, 32 cores or more	As a host, plus Above-4G decoding
Memory	32 GB (16 GB for a trial)	128 GB or more	At least twice the total GPU memory
Disks	240 GB or more SSD to boot; NVMe for data	Two mirrored boot disks, plus NVMe for VM disks	NVMe scratch
Network	1 GbE; 10 or 25 GbE recommended	Two 10 or 25 GbE ports, bonded, with VLAN tagging	InfiniBand or RoCE for multi-node training
BMC	Optional	Strongly recommended	Strongly recommended
TPM 2.0 and Secure Boot	Recommended	Recommended; required on servers Frazer owns	Recommended
Egress	TCP 443 and UDP 51820 (or 443 only), NTP, DNS	The same, plus L2 between hosts	The same

Pricing

A server you bring pays a fee per physical core, metered by the hour while it is accepted; hyperthreads never count, and clusters and VMs on it add no fee of their own. A rented server pays the provider’s price plus a markup, with the management fee included.

Servers and pools	On your servers	On the Frazer fleet
Price	$1.50 per physical core / month · min $15, max $150 per server · first server ≤ 16 cores free	Rented: provider price + 25 %
your cloud accounts	Observed or connected clusters: $0, with 10,000 series and 10 GB logs a month each · managed: $0.75 per node vCPU / month, max $150 per node	Not on the fleet: the nodes run in your cloud account

Every line, with the estimator: pricing.

Troubleshooting

Symptom	Cause	Fix
A server never appears as pending	Outbound 443 is blocked, the clock is off, or the token is used or expired	Run sudo frazer-nodelet diagnose on it. Set HTTPS_PROXY if 443 needs a proxy, fix time sync if the clock is more than 5 minutes off, or create a new token.
A provider server waits for a person	Its token, service name or IP did not match the pool’s rule	Compare the code and accept it by hand, or reject it if you did not expect it.
A server is lost	No heartbeat for 5 minutes	Check its power and network. It returns to its previous state when the agent reconnects; after 24 hours it is failed.
A server is out of scheduling	A critical health condition is false	Open the server’s page to see the condition. It returns to the schedulable set when the condition recovers.
POOL_CAPACITY_EXHAUSTED when creating a cluster or VM	The project is at its limit, or nothing unguaranteed is left	Request more guaranteed capacity from the plan, place on the Frazer fleet where allowed, or add servers.
POOL_NOT_SHARED	The pool is not shared with your project	Ask an admin of the infrastructure project to share it.
A shrink of a share is refused	The new guarantee or limit is below what the project uses	Reduce the project’s use first. The plan names the resources.
The infrastructure project cannot be deleted	A lien stands while it has servers or active shares	Unshare its pools and release or move its servers.
Frazer cannot wipe a server	The agent is unreachable	Recover the server through its BMC or on site before it can be wiped.

Reference

The error reasons Servers and pools returns, each with its fix, are in the table below.

CLI

Command	What it does
frazer servers add	Add a Linux server you run. Over SSH from your laptop (frazer servers add root@host), or on the server itself; installs the agent and accepts the server as you.
frazer servers join-tokens create	Create a join token for a site and pool, for the install command in automation and cloud-init.
frazer servers import ovh	Plan, then reinstall and join servers from a connected OVHcloud account into a pool.
frazer servers server-pools share	Share a server pool with projects of the organization, with a guaranteed amount and a limit for each.
frazer servers server-pools unshare	Remove a project from a pool's share settings. Its entry drains for the grace period, then its VMs stop.
frazer servers server-pools list	List server pools in a project; with --include-shared, also the pools shared with it.

All of them: CLI reference.

Terraform and OpenTofu

Resource	What it manages
frazer_servers_server_pool	A server pool in a host project, with its share settings.

All of them: Terraform and OpenTofu reference.

Events

Event type	When it is sent
frazer.server.joined.v1	A server presented a valid join token and appears as pending.
frazer.server.accepted.v1	A person or a pool rule accepted a pending server.
frazer.server.unhealthy.v1	A server reported a failing health condition.
frazer.server.burnin.completed.v1	A server finished its burn-in tests.
frazer.server.removed.v1	A server was removed from Frazer.

All of them: Events reference.

Errors

Reason	What it means	How to fix it
GUARANTEE_EXCEEDS_CAPACITY	The guarantees in a share add up to more than the pool's capacity minus one surge VM, or a guarantee is above its limit.	Lower a guarantee, set each limit at or above its guarantee, or add servers to the pool.
POOL_CAPACITY_EXHAUSTED	The placement would take the project past its limit, or the pool has no capacity left that no project holds.	Request more guaranteed capacity from the pool's admins (the plan offers it), place on the Frazer fleet where allowed, or add servers.
POOL_NO_FIT	The VMs do not fit on any server's free capacity with the pool's anti-affinity and placement mode.	Add servers to the pool, or plan fewer or smaller VMs.
POOL_NOT_SHARED	The pool is not shared with the project that tried to place on it.	Ask an admin of the pool's host project to add the project to the pool's share settings.
POOL_SHARE_DRAINING	The project's share of the pool is draining after an unshare and takes no new placements.	Place new work on another pool shared with the project, or on the Frazer fleet, and move running work before the deadline.
POOL_IN_USE	The pool still has consumer projects or servers.	Unshare the pool from every project and move or release its servers, then delete it.

All of them: Errors reference.

On the server

frazer-nodelet has a few local commands that work without the console:

Command	What it does
frazer-nodelet status	Show the agent’s state.
sudo frazer-nodelet diagnose	Find out why a server does not appear or does not connect.
sudo frazer-nodelet detach	Remove Frazer from the server, even when Frazer is unreachable.
frazer-nodelet reset-trust	Reset the pinned approval key, at the server’s console.

Resource names

projects/{project}/locations/{location}/sites/{site}
projects/{project}/locations/{location}/servers/{server}
projects/{project}/locations/{location}/serverPools/{server_pool}
projects/{project}/locations/{location}/joinTokens/{join_token}
projects/{project}/orders/{order}

Related

• Bring your first server: install the agent, accept the server, see it in a pool.
• Kubernetes: clusters as VMs on your pools.
• VMs: virtual machines on the same pools.
• How it works: the agent, Frazer Link and the control plane.
• Real hardware and Regions and sites.