SayGM is a privacy-first AI inference gateway. Developers swap a base URL and keep the same behavior, but every request is routed through a hardware-attested Trusted Execution Environment. The prompt content is not visible to SayGM, and not visible to the host machine it runs on. Pricing is driven by competition among operators, so buyers typically pay below list rate. To make "we cannot read it" verifiable rather than aspirational, SayGM moved its entire supply side, the operators who run its routing and attestation layer, onto Phala Cloud as confidential workloads. Every request routes through a hardware-attested Phala confidential VM running an image the SayGM team built, published, and pinned to a digest. This is not a proof-of-concept.
Each operator runs the same approved image as its own confidential VM. A registry checks the image hash before that workload is allowed to take traffic, and every workload carries its own attestation.
The missing layer: hardware-enforced privacy
Anyone building a trust layer for AI hits the same wall. To make a genuine privacy promise, you need to prove three things for every request.
The data was sealed during processing. Not encrypted at rest and in transit. Sealed inside the compute, where the host operator cannot read it.
The runtime was genuine. The code that processed the request was the expected code, unmodified, on real hardware. Not "we ran it on a cloud provider" but a cryptographic receipt saying exactly what ran.
The identity was the workload's, not just the operator's. A node operator's endpoint and the confidential workload serving that request are two different things. The attestation must bind to the workload, not to the operator who happens to run it.
Two of these require hardware-enforced compute. Most teams either do not have access to it, or underestimate how much work it is to build. This is the gap Phala fills. We do not build the gateway ourselves. We are the substrate that makes any privacy claim provable.
Why we built Phala the way we did
Phala has spent years on one problem: how do you give a workload a sealed runtime, attested by hardware, with keys that never leave a TEE, in a way that any third party can verify?
The answer is built on two pieces.
dstack is the open-source TEE framework. Apache 2.0, hosted by the Linux Foundation. Anyone can audit it, fork it, run it themselves. It handles hardware attestation, key provisioning, encrypted storage, and a Docker Compose-native deployment model.
Phala Cloud is the managed layer on top. Provisioning, scaling, monitoring, billing. Operationally necessary, but cryptographically excluded from the data path. Phala's control plane cannot read your workload's memory or storage keys.
This separation is what makes the privacy claim stick. When a team builds on Phala, it is not taking Phala's word that the data is safe. It can verify it. The attestation quote is signed by trusted hardware. The OS measurements match the open-source build. The container hash binds the attestation to the exact image you deployed. Keys are derived inside another TEE and released only after verification passes.
No trust in vendor. No trust in operator. No trust in host admin. Verifiable from the silicon up.
How SayGM built on Phala
Each operator on the SayGM network deploys a container image, built and published by the SayGM team and pinned to a digest, as a Phala Cloud confidential VM. The operator's CLI handles the full lifecycle: key registration, provider credentials, and deployment. It calls Phala Cloud directly to launch and verify the workload.
On deploy, SayGM's registry verifies the deployed image hash against its list of approved versions before the operator is allowed to register and start serving. Only known, vetted images can join the network. The confidential VM then runs two core components side by side.
- An attestation server, which serves fresh, hardware-attested quotes on request.
- A data-plane proxy, which carries the operator's traffic to the upstream providers that operator has configured. It runs inside the sealed workload too, so the routing logic and the credentials it uses never leave the confidential VM.
Both live inside the same attested workload, on the same confidential VM.
This is the part worth highlighting. Because the data plane and the attestation server are one measured workload, verifying the workload verifies the component that receives the request and decides where it goes. There is no separate, unmeasured hop the buyer has to take on faith. The quote covers the code doing the routing, and the registry check covers the image that code came from.
Both components run inside the same measured workload. Operator credentials are injected into the encrypted VM at deploy time, and the hardware seal underneath is what makes the workload identity attestable.
Crucially, the operators bring their own upstream provider credentials. Those credentials are injected into the encrypted confidential workload at deploy time, so SayGM never holds them, and the host cannot read them once the workload is running.
Remote attestation proves that a specific, approved, measured workload is what is actually running, with a fresh nonce per verification to prevent replay. A separate registered identity identifies the operator's endpoint. The attestation quote and the attested TLS binding confirm the workload identity, not the identity of the operator who runs it. This distinction is the mechanism behind SayGM's buyer-facing privacy guarantee.
Operators can scale horizontally by attaching additional confidential VMs as separate workers under the same network key, so a single operator's capacity is not capped at one VM.
Verifiable privacy, not trust-based privacy
The value we add to SayGM comes down to one thing: your users no longer have to trust the gateway. They can verify it.
- When a buyer sends a request to SayGM, that the exact approved workload is the one that received the request and decided where it went, and that it was the workload's identity making that decision, not merely the operator's endpoint. This is not "the gateway promised not to look." It is "we can prove it could not look."
That is the difference between a privacy policy and a privacy guarantee. SayGM chose the latter, and made that possible by running its entire supply side on Phala.
What this means for the supply layer of AI inference
Independent operators you do not control can still be held to code you approved. Each one runs a measured workload. Each one's image is checked before that workload is allowed to take traffic. The gateway does not need your trust. The receipt is there.
Across the AI ecosystem, inference is served by networks of independent operators: inference markets, model gateways, decentralized compute, enterprise clouds. All competing to answer the same request. The tension is obvious: how do you prove, across operators you do not control, that a request was handled by the code you approved, on hardware nobody has tampered with?
Phala provides that layer, and SayGM shows how it works in production. Since go-live, 60 distinct coldkeys have been deployed on the network, 18 are operating today, and 57 confidential VMs are currently active. Every SayGM operator deploys an approved, pinned image as a confidential VM, and a registry checks that image before that operator's workload is allowed to take traffic. Inside the VM, an attestation server and a data-plane proxy run side by side, so the code that receives a request and decides where it goes is the same measured workload a buyer can verify. Operators bring their own upstream credentials, injected into the encrypted workload so SayGM and the host never see them. Remote attestation proves the workload, not the operator who runs it. That distinction is SayGM's buyer-facing privacy guarantee.
That is what a confidential compute substrate is for. No more trust in the gateway, the operator, or us. The hardware protects. The attestation proves.
Start building
An attested workload replaces policy with proof. Your data plane and its attestation run as one measured unit, and every operator that joins your network is checked before it can serve.
If you are building a gateway, an inference market, or any service where your users currently have to take your word for it, you do not need to build the substrate. It exists.
- dstack (open source): github.com/Dstack-TEE/dstack
- Phala Cloud (managed): phala.com
- SayGM (privacy-first inference gateway): saygm.com
