Repository navigation
aproxy: Support Trustee KBS (CoCo) as the backend storage and verifier for attestation - #1122
armenon-rh wants to merge 5 commits into
Conversation
luigix25
left a comment
There was a problem hiding this comment.
I did a quick review.
Don't forget to run make clippy cargo fmt and make test before submitting :)
a39bf21 to
bc5b94e
Compare
|
@armenon-rh please resolve all the conversation that no longer applies. I run not sure why, i didn't investigate, but reverting your changes it passes |
Hi Luigi, |
bc5b94e to
f35b97a
Compare
luigix25
left a comment
There was a problem hiding this comment.
I see a lot of changes in Cargo.lock file, I don't think this is expected..
other than that, I'm not an attestation expert, I don't see anything wrong with the code!
| > git clone https://github.com/coconut-svsm/svsm.git | ||
| 2. The aproxy on the host. | ||
| 3. Trustee KBS server (For now, the KBS softhsm container, is not upstream) | ||
| > git clone https://github.com/armenon-rh/trustee.git |
There was a problem hiding this comment.
this should be kbs-softhsm branch, right? is there an ongoing work to upstream it?
There was a problem hiding this comment.
yes, the POC was done, with changes in my public fork of trustee. I will be sending a PR to the confidential-containers/trustee project.
There was a problem hiding this comment.
confidential-containers/trustee@51bd7bb
I am not sure if these changes should be in trustee upstream. This was only created for convenience.
It has the necessary "manual testing" configuration toml files, that help us set up the container and connect to it quickly with a few instructions.
Please guide @tylerfanelli
stefano-garzarella
left a comment
There was a problem hiding this comment.
@armenon-rh thanks for this, but I think there is a critical issue about moving the hashing in the proxy.
That said, maybe commit 2 should be split in multiple commits if possible.
8e55d86 to
3e50391
Compare
stefano-garzarella
left a comment
There was a problem hiding this comment.
I'm concerned about the protocol between SVSM and aproxy. @tylerfanelli tried to make it generic, but we're adding a lot of non-generic changes here (fixed parameter order for the Raw type, fixed JSON field names for the JSON format). At this point, the protocol is no longer generic IIUC, so is it still worth using this intermediate protocol, or should we remove it and incorporate the attestation protocol directly into SVSM and implement a simple https proxy in the host?
I totally understand. There are 2 enums that I have introduced and it can become confusing. I can explain the changes. AFAIK, SVSM can be deployed anywhere. So when its communicating to a backend storage server (to retrieve the secret key) or a verifier to produce evidence, it needs to know what protocol to use. Since we can have multiple such backends, we have tools/aproxy/backend where all of them can be defined in the future. Apart from that it also helps in the communication (HTTP). Previously we only used to communicate with the kbs-test backend. With this series, we can also communicate with the CoCo Trustee KBS backend. You are right about the order of parameters in the raw binary format. I can bring back the negotiation param and thereby order the payload in RawBinary hash according to the order mentioned in the vector. About the fixed JSON field names,
If a future protocol uses a different schema, we simply:
I also agree that we can revert the renames in libaproxy and SVSM, going back to report and certs_buf. The naming is only required in aproxy (specifically in TrusteeEvidence). So to summarize the flow, SVSM can run in any environment. Aproxy will be started on the host with appropriate command line arguments (kbs, kbs-trustee or anything else in the future). This will tell aproxy to fix a communication protocol (Payload format + hash algo). So when aproxy forwards the nonce from storage backend to SVSM, it explicitly also mentions the protocol. SVSM constructs the runtime_data hash. This is statically fixed. To draw a parallel, just like the kernel uses a compiled-in driver (like VirtIO) to talk to a specific hardware, SVSM uses compile-time static PayloadFormatter for specific attestation standards.
The trustee kbs in our case, takes the cleartext runtime_data -> serializes into JWS JSON -> hashes it. Then compares this hash with Tee evidence report_data (hardware signed SNP report). Please correct me if I am wrong. Note to self: |
Not really, the goal was to make SVSM completely attestation protocol-agnostic. Let me summarize what @berrange, @tylerfanelli , and I discussed some time ago. We went with Option 1 mainly to avoid making changes to SVSM as much as possible. If we see that Option 1 is difficult to maintain, I'm open to switching to Option 2, but not to having something in between that just complicates things.
The goal is clear, I'm worried about the design of the protocol between SVSM and the proxy.
This really brings us back to the discussion of the options I mentioned above. This seems to fall right in the middle of the two, and IMO, we should avoid it at all costs because it takes away the advantages of option 1 and adds the disadvantages of option 2. So IMHO it’s basically the worst-case scenario.
IMO this should be avoided, if we implement option 1, SVSM - proxy protocol should don't know anything about the attestation protocol used, everything should be handled by the proxy and translated into the generic protocol. Or we should switch to option 2 and have a simple http proxy and let SVSM handle everything. I hope I've made it clear that I'm concerned we're no longer doing either Option 1 or Option 2, but something in between that loses the benefits of both. Thanks, |
Thank you so much for explaining the design choices. You have perfectly captured the architectural tradeoffs. I understand the design choice sits right between option1 and option2
The middleground is that the proxy must handle the HTTP layer but SVSM must handle the JwsJson formatting and hashing. This achieves security (from host key substitution) and network safety. I totally understand the concerns. But I could not figure out a way to achieve both at the same time. |
We already discussed about that, and we agreed we can't do that. Indeed we introduced
Right, so the proxy should tell to SVSM how to construct the json, what to put in some field, what name to use, etc. Like we did with
I don't see the point of doing this TBH. You mean ask other attestation protocols to change to be adapted to our internal protocol? Why? At this point we should go through option 2, I don't understand why we have to complicate our own lives and those of others. What are the advantages of this approach, compared with option 2?
We did that without any TCP/IP stack in SVSM. It was my first attempt, let me find the code: https://github.com/stefano-garzarella/RAClients The protocol was very simple: The proxy will be used exactly to avoid TCP/IP in SVSM, otherwise we don't need the proxy at all if SVSM can talk TCP/IP. SVSM will create the entire payload of the message, send it to the proxy with our custom protocol, the proxy will put it in a http packet and send it to the server, opposite on the other direction. |
|
Hi, I have implemented option2. In the new design, aproxy is a simple HTTP forwarder. SVSM does the job of constructing secure runtime_data and hashing it. A combination of Payloadformat and Hashalgo (mentioned using --protocol) allows the user to indicate what kind of data format is expected by the backend while bringing up aproxy. Since we no longer import the external sev crate to match the AttestationReport in aproxy, SVSM defines a few structs in kernel/src/greq/pld_report.rs to help with serialization of binary data into JSON before sending it across to the backend. I still need to add documentation in the PR, but I will start working on it once the design is approved. Thanks |
There was a problem hiding this comment.
Thanks @armenon-rh. I have one main suggestion for this PR. @stefano-garzarella I'd also like your thoughts.
In the negotiation phase of the protocol, we originally designed the NegotiationResponse to contain a set of negotiation parameters that the server expects to be included in the HOST_DATA hash.
struct NegotiationResponse {
/// Challenge returned from the attestation server to verify freshness of attestation evidence.
pub challenge: Vec<u8>,
/// Parameters to be hashed in the specific order defined by the array
pub params: Vec<NegotiationParam>,
}
However, perhaps this is too generic. AFAIU, each server implementation (Trustee KBS being at the forefront of my mind when considering this) has only one expected format that HOST_DATA hashes should follow. With this in mind, why don't we just throw out the negotiation parameters field altogether and instead specify which server implementation is being used, so SVSM can hash the parameters in the order and format expected by the server. For Trustee KBS, this could be as follows:
struct NegotiationResponse {
/// Challenge returned from the attestation server to verify freshness of attestation evidence.
pub challenge: Vec<u8>,
/// The protocol (ordering and formatting of negotiation parameters) expected by the server.
pub hash_protocol: HashProtocol,
}
HashProtocol can be an enum specifying the server implementation, like so:
pub enum HashProtocol {
/// Sample KBS used for testing.
SampleKbs,
/// Trustee KBS.
TrusteeKbs,
}
Then, SVSM can compile the HOST_DATA hash based on the result of the hash_protocol specified in the negotiation response. With this, the hash can be constructed in the correct ordering and formatting (in Trustee's case, JWS JSON).
The downside to this is we're essentially bringing server-specific traits to the SVSM kernel. However, I think this is a relatively small area (i.e. how the negotiation parameters are hashed) requiring this change and thus worth the trouble.
Furthermore, this change is extensible, as when another server proxy comes along, it can specify another HashProtocol enum value and write a module of how to compile the HOST_DATA hash in the way expected by that server.
IMO, this means we lose all the benefits of having a generic protocol between SVSM and the proxy, since if we need to support a new attestation protocol or a new version of it, we have to update SVSM and then have a new launch measurement, etc. At this point, what would be the advantage of having a complicated generic protocol between SVSM and the proxy compared to having a simple HTTP proxy and implementing the entire protocol in SVSM? The advantages of the latter are a very simple proxy, a simple protocol between the proxy and SVSM, and the entire attestation protocol implemented in a single trusted location (i.e. SVSM), so we can avoid any bad assumption (e.g. making the host to perform the hash, etc.). |
3e50391 to
f0f21bd
Compare
Thank you @tyler for the detailed explanation. This is what I am adding. NegotiationParam is still kept so that kbs-test work with the RawBinary formatter |
f0f21bd to
ea27d87
Compare
The sev crate included brings in some dependencies. |
There was a problem hiding this comment.
This is a breaking change and we need to update all the doc accordingly as we don't support kbs anymore.
aproxy --protocol kbs would not work.
Is it really worth it to rename it? Open question for you and the maintainers.
There was a problem hiding this comment.
I had kept it the same before. But a more descriptive name indeed helps us distinguish between the two.
I do not have a strict preference. Kbs can mean the mock KBS and KbsTrustee would indicate the other.
caee516 to
7e7f6a0
Compare
|
changes in v3:
|
luigix25
left a comment
There was a problem hiding this comment.
there are some typos in the commit messages, please fix them
| serde.workspace = true | ||
| serde_json.workspace = true | ||
| vsock = "0.5.1" | ||
| sev = { version = "8.0.0", features = ["snp", "serde"] } |
There was a problem hiding this comment.
can we add default-features = false so we pull in less dependencies?
| if request.version != (0, 1, 0) { | ||
| return Err(anyhow!("invalid request version")); | ||
| if request.version != (0, 2, 0) { | ||
| return Err(anyhow::anyhow!("invalid request version")); |
There was a problem hiding this comment.
mmh why this anyhow::anyhow?
There was a problem hiding this comment.
The use of bail was removed, or I can say, impl TryFrom<&AttestationRequest> for KbsEvidence - was removed . So the import was also removed.
I will explicitly bring anyhow in the scope use anyhow::{anyhow, Context};
|
|
||
| Ok(Self::Snp { | ||
| snp_report: BASE64_STANDARD.encode(report), | ||
| certs_buf: certs_buf.clone().map(|certs| BASE64_STANDARD.encode(certs)), |
There was a problem hiding this comment.
what about using as_ref() instead of clone?
| NegotiationParam::Challenge => { | ||
| buffer.extend_from_slice(challenge); | ||
| } | ||
| #[allow(irrefutable_let_patterns)] |
There was a problem hiding this comment.
this is pre-existing, but I don't think we need it, do we?
There was a problem hiding this comment.
yes, make did not complain after removing it.
Against the production-ready Trustee KBS (CoCo), attestation fails immediately because the cryptographic hashes do not match due to 1. differences in payload format (JWS JSON vs binary), 2. hashing algorithms (SHA-384 vs SHA512) and 3. key binding structures. Structured formats like JWS JSON have rigid, nested schema rules (mandated by RFCs/KBS standards). A flat list of fields (a vector of negotiation params) is semantically incapable of describing nested metadata (e.g., "alg", "crv", "kty") or serialization details. However, it is interesting to note that both the legacy mock protocol and the Trustee KBS spec have strict payload schemas. The remote verifier (Trustee) statically validates a predefined JSON object containing the public key and the challenge nonce. It does not support dynamic/partial fields. In this commit we introduce 2 enums, HashAlgo from the kbs_types crate and PayloadFormat (with RawBinary, JwsJson variants). Pass them in NegotiationResponse so that SVSM's hash dispatcher can match on them directly. NegotiationParam was originally introduced to hash the fields in a particular order. That will not be necessary with JSON data. Update SVSM's attestation driver to dynamically consume HashAlgo and PayloadFormat from NegotiationResponse. Remove the hardcoded Sha512 loop. prepare_report_data() function allows us to fit in the hash digest (48 bytes if sha384 algorithm is used) into report data (64 bytes). Bump the negotiation request versin. Additionally, introduce new unit test cases to verify both RawBinaryFormatter and JwsJsonFormatter implementations. Signed-off-by: Arun Menon <armenon@redhat.com>
Restructure the KBS backend to use a trait (KbsBackend) to support multiple backends modularly. Split KBS backend into a directory structure: - tools/aproxy/src/backend/kbs/mod.rs: handles common KBS negotiation and attestation. - tools/aproxy/src/backend/kbs/sample.rs: dedicated for the legacy/mock Sample KBS backend. This prepares the proxy for easily adding more backends, such as the Trustee KBS backend, by implementing the KbsBackend trait. Signed-off-by: Arun Menon <armenon@redhat.com>
…ructure The trustee KBS depends on AttestationReport from the virtee sev crate. Add the official sev crate to match the AttestationReport structure in aproxy. Introduce TrusteeKbs in tools/aproxy/src/backend/kbs/trustee.rs implementing the KbsBackend trait, decoding the SNP attestation report using sev::parser::Decoder. Signed-off-by: Arun Menon <armenon@redhat.com>
Extend ArgsBackend command line enum with a new TrusteeKbs variant. SampleKbs remains the default to retain backward compatibility. Depending on the selected CLI flag one of the two backends will be selected for communication. The TrusteeKbs variant supports JWS JSON payload format and Sha384 algorithm to hash. Wire up both SampleKbs and TrusteeKbs generic protocol backend variants under the Protocol enum to enable static dispatch of the attestation phases. Signed-off-by: Arun Menon <armenon@redhat.com>
KBS-SVSM.md guide details how to build, install, and run SVSM in cooperation with Trustee KBS. Signed-off-by: Arun Menon <armenon@redhat.com>
7e7f6a0 to
01aef2b
Compare
|
changes in v4:
|
Currently, attestation only works with the mock kbs server.
Against the production grade Trustee KBS, attestation fails due to
This PR fixes the attestation flow against a trustee KBS (CoCo) server.
In order to test the changes, follow the steps below
Build svsm, aproxy.
FW_FILE=</path/to/OVMF.fd> make FEATURES=svsm:vtpm,svsm:attest,svsm:virtio-drivers,svsm:block igvm; make aproxy; make clippyStart the kbs container or the kbs-test program. (See Documentation/docs/developer/KBS-SVSM.md in the current PR)
podman container start kbs-resource-serviceOR
kbs-test steps
cargo run -- --measurement <hash># for example cargo run -- --measurement fd4b9d0cd4c5956c579c09b76e9136168eb685514427a33ca640bc1ba3e853fe1267efdc86bd1b720140fdf2f3b58d66Start aproxy in trustee-kbs or sample-kbs mode.
./bin/aproxy --protocol trustee-kbs --url http://0.0.0.0:8080 --vsockOR
./bin/aproxy --protocol sample-kbs --url http://0.0.0.0:8080 --vsockStart the guest using launch_guest.sh script.
./scripts/launch_guest.sh --vsock 3 --qemu </path/to/qemu-system-x86_64> --image <qcow image>Verified that the attestation token is successfully issued by the trustee KBS instance, see kbs container logs below
Changes in v2: