diff --git a/posts/secure_llms.md b/posts/secure_llms.md index 3edeb7e9..b3e564bc 100644 --- a/posts/secure_llms.md +++ b/posts/secure_llms.md @@ -253,7 +253,7 @@ Ideally, I would like to see a “multi-layer defense” approach to using remot * **Inference in TEEs**: trusted execution environments are pieces of computer hardware designed to prevent any information leaking other than the output of the code being run inside of them, and able to cryptographically attest to which programs they are running. So you can verify an attestation from the hardware that it’s running _just_ a program that decrypts data, runs LLM inference on it, and encrypts the output, and does not do any logging in the middle. TEEs do get [broken all the time](https://tee.fail/), so one should not view this as cryptographic security; however, inference inside TEEs still greatly reduces your data leakage, as long as you’re actually verifying the TEE attestation signatures locally. In the long run, ideally we make [FHE](https://vitalik.eth.limo/general/2020/07/20/homomorphic.html) efficient enough that we can get full cryptographic privacy for LLMs. Today, this seems to still be far away: the overhead of FHE is high enough, that any model that you can afford to FHE remotely, you can also afford to run directly locally. But tomorrow, that may change! -* **Input sanitization**: a local modl can strip out private data before passing the query along to a remote LLM. Ideally, we have a future where any tasks you need are done by local models “at the top level”, and the local model itself is smart enough to know when it needs to call out to a stronger remote model for support, and what question to ask to leak as little information about you as possible. +* **Input sanitization**: a local model can strip out private data before passing the query along to a remote LLM. Ideally, we have a future where any tasks you need are done by local models “at the top level”, and the local model itself is smart enough to know when it needs to call out to a stronger remote model for support, and what question to ask to leak as little information about you as possible. ### ZK API and mixnets for everything