tl;dr because most llm applications stream text tokens to you, if you can inspect packet sizes between the client and server you can try to predict what the underlying text with the variance of...
tl;dr because most llm applications stream text tokens to you, if you can inspect packet sizes between the client and server you can try to predict what the underlying text with the variance of the token sizes in a sequence. Researchers made a model where they claim to have 29% accuracy on predicting the semantic value of a sequence of encrypted tokens.
I love this kind of research: it’s impressive that they managed it at all, it exposes a flaw that a lot of people just wouldn’t have thought of, and as far as I can see it gives a very...
I love this kind of research: it’s impressive that they managed it at all, it exposes a flaw that a lot of people just wouldn’t have thought of, and as far as I can see it gives a very straightforward mitigation in just padding or chunking tokens to disguise their length.
Haven’t had a chance to read in detail yet, but it’s interesting that Google’s platform is already immune - they do love their protobufs, so I’m thinking maybe that’s inherently obscuring the individual tokens?
tl;dr because most llm applications stream text tokens to you, if you can inspect packet sizes between the client and server you can try to predict what the underlying text with the variance of the token sizes in a sequence. Researchers made a model where they claim to have 29% accuracy on predicting the semantic value of a sequence of encrypted tokens.
I love this kind of research: it’s impressive that they managed it at all, it exposes a flaw that a lot of people just wouldn’t have thought of, and as far as I can see it gives a very straightforward mitigation in just padding or chunking tokens to disguise their length.
Haven’t had a chance to read in detail yet, but it’s interesting that Google’s platform is already immune - they do love their protobufs, so I’m thinking maybe that’s inherently obscuring the individual tokens?
Very cool. Thank you for sharing this.