User-friendly and privacy-friendly LLM experience?
I've been thinking perhaps I'll need to get one of the desktop LLM UI. I've been out of touch with the state of the art of end user LLM as I've been exclusively using it via API, but tech-y people (who are not developers) mostly talk about the end-user products that I lack the knowledge of.
Ethical problems aside, the problem with non-API usage is, even if you pay, I can't find one that have better privacy policy than API. And the problem with API version is that it is not as good as the completed apps unless you want to reinvent the wheel. The apps also may include ads in the future, while API technically cannot as it would affect some downstream usecases.
| Provider | Data Retention (API) | Data Retention (Consumer) | UI-only features |
|---|---|---|---|
| ChatGPT Plus | 30 days, no training | Training opt-out, 30 days for temp. chat, unknown retention otherwise | Voice, Canvas, Image generation in chat, screensharing, Mobile app |
| Google AI Pro | 0 | 72 hours if you disable history, or up to 3 years and trained upon otherwise | Android assistant, Canvas, AI in Google Drive/Docs, RAG (NotebookLM), Podcast generation, Browser use (Mariner), Coding (Gemini CLI), Screensharing |
| Gemini in Google Workspace | See above | 0-18 months, but no human review/training | See above |
| Claude Pro | 30 days | Up to 2 years (no training without opt-in) | Coding, Artifact, Desktop app, RAG, MCP |
As a dual use technology, the table doesn't include the extra retention period if they detect an abuse. Additionally, if you click on thumbs up/down it may also be recorded for the provider's employee to review.
I don't think OpenWebUI, self hosted models, etc. would suffice if they are not built to the same quality as the first party products. I know I'm probably asking for something that doesn't exists here, but at least I hope it will bring to people's attention that even if you're paying for the product you might not get the same privacy protection as API users.
There are recent legal developments concerning ChatGPT's data retention due to a lawsuit, OpenAI has written more about it in this blog post. Relevant parts:
I've built a machine for local inference. So far I've run quant and mixtral models, both through ollama and open-webui.
It's shockingly fast.
I tore it down and am rebuilding the OS and such properly (did a build and quick POC first).
My goal was to challenge myself to do it, and see if I could spend a pile of money to stop paying $20 / month to Open AI.
Yeah @whs not sure what your needs/use case are, but Ollama + omost & phi & various other specialized models have done very well for me locally.
Also (if this isn't allowed let me know, no affiliation beyond appreciation) plug for boodlebox, which is a silly name but a great option lol. They provide siloed access to all major bot options for $20/mo (so same as one mainstream subscription), with none of your info going back to the mothership (so they say), & you can upload your personal docs to your "box" to create an individualized reference library to get more specific results. The founders are former milsec, idk if that's a pro or a con for you but they talk very seriously about privacy, & if you're at all connected to higher ed you can get a year of access for free. The only real con is no API access, but worth it imo.
I have Ollama running locally, I'd say Gemma3 27B is quite good. I also have OpenWebUI running behind CF Access. When Debian finally fixes Intel Arc driver I'll finally have my home server running local AI instead of calling to my desktop which is rarely online when I'm on the phone.
But I think I've been missing out on NotebookLM or perhaps just sharing screen to Gemini and ask it to solve things. I also doesn't want to spend time build whole stack adding MCP after MCP, and then catch up on what commercial AI providers has been offering each months. I also have never tried Canvas, I think Gemini at work suggested it once and I was quite impressed. I tried Cursor a bit when work gave me Cursor, it felt quite good when compared to even GitHub Copilot on VSCode. I tried Continue on IntelliJ with local models and it just ate my undo history, and Roo Code's completions doesn't feel as good as cursor even with commercial models (although I'm impressed when it gave me choice of answers).
In terms of AI, I feel like I'm in the DOS era while people are working on iPads.
My Ollama also doesn't work when I'm outside with my phone while Gemini is just one simple home button hold away. When I realize there's no open source software exists today to replace Gemini with any API clients, I think I'd need to do something.
I was going to just upgrade my Google One to AI (since my dad is also a heavy AI user, and he is already paying for Google One Family) but then I realize that the privacy policy is quite bad - you'd need to lose the history feature, which is valuable, in order to get any privacy. There's no "I can see this message but Google promise to never look at it" option like the Workspace version, even if you pay. It feel like the feature is right there - they're selling the thing I want business today - but you can't buy it without accepting the downsides of the enterprise version. (Minimum order quantity, features may be available to enterprise later, lower limits than the consumer version, can't upgrade existing account, etc.)
Nah, not really. Most of it is still overhyped and it barely does what it promises, if it does still with plenty of stuff needing you to babysit it. To be very frank, you seem to be enticed by what all these offerings appear to do. While I can see you wanting to explore these options, just to keep yourself informed I can confidently say that you are not lagging behind in any meaningful way.
A few days ago this blog post was posted on Tildes. While not everyone likes the authors writing style, it is exactly about this. A lot of the hype around AI is still hype. A lot of the folks advocating that you should be on the latest and greatest to "use it the right way" were saying that exact same thing a few months ago.
So again, I don't think you should feel like you have to be trying these products out just to not miss out.
I've been speaking with my former work buddy who is also big into this world. He's been doing a lot with Desktop Commander, whereas I have not. I'm trying to play catch up with my lovely local
inference-boxas I'm calling it. I basically summed up my experience with running locally as -"I can't get anything approaching OpenAI or Anthropic's all-in-one, easily, today. For example, in OpenAI, I can easily transition a conversation between image recognition, code, conversation, etc. If I wanted to do that locally, I would need to be setting up a lot more infrastructure than I've done so far. Right now, I can basically spin up a model that's good at one thing at a time, and work with that."
There's probably a really good niche for an OSS company with a freemium model to make two tiers of products: a easy to use 'run local' for semi-non technical folks; and a highly tunable (but still in-a-box) product for power users.
Related thread: "LLMs and privacy"
Just use DDG's duck.ai
It provides an anonymization layer, and has access to all the big models.
Alternatively follow the other advice in this thread for using Llama, and you can have your own DeepSeek et al running in no time.
Sorry, but without any clarification about what that anonymization layer does it is really doesn't mean all that much.
I did have a look on their website and found this page. So yeah, your call goes through them so things like IP and all that is not present. But I do not agree with their statement about inputting personal data in the prompt.
While technically true, it still means that everything you put in the prompt goes to those providers. Of course, this is also the case if you use a front-end and a provider yourself.
On a more practical note it seems it only supports small models which will influence accuracy a lot. Meaning it isn't something on par with the offerings of openai, anthropic and google in that sense since you are only getting the smaller models.
Have you looked into T3 chat? I personally don't really care about the data retention issues, so I haven't dug through their policy, but it might fit the bill for you. I use it because the user experience is actually noticeably better than the alternative products. I get Gemini pro for free because I have a school .edu email address, but I would still rather pay for T3 chat since the interface is better. And since they use the services' APIs, you get a lot of the benefits of using the APIs that you detailed in your post.
The best front end currently available IMO is Msty: https://msty.app/
It’s like an IDE for LLMs.