5
votes
What programming/technical projects have you been working on?
This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?
I've just pulled apart, cleaned, and updated my rackserver somewhat- replacing the ProArt B650-Creator board with the newer ProArt X870E-Creator, largely because of the jump from PCIE 4.0 to PCIE 5.0 on all slots but the chipset-provided slot, and partly for the jump from 2.5G to 10G networking without needing a PCIE-based NIC of some description. I also jumped from the Intel Pro B50 to the Intel Pro B70 for the 32GB of VRAM, allowing me to experiment with Qwen 3.6 27B locally. On that front, I have a few queries:
- How do you figure out which model will get the best performance to quality ratio given what hardware you have? I've seen a lot of discussion about whether to use the 27B model or the 35B A4B (mixture of experts?) model, and how the former is better for complex programming tasks, whereas the latter will generate tokens a lot quicker due to only loading a portion of the parameters into memory at a time, but at a loss of quality. And that's just two sub-models- what about comparing Llama to Gemma to Qwen, etc?
- Does anybody use Open WebUI? How much more comprehensive is it than llamacpp's built-in llama-ui? Is it easy to switch them out?
On other fronts, I continue to refine a number of tools which I use for my smart home control, various REST endpoints, and my RSS / misc. app bot.