10
votes
Opinions on NPU laptops?
Looking to buy a new laptop, and of the model I'd like (thinkpad carbon x1), it seems like I can choose one with or without an NPU.
My surface-level understanding is they make built in AI processing and a couple other niche uses more efficient. Flip side, they're maybe a marketing gimmick?
Price aside (price difference doesn't seem too great), I'm wondering: should I buy a model with an NPU to help potentially future proof? Are there potential downsides to an npu model? Upsides?
I know there are lots of technical people here, I'd love to hear your thoughts or experiences!
They aren’t a gimmick per se, they do genuinely do the job of accelerating tensor calculations well, but in my opinion it’s too early days to realistically worry about which one you get (if at all). There just isn’t any specific killer application or agreed upon standard yet, so it ends up being an exercise in trying to predict what you might need in the next few years, and the software (and hardware) landscape is just moving too fast for that right now.
GPUs are generally faster, more capable, and better supported - not to mention you can use them for gaming as well! If you want to be able to mess around with local models, get an NVIDIA GPU with as much VRAM as you can afford (or an AMD SoC like Strix Halo - they have a shared memory architecture that works well for this, at the expense of worse software support).
If the NPU is bundled in with the laptop anyway and doesn’t affect the price too much, it’s probably a nice to have - they’re cheaper and more power efficient than GPUs, in theory - but it’s a coin flip on whether one you get now will be properly tailored to what’s needed in a couple of years, and whether it’ll still have software and driver support even if it is.
Strong second on those AMD SoCs with unified memory and integrated GPUs. Better power efficiency and likely better software support in the long term than some bolted-on NPU. But if you aren't already using local AI for daily tasks, I wouldn't worry about either. That cheap cloud AI gravy train is still flowing (for now) and it's much harder to rewrite client software to use a breadth of different local hardware and software anyway, so a lot of companies may never do it.
This is very helpful, thank you! I've realised both the models I was initially looking at contain (intel) processors with an NPU. I may look at some different lenovo models just to compare what's available. Though in the end, I doubt anything I end up doing on my laptop will push a processor to its limits (even for for gaming, I use my partner's desktop), so it'll likely all be picking at straws.
Still, I was getting totally overwhelmed by new terms and marketing, and I'd at least like to have a general idea of what I'm purchasing :)
I'm pretty sure my laptop has this but I don't even know if it's being used.
It seems like even if you are planning on using it to develop AI applications you would do better by supporting GPUs first as that's what everyone else is primarily supporting
Thank you, these threads have more detailed info than the ones I (admittedly briefly) searched for!