Anyone using a DGX Spark?
The prices have increased, and even before people called it too expensive.
I am just wondering - do you have one? What do you use it for? I've been thinking of getting one (as a business expense, I guess the price is high, but then again - what other option do you have for compute? renting GPUs is quite expensive, any comparable amount of VRAM is very expensive - ok if you have space and mostly want to do inference, I guess you have better options (3090 or 5090)
For me, I have just been considering having some more compute available for robotics, finetuning, trying out selfhosted and fine tuned LLMs, among other things. My other setup (M1Pro 32GB and a 1070 Mini) could work for some stuff, but especially for the robotics part it's mostly about NVIDIA...
Curious about any experiences.
It seems like the CUDA monopoly is slowly breaking, but it will still have the best software support for the moment.
If you're pricing it out versus hosted options like OpenRouter, make sure to factor in electricity costs as well. I've not run the numbers myself, but it will likely take at least 5–10 years to pay for itself at current rates. For that reason, it's probably only worth it if you value privacy and independence from cloud providers.
That said, open weight models are excellent, and continue to get better. Everybody is very excited for Qwen 3.8 right now, but there's always something new on the horizon. New models come in all different sizes and quants. Kimi K3 was a whopping 2.8T parameters.
The /r/LocalLLaMA community has a lot of resources on local hosting, and though it's less serious than it used to be, a lot of people there have put down real money on testing different hardware setups. I'm sure you can find more information there.
I realized I never answered you, and I've been since thinking about the sparks often. A small update from my side:
I'm pretty happy with having these local machines, since I want to also work in this field, and having some compute for local developments takes off the coordination task for renting cloud GPUs (even if they would've been cheaper)
In the end, it seems more like the investment of this hardware is less of a business cost, and more of an investment, unless the prices come down eventually (which would be a great thing anyways) - so I guess it's either great that I have secured some computing power, or it's great because prices have normalized.
They’re only approaching worth it if you need to experiment with 200GbE and RDMA, or really need CUDA specifically, in my opinion. Which is no small thing, that’s an incredibly important part of the tech stack to have experience with if you’re going to be deploying across nodes in a datacenter, and being able to play with that hands on is pretty damn useful! But everything else the Spark does I see done better elsewhere.
If you’re going “true” homelab you can get better performance and the same 200GbE networking with used GPUs and ConnectX cards from eBay, so as I see it the Spark specifically is for companies/universities that want to give a few engineers some datacenter-like boxes to play with without allocating actual servers that could be doing real work, or scouring eBay to hack something together like us plebs.
If you want local models specifically and don’t care about the networking side, a 128GB or (soon) 192GB AMD or Apple device with unified memory is the way to go for a sane option, or a box of old 3090s is the price/performance king if you don’t mind figuring out power draw and PCIe contention and things.
To actually run LLMs efficiently, you won’t beat the 1/1000th of a server that you pay for a few hundred thousand tokens at a time from CoreWeave or Runpod or Nebius or whoever, those are optimised to within an inch of their lives and kept at capacity 24/7 to get the most out of the capital spend. Local running is for privacy or education, not cost. And privacy/education are both super important, you’ve just gotta know you’re paying a premium for them, sadly.
Robotics is the one thing I’ll say could change the equation if it’s still really NVIDIA focused: not my field, so I can’t tell you anything useful you don’t know there, but if there’s significant vendor lock in it could make sense. That said, the stuff I see in other parts of the industry is wayyyy more GPU-agnostic than it was even a year ago, so it’s less of a push at the single device side for me (they’re still the kings for raw performance if you’re speccing a big server though).
Source: have a rack of used GPUs on 100GbE network for my consulting business (scientific ML, not LLM; I need less VRAM and more fp32 performance, which is why that makes sense. Also I bought before the rampocalypse), chose a 128GB HP G1a (AMD Strix Halo) when the company I’m currently working for separately on LLM stuff asked what hardware I needed. I’m sure I could request a Spark (probably even two plus a network switch) from them if needed, but I wouldn’t spend my own money and I didn’t see the need over the Strix Halo laptop even when I was spending someone else’s 🤷
Thanks for the insightful answer.
I also think that the Apple Studio might be a good purchase for pure LLM work. I just like having access to NVIDIAs ecosystem, and even though the Tokens per second aren't the fastest, they're good enough for me to prototype and I might try putting it into a CoLocation to see how well it performs with good networking. I just really also cared about the portability + robotics aspect, which is why I went with the spark.
I am currently running different projects for robotics (using a LLM harness) and LoRa with my sparks.
FWIW both devices have just increased in price since I bought them, so I am not that worried about the cost - writing them off makes it not THAT big of an investment, and having the option to resell them for potentially more than I bought them for, is even a plus. I guess in a way it's both compute and a diversified portfolio (is that how watch people justify their expenses?)
Yeah it sounds like you very much fall into the niche that’ll actually get value out of these, and more broadly I do think that most people who buy at least two of them and actually use the 200GbE are also mostly in that bucket! The rack full of eBay and AliExpress parts from datacenter teardowns plus the dedicated heat pump to cool the damn thing is a commitment all its own…
I was more spitballing for the “audience”, especially after the price rises, because I’ve actually had a few acquaintances ask me for advice on this recently (which has then been “try managed inference first, then a cloud VM, then buy a Strix Halo or Mac Studio once you know your use case”). But those aren’t the people building robots or worrying about microsecond-level RDMA timings!
I despise unnecessarily gendered memes, but if ever there were a good case for a “boy math” tag… (and I say that knowing 100% I’m no better!)