First mining, and now AI. It seems that NVIDIA is shaking up to be a hydra in the consumer electronics space. As a gamer who would like to upgrade my video card for 4K raytracing, it seems that...
First mining, and now AI. It seems that NVIDIA is shaking up to be a hydra in the consumer electronics space. As a gamer who would like to upgrade my video card for 4K raytracing, it seems that the cost is likely to remain out of reach for me for some time. I haven't upgraded since a midtier card cost under $500 and mentally that is still my personal limit for a componebt. I had some hope last year when mining went proof of stake finally, and there were reports of NVIDIA scrambling to cancel contracts for chips with TSCM.
If this continues it may happen that NVIDIA becomes more of a commercial retailer (selling to businesses for AI and the like) and AMD could possibly take over as the consumer grade choice. Maybe that's just wishful thinking in my part, though. AMD is looking very attractive to me at this point.
I think there are problems with looking to AMD as a saviour here. AMD too would like a bite of the ML/AI GPU market and may one day succeed, meaning all the same incentives to focus away from...
I think there are problems with looking to AMD as a saviour here. AMD too would like a bite of the ML/AI GPU market and may one day succeed, meaning all the same incentives to focus away from gamers will exist for them too. Secondly, even today AMD has to make fab allocation decisions between high margin datacenter CPUs and less high margin consumer products which is why the Ryzen 3 series has all but disappeared and the Ryzen 5 x600 SKUs are no longer the value champions they once were.
All great points here, thank you for replying. As a budget minded consumer, it's just hard not to feel like I've been edged out of the market. Your comment made me recognize that these companies...
All great points here, thank you for replying. As a budget minded consumer, it's just hard not to feel like I've been edged out of the market. Your comment made me recognize that these companies are not my friend, they're not out to get me into their product, they're out to get as much of my money as possible.
NVIDIAs end goal may actually be to get people like me to move to something like GeForce NOW.
Last console generation I had a PS4 and grew displeased with it -- eventually moved back to PC, playing my Switch on the side. But even in 2020 the prices to upgrade were so high, and the value...
Last console generation I had a PS4 and grew displeased with it -- eventually moved back to PC, playing my Switch on the side.
But even in 2020 the prices to upgrade were so high, and the value proposition from Xbox so good, that I just went there instead. I still have my PC and it still gets use, but I haven't upgraded anything since... and haven't really felt much of a pinch. Anything demanding I play on Xbox, many indies I play on Switch unless they're given out in bundles or free on PC. I mostly only use my PC to play older strategy games or FPSes or emulate.
I haven't upgraded my GPU since I got my 1070ti and it's going to stay there for the foreseeable future. If it's not enough to run a game, I guess I won't play it. I've got thousands more to...
I haven't upgraded my GPU since I got my 1070ti and it's going to stay there for the foreseeable future.
If it's not enough to run a game, I guess I won't play it. I've got thousands more to choose from.
Yeah if it's too demanding I just play on my XSX. I've still got a 980 in mine, so not as beefy as your card (especially the VRAM) but it still survives somehow. For example I'll play Starfield on...
Yeah if it's too demanding I just play on my XSX. I've still got a 980 in mine, so not as beefy as your card (especially the VRAM) but it still survives somehow.
For example I'll play Starfield on my XSX, my PC definitely won't be able to handle it. But I started playing the new System Shock remake on it at medium settings and it plays just fine. Does it look immaculate? No, but I'm not really a graphics snob anyway. I also don't have much desire to play in 4k so that helps.
As someone in the AI space currently, GPU availability is no joke. We are finding it impossible to get smaller allocations. We had a quote for 64x H100 the other day as a minimum. Our tiny startup...
As someone in the AI space currently, GPU availability is no joke. We are finding it impossible to get smaller allocations. We had a quote for 64x H100 the other day as a minimum. Our tiny startup doesn't have nearly enough cash or credit for a debt like that. As a result, advanced features are sitting idle, waiting on us finding capacity. Oh, kicker was that it was a 3 year contract and cost over $3M. Once we have customers and some cushion, it's not totally unreasonable, but right now I just need 2-4 of them, not 64.
You might be able to get them from Dell. My company got a quote a few months back for a server with 2xH100s, plenty of RAM, and 3rd Gen Xeons. Edit to add: with a $3M+ contract, they're pricing...
You might be able to get them from Dell. My company got a quote a few months back for a server with 2xH100s, plenty of RAM, and 3rd Gen Xeons.
Edit to add: with a $3M+ contract, they're pricing something like $47k per card (assuming 64 cards). I thought the list was closer to $35k? The quote you got seems totally out to lunch.
$2/hr/h100 * 3y comes out to about $3.3M (for 64x). A full 8xh100 server from LambdaLabs is $300-$350k (or more if you go for all the options). That'd be $2.4-$2.8M to buy the same set of servers....
$2/hr/h100 * 3y comes out to about $3.3M (for 64x). A full 8xh100 server from LambdaLabs is $300-$350k (or more if you go for all the options). That'd be $2.4-$2.8M to buy the same set of servers. I know volume brings discounts normally, but I also know that supply is greatly constrained right now. Throw in hosting costs and the rental price doesn't seem that crazy.
Ah, I see. I totally misunderstood and thought you were being quoted to buy the cards outright, which is why the price seemed high. If they're coming in servers with latest-Gen Xeon chips,...
Ah, I see. I totally misunderstood and thought you were being quoted to buy the cards outright, which is why the price seemed high. If they're coming in servers with latest-Gen Xeon chips, Infiniband, and terabytes of RAM, then I can see that pricing structure making sense.
Maybe you’ve found a new business opportunity in reselling small allocations from a big contract like that. There’s enough small companies out there to make use of the capacity. Add a small markup...
Maybe you’ve found a new business opportunity in reselling small allocations from a big contract like that. There’s enough small companies out there to make use of the capacity. Add a small markup and it’s easy money.
First mining, and now AI. It seems that NVIDIA is shaking up to be a hydra in the consumer electronics space. As a gamer who would like to upgrade my video card for 4K raytracing, it seems that the cost is likely to remain out of reach for me for some time. I haven't upgraded since a midtier card cost under $500 and mentally that is still my personal limit for a componebt. I had some hope last year when mining went proof of stake finally, and there were reports of NVIDIA scrambling to cancel contracts for chips with TSCM.
If this continues it may happen that NVIDIA becomes more of a commercial retailer (selling to businesses for AI and the like) and AMD could possibly take over as the consumer grade choice. Maybe that's just wishful thinking in my part, though. AMD is looking very attractive to me at this point.
I think there are problems with looking to AMD as a saviour here. AMD too would like a bite of the ML/AI GPU market and may one day succeed, meaning all the same incentives to focus away from gamers will exist for them too. Secondly, even today AMD has to make fab allocation decisions between high margin datacenter CPUs and less high margin consumer products which is why the Ryzen 3 series has all but disappeared and the Ryzen 5 x600 SKUs are no longer the value champions they once were.
All great points here, thank you for replying. As a budget minded consumer, it's just hard not to feel like I've been edged out of the market. Your comment made me recognize that these companies are not my friend, they're not out to get me into their product, they're out to get as much of my money as possible.
NVIDIAs end goal may actually be to get people like me to move to something like GeForce NOW.
Last console generation I had a PS4 and grew displeased with it -- eventually moved back to PC, playing my Switch on the side.
But even in 2020 the prices to upgrade were so high, and the value proposition from Xbox so good, that I just went there instead. I still have my PC and it still gets use, but I haven't upgraded anything since... and haven't really felt much of a pinch. Anything demanding I play on Xbox, many indies I play on Switch unless they're given out in bundles or free on PC. I mostly only use my PC to play older strategy games or FPSes or emulate.
I haven't upgraded my GPU since I got my 1070ti and it's going to stay there for the foreseeable future.
If it's not enough to run a game, I guess I won't play it. I've got thousands more to choose from.
Yeah if it's too demanding I just play on my XSX. I've still got a 980 in mine, so not as beefy as your card (especially the VRAM) but it still survives somehow.
For example I'll play Starfield on my XSX, my PC definitely won't be able to handle it. But I started playing the new System Shock remake on it at medium settings and it plays just fine. Does it look immaculate? No, but I'm not really a graphics snob anyway. I also don't have much desire to play in 4k so that helps.
As someone in the AI space currently, GPU availability is no joke. We are finding it impossible to get smaller allocations. We had a quote for 64x H100 the other day as a minimum. Our tiny startup doesn't have nearly enough cash or credit for a debt like that. As a result, advanced features are sitting idle, waiting on us finding capacity. Oh, kicker was that it was a 3 year contract and cost over $3M. Once we have customers and some cushion, it's not totally unreasonable, but right now I just need 2-4 of them, not 64.
You might be able to get them from Dell. My company got a quote a few months back for a server with 2xH100s, plenty of RAM, and 3rd Gen Xeons.
Edit to add: with a $3M+ contract, they're pricing something like $47k per card (assuming 64 cards). I thought the list was closer to $35k? The quote you got seems totally out to lunch.
Is this the card you guys are discussing?
https://www.cdw.com/product/nvidia-h100-gpu-computing-processor-nvidia-h100-tensor-core-80-gb/7367181
$2/hr/h100 * 3y comes out to about $3.3M (for 64x). A full 8xh100 server from LambdaLabs is $300-$350k (or more if you go for all the options). That'd be $2.4-$2.8M to buy the same set of servers. I know volume brings discounts normally, but I also know that supply is greatly constrained right now. Throw in hosting costs and the rental price doesn't seem that crazy.
Ah, I see. I totally misunderstood and thought you were being quoted to buy the cards outright, which is why the price seemed high. If they're coming in servers with latest-Gen Xeon chips, Infiniband, and terabytes of RAM, then I can see that pricing structure making sense.
Maybe you’ve found a new business opportunity in reselling small allocations from a big contract like that. There’s enough small companies out there to make use of the capacity. Add a small markup and it’s easy money.
Then we are in the hosting business and have to service the clients. I'd thought that same, but I have no interest in being in that specific business.