Hrm, this might hold some truth to it but I also have to wonder about some of the arguments made when the article includes things like this Which makes me wonder how much the author knows and how...
Hrm, this might hold some truth to it but I also have to wonder about some of the arguments made when the article includes things like this
To understand this debate, it’s really helpful to understand what it means to actually train an AI model. Writing that up would take too much time and isn’t the focus of this post, so I asked ChatGPT to describe the training process in detail. Here’s its explanation. What’s important to understand about training a model like GPT-4 is
Which makes me wonder how much the author knows and how much of the article is based on querying chatGPT and trusting it blindly...
Regardless, the argument that flows out under there is a bit problematic. The author seems to only want to talk about the training of one model (gpt-4) as if that is the whole sum of it. Which is far from the truth, as it is my understanding that companies like openAI are training multiple models at once for various purposes and sometimes dead-ends. Then there is the fact that openAI is far from the only company spending these resources on training models.
The entire article hangs together from similar arguments. As I said, there likely are some truths in there, but they are also distorted by the author trying to frame them in a very narrow definition.
That doesn't really change it - the point is more that training and research is amortized across the uses of the final result. It's no different than how manufacturing companies will use very...
That doesn't really change it - the point is more that training and research is amortized across the uses of the final result. It's no different than how manufacturing companies will use very energy and cost inefficient production methods during R&D - because at this point, you just want to see if something works at all. In the end, the actual production method matters far more than whatever chicanery you did during R&D, because the final production steps will be repeated millions of times, whereas the researchers did what they did a handful.
That is, frankly, not the point I am making about the article. Regardless, I find your response a bit too much of handwaving for me own liking. Yes, R&D often consumes more resources than typical...
That is, frankly, not the point I am making about the article. Regardless, I find your response a bit too much of handwaving for me own liking. Yes, R&D often consumes more resources than typical production does. At the same time we seem to be in a period of such a mad rush in AI R&D where it mostly comes down to raw power that it is not something you can simply ignore. Even if it “will be fine in the future”, it is an issue now.
As I said, there are likely truths in the article. It is just that the article also has some clear points, to me anyway, where I find it difficult to take them at face value.
Edit:
To make it extra clear, I don't even disagree on many points. That doesn't mean you can't be critical of ways an argument is framed.
Hrm, this might hold some truth to it but I also have to wonder about some of the arguments made when the article includes things like this
Which makes me wonder how much the author knows and how much of the article is based on querying chatGPT and trusting it blindly...
Regardless, the argument that flows out under there is a bit problematic. The author seems to only want to talk about the training of one model (gpt-4) as if that is the whole sum of it. Which is far from the truth, as it is my understanding that companies like openAI are training multiple models at once for various purposes and sometimes dead-ends. Then there is the fact that openAI is far from the only company spending these resources on training models.
The entire article hangs together from similar arguments. As I said, there likely are some truths in there, but they are also distorted by the author trying to frame them in a very narrow definition.
That doesn't really change it - the point is more that training and research is amortized across the uses of the final result. It's no different than how manufacturing companies will use very energy and cost inefficient production methods during R&D - because at this point, you just want to see if something works at all. In the end, the actual production method matters far more than whatever chicanery you did during R&D, because the final production steps will be repeated millions of times, whereas the researchers did what they did a handful.
That is, frankly, not the point I am making about the article. Regardless, I find your response a bit too much of handwaving for me own liking. Yes, R&D often consumes more resources than typical production does. At the same time we seem to be in a period of such a mad rush in AI R&D where it mostly comes down to raw power that it is not something you can simply ignore. Even if it “will be fine in the future”, it is an issue now.
As I said, there are likely truths in the article. It is just that the article also has some clear points, to me anyway, where I find it difficult to take them at face value.
Edit:
To make it extra clear, I don't even disagree on many points. That doesn't mean you can't be critical of ways an argument is framed.