Neural network-optimized hardware like Google's TPUs are still built out of transistors, so they are as much subject to Moore's law as anything else. NNs aren't going to replace the majority of...
That being said, it isn't really the last evolution. You know how AI is a buzzword and neural networks are a thing? It turns out we can optimize hardware to support those things a lot better than we can program our way out of their limitations.
Neural network-optimized hardware like Google's TPUs are still built out of transistors, so they are as much subject to Moore's law as anything else. NNs aren't going to replace the majority of software anytime soon, and while it's possible to create hardware that is more specifically suited to one type of computational workload, there are always tradeoffs. And going down that route could stifle innovation, because the strength of general-purpose CPUs is that they are pretty good at everything, even if there's a much smaller subset of things they truly excel at. So, you can build anything and it'll be pretty decently performant on stock hardware. In the 1990s we started making separate chips optimized for graphics, and now people are starting to make chips optimized for neural networks. But we can't keep building new hardware for every niche.
The real "last step" will be whenever quantum computers become viable, which isn't looking like anytime soon. It seems to me that field is somewhere analogous to 1940-1950 in the world of conventional digital computers, where computers are physically very large, with next to no memory or computational power, and largely confined to research.
You may be right about that, but it's an issue if only the biggest companies can afford to create the hardware they need to be performant. I know FPGAs exist, but I wonder if there's anything like...
To clarify, my point is that I think application specific hardware design will be the next step.
You may be right about that, but it's an issue if only the biggest companies can afford to create the hardware they need to be performant. I know FPGAs exist, but I wonder if there's anything like 3D printing for chipsets in the pipeline. I know that isn't viable right now, but it would be a sad day if we run into the limits of general-purpose CPUs before there's a way for a startup to cheaply and efficiently create hardware that can compete with the Googles and Facebooks of the world - which you can buy off the shelf today.
The EUV tech that makes sub-7nm process nodes possible is astronomically expensive as things stand, so I doubt that that will make it to consumer or even small/medium business level hardware...
The EUV tech that makes sub-7nm process nodes possible is astronomically expensive as things stand, so I doubt that that will make it to consumer or even small/medium business level hardware anytime soon.
As an armchair observer, I'll be interested to see if anyone starts offering easy/cheap custom chip fabrication at the 20-100nm range in the same way that people started offering simple custom PCB fabrication not too too long ago. I'll also be interested to see fundamentally different types of computing hardware pop up, such as some of the weird digital/analog crosses I've seen in research about neural network accelerators (essentially, neuron-like structures that can accelerate neural networks by taking into account amplitude and other factors that a digital chip can't deal with).
Your work sounds super cool! Have you worked directly on any biosensor systems? Or mostly just on lower level components that would be useful across all types of applications?
Your work sounds super cool! Have you worked directly on any biosensor systems? Or mostly just on lower level components that would be useful across all types of applications?
That's awesome. I love hardware hacking projects, and the type of thing where you're interfacing with or monitoring some biological system seems even more interesting. Good luck with your PhD work!
That's awesome. I love hardware hacking projects, and the type of thing where you're interfacing with or monitoring some biological system seems even more interesting. Good luck with your PhD work!
I’m a bit jaded about “this new tech is gonna be the next big thing in chip production,” but this article has me pretty excited to see what the tech can do. It seems like they’re already a decent...
I’m a bit jaded about “this new tech is gonna be the next big thing in chip production,” but this article has me pretty excited to see what the tech can do.
It seems like they’re already a decent chunk of the way to this being something that can be done in production, and they’re definitely showing real benefits from their setup.
Neural network-optimized hardware like Google's TPUs are still built out of transistors, so they are as much subject to Moore's law as anything else. NNs aren't going to replace the majority of software anytime soon, and while it's possible to create hardware that is more specifically suited to one type of computational workload, there are always tradeoffs. And going down that route could stifle innovation, because the strength of general-purpose CPUs is that they are pretty good at everything, even if there's a much smaller subset of things they truly excel at. So, you can build anything and it'll be pretty decently performant on stock hardware. In the 1990s we started making separate chips optimized for graphics, and now people are starting to make chips optimized for neural networks. But we can't keep building new hardware for every niche.
The real "last step" will be whenever quantum computers become viable, which isn't looking like anytime soon. It seems to me that field is somewhere analogous to 1940-1950 in the world of conventional digital computers, where computers are physically very large, with next to no memory or computational power, and largely confined to research.
You may be right about that, but it's an issue if only the biggest companies can afford to create the hardware they need to be performant. I know FPGAs exist, but I wonder if there's anything like 3D printing for chipsets in the pipeline. I know that isn't viable right now, but it would be a sad day if we run into the limits of general-purpose CPUs before there's a way for a startup to cheaply and efficiently create hardware that can compete with the Googles and Facebooks of the world - which you can buy off the shelf today.
The EUV tech that makes sub-7nm process nodes possible is astronomically expensive as things stand, so I doubt that that will make it to consumer or even small/medium business level hardware anytime soon.
As an armchair observer, I'll be interested to see if anyone starts offering easy/cheap custom chip fabrication at the 20-100nm range in the same way that people started offering simple custom PCB fabrication not too too long ago. I'll also be interested to see fundamentally different types of computing hardware pop up, such as some of the weird digital/analog crosses I've seen in research about neural network accelerators (essentially, neuron-like structures that can accelerate neural networks by taking into account amplitude and other factors that a digital chip can't deal with).
Your work sounds super cool! Have you worked directly on any biosensor systems? Or mostly just on lower level components that would be useful across all types of applications?
That's awesome. I love hardware hacking projects, and the type of thing where you're interfacing with or monitoring some biological system seems even more interesting. Good luck with your PhD work!
I'm sure this is a whole topic unto its own... but what are your brief thoughts on Elon Musk's Neuralink?
I’m a bit jaded about “this new tech is gonna be the next big thing in chip production,” but this article has me pretty excited to see what the tech can do.
It seems like they’re already a decent chunk of the way to this being something that can be done in production, and they’re definitely showing real benefits from their setup.