I didn't play with it long, but the tool got hung up on read/write operations and subprocess calls that cannot be sped up. The AI part in it obviously couldn't help. I did not yet see a way to...
I didn't play with it long, but the tool got hung up on read/write operations and subprocess calls that cannot be sped up. The AI part in it obviously couldn't help. I did not yet see a way to exclude these from the analysis, or to otherwise show anything but these.
UMass Amherst Computer Scientists Use AI to Accelerate Computing Speed by Thousands of Times
UMass Amherst Computer Scientists Use AI to Accelerate Computing Speed by Thousands of Times
A team of computer scientists at the University of Massachusetts Amherst, led by Emery Berger, recently unveiled a prize-winning Python profiler called Scalene. Programs written with Python are notoriously slow—up to 60,000 times slower than code written in other programming languages—and Scalene works to efficiently identify exactly where Python is lagging, allowing programmers to troubleshoot and streamline their code for higher performance.
“Python is a ‘batteries-included’ language,” says Berger, who is a professor of computer science in the Manning College of Information and Computer Sciences at UMass Amherst, “and it has become very popular in the age of data science and machine learning because it is so user-friendly.”
“But Python is crazy inefficient. It easily runs between 100 to 1,000 times slower than other languages, and some tasks might take 60,000 times as long in Python.”
Programmers have long known this, and to help fight Python’s inefficiency, they can use tools called “profilers.” Profilers run programs and then pinpoint why and which parts are slow.
Unfortunately, existing profilers do surprisingly little to help Python programmers. At best, they indicate that a region of code is slow, and leave it to the programmer to figure out what, if anything, can be done.
Berger’s team, which included UMass computer science graduate students Sam Stern and Juan Altmayer Pizzorno, built Scalene to be the first profiler that not only precisely identifies inefficiencies in Python code, but also uses AI to suggest how the code can be improved.
“Scalene first teases out where your program is wasting time,” Berger says. It focuses on three key areas—the CPU, GPU and memory usage – that are responsible for the majority of Python’s sluggish speed.
Scalene is already in wide use and has been downloaded more than 750,000 times since its public unveiling on GitHub. The research that led to the development of Scalene was supported by the National Science Foundation.
A paper describing this work appeared at this year’s USENIX Conference on Operating System Design and Implementation, where it won a Best Paper Award.
Am I crazy in thinking python will never be suitable for performance critical applications? Like, is it fine as a learning language? Sure. Is it suitable for proof of concepts or open source...
Am I crazy in thinking python will never be suitable for performance critical applications? Like, is it fine as a learning language? Sure. Is it suitable for proof of concepts or open source projects to draw in contributors? Absolutely. But should we be productionalizing unoptimized notebooks handed to us from data scientists and then be continually shocked that it’s not great engineering? Or we try to clean up those notebooks and rewrite them with best practices and optimizations, and suddenly you’re not writing beginner-friendly python any more. There’s plenty of other “user-friendly”, “batteries-included” languages, like this article praises python for, that actually compile to far more efficient code. Using AI as a crutch to solve bad engineering decisions is not a great solution.
Yeah I think that’s uncharitable (or crazy). Python runs in production for many high load sites - most notably Instagram and also Threads runs on a modified Django. Your site is not going to have...
Yeah I think that’s uncharitable (or crazy). Python runs in production for many high load sites - most notably Instagram and also Threads runs on a modified Django. Your site is not going to have more load than Instagram. It’s just not.
For the ML notebooks specifically, almost none of the performance bottlenecks is in python, and all the hot paths are FFI with code in C or Fortran.
The bigger issue with running these in production is the brittleness of a highly late binding language, but static type checkers help with that.
Thats probably fair, I’m being uncharitable. I’ve just got a bad view of python cause of the last project I worked on. Complete mess of a project: Data science wrote up a new machine learning...
Thats probably fair, I’m being uncharitable.
I’ve just got a bad view of python cause of the last project I worked on. Complete mess of a project:
Data science wrote up a new machine learning model, and some processing around it using python-only 3rd party libraries.
They had one member with a bit of engineering background who tried to Dockerize the whole thing. That person leaves after the initial deployment.
It falls to my team and specifically me to maintain this project. Meanwhile we’re maintaining a dozen other microservices which are decently written and battle tested.
The python project works fine locally on small inputs, which is probably what the scientists were testing with. But in production, it’s filled with flaws that reveal themselves at scale:
It required massive amounts of memory to load the machine learning model
It needs GTX 2080TI cards to run at decent speeds
The batching logic was naive and poorly thought out, and took weeks to untangle.
There were absolutely no unit tests, because everything was written simply and quickly and it works fine locally, so what’s there to test?
When we considered breaking it out into separate services and rewriting the non-ML portions in a compiled language, we got pushback from the data scientists that they wouldn’t be able to keep hacking in more features themselves.
Meanwhile we’d keep hitting bugs in the python because typing is optional or None kept popping up or cuda errors would corrupt the whole container…
Anyways, I know every language is valid etc. but that project really left a bad taste in my mouth.
Everything performance critical will be a C-based module or something, yes. Which is how AI works in Python already. It's a good setup I'd say, it balances bottlenecks perfectly
Everything performance critical will be a C-based module or something, yes. Which is how AI works in Python already. It's a good setup I'd say, it balances bottlenecks perfectly
Which languages? Python is easy AND seems to be widely used, that's why I use it for hobby stuff and I'd imagine that's why its so widely used by low skilled people like me... so its widely used...
There’s plenty of other “user-friendly”, “batteries-included” languages, like this article praises python for, that actually compile to far more efficient code.
Which languages? Python is easy AND seems to be widely used, that's why I use it for hobby stuff and I'd imagine that's why its so widely used by low skilled people like me... so its widely used because its widely used.
I think we should break this down: Are notebooks a suitable format for production code? I'm a big fan of notebooks but I think of them as a way of exploring ideas and sharing tutorials. I'm...
I think we should break this down:
Are notebooks a suitable format for production code?
I'm a big fan of notebooks but I think of them as a way of exploring ideas and sharing tutorials. I'm skeptical of using them for production code. Wouldn't rewriting the code as a library be better? But I haven't actually seen how they're used. Maybe it's fine?
Is Python fast enough for production code?
Sometimes. Numpy is pretty impressive. Since the interpreter often isn't doing the heavy lifting, this is sort of like asking if a shell script is fast enough to start up a program, or whether make is fast enough to do a build.
(I would prefer not to write C or C++ if I can avoid it, though. The heavy dependency on C extensions seems worse than more modern languages.)
Is Python a reasonable way to deploy production code?
The deployment story doesn't seem as good as Go, Rust, or another programming language where you can build a binary. But you can put anything in a Docker file, so maybe it's fine?
Python, Perl, Javascript, and various shell scripting languages are all "glue" languages. Making it easy to use other precompiled tools quickly. Will a shell script written in Bash leveraging a...
Python, Perl, Javascript, and various shell scripting languages are all "glue" languages. Making it easy to use other precompiled tools quickly.
Will a shell script written in Bash leveraging a dozen C++ tools be faster than a purpose-built C++ tool? Almost certainly not. Will the developer time saved from being able to use Bash as a glue greatly exceed any computational savings over 10 years? For an absolutely staggering number of use cases, that answer is a resounding yes.
I didn't play with it long, but the tool got hung up on read/write operations and subprocess calls that cannot be sped up. The AI part in it obviously couldn't help. I did not yet see a way to exclude these from the analysis, or to otherwise show anything but these.
UMass Amherst Computer Scientists Use AI to Accelerate Computing Speed by Thousands of Times
Am I crazy in thinking python will never be suitable for performance critical applications? Like, is it fine as a learning language? Sure. Is it suitable for proof of concepts or open source projects to draw in contributors? Absolutely. But should we be productionalizing unoptimized notebooks handed to us from data scientists and then be continually shocked that it’s not great engineering? Or we try to clean up those notebooks and rewrite them with best practices and optimizations, and suddenly you’re not writing beginner-friendly python any more. There’s plenty of other “user-friendly”, “batteries-included” languages, like this article praises python for, that actually compile to far more efficient code. Using AI as a crutch to solve bad engineering decisions is not a great solution.
Yeah I think that’s uncharitable (or crazy). Python runs in production for many high load sites - most notably Instagram and also Threads runs on a modified Django. Your site is not going to have more load than Instagram. It’s just not.
For the ML notebooks specifically, almost none of the performance bottlenecks is in python, and all the hot paths are FFI with code in C or Fortran.
The bigger issue with running these in production is the brittleness of a highly late binding language, but static type checkers help with that.
Thats probably fair, I’m being uncharitable.
I’ve just got a bad view of python cause of the last project I worked on. Complete mess of a project:
Anyways, I know every language is valid etc. but that project really left a bad taste in my mouth.
Everything performance critical will be a C-based module or something, yes. Which is how AI works in Python already. It's a good setup I'd say, it balances bottlenecks perfectly
Which languages? Python is easy AND seems to be widely used, that's why I use it for hobby stuff and I'd imagine that's why its so widely used by low skilled people like me... so its widely used because its widely used.
I think we should break this down:
Are notebooks a suitable format for production code?
I'm a big fan of notebooks but I think of them as a way of exploring ideas and sharing tutorials. I'm skeptical of using them for production code. Wouldn't rewriting the code as a library be better? But I haven't actually seen how they're used. Maybe it's fine?
Is Python fast enough for production code?
Sometimes. Numpy is pretty impressive. Since the interpreter often isn't doing the heavy lifting, this is sort of like asking if a shell script is fast enough to start up a program, or whether make is fast enough to do a build.
(I would prefer not to write C or C++ if I can avoid it, though. The heavy dependency on C extensions seems worse than more modern languages.)
Is Python a reasonable way to deploy production code?
The deployment story doesn't seem as good as Go, Rust, or another programming language where you can build a binary. But you can put anything in a Docker file, so maybe it's fine?
Tildes is written in Python, BTW.
Python, Perl, Javascript, and various shell scripting languages are all "glue" languages. Making it easy to use other precompiled tools quickly.
Will a shell script written in Bash leveraging a dozen C++ tools be faster than a purpose-built C++ tool? Almost certainly not. Will the developer time saved from being able to use Bash as a glue greatly exceed any computational savings over 10 years? For an absolutely staggering number of use cases, that answer is a resounding yes.