9
votes
What programming/technical projects have you been working on?
This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?
After I tried DIY remastering some music albums ruined by loudness wars (like Californication by Red Hot Chilli Peppers or God Hates Us All by Slayer - not Death Magnetic, that one is hopeless) with success, I became frustrated by the fact that there's no useful and affordable declipper. A declipper restores parts of the waveform that were cut off in the process to make the recording louder, which creates unpleasant distortion, so declipping firstly slightly increases dynamic range, but mostly it's necessary to remove that distortion.
There are very few options available and the price of the good ones (that I used in the process above) starts at 200€ I think.
So I started looking through scientific papers on the topic and decided to implement one of the close to state of the art methods - in python, because that's what I work with right now, so it's going to be slow, but at least something free will exist.
The paper actually provides a test implementation in Matlab, which is not very useful on its own, but would serve as a good base. The main problem here is that as I went into it, I knew almost nothing about the math used and did not understand any of the methods described in it. I decided to just dive into it using ChatGPT o1, let it translate the whole thing into python piece by piece and then gradually repair what doesn't work.
ChatGPT o1 is almost like magic. The process worked and let me gradually learn how the algorithm works as I was repairing it, o1 was excellent for explaining things and, as opposed to GPT 4o, it actually found and correctly fixed bugs when I asked it to and described the problem.
I'm now at a stage where I mostly understand how the whole thing works, but there's an issue somewhere that likely has something to do with specific implementations of (inverse) short time fourier transform in scipy and Matlab, which causes the optimisation algorithm that's the core of the whole process to converge at least 3x slower per each step than the original Matlab code. No idea why yet. But I'm honestly chuffed by the fact that the whole process even worked and allowed me to create something that does actually declip, although not ideally, in just a couple days.
As someone who listens to and produces a lot of EDM, the idea that the loudness wars are still something people complain about is very funny to me. I’ve come to love heavy clipping in music, and the examples you listed seem nowhere near loud enough to warrant any “fixing”, that’s just how the albums sound. I do appreciate that you’re trying to make a free alternative declipper, some audio software is prohibitively expensive.
Ha! I enjoy controversial opinions, but Californication is perhaps the second most talked about album with regards to being ruined by loudness wars, after Metallica's Death Magnetic (which, true, is yet on another level). Slayer not so much because fewer people listen to Slayer overall and the old albums are the most popular anyway (though their last album, recorded by a different producer who attempted to compress it more transparently, has more plays than any of the aggressively clipped albums despite arguably being musically worse).
Both albums contain audible clipping distortion and both have reduced low-end energy in the mix so that they could be pushed louder. That is not a great decision with RHCP, a band whose possibly most influential member is the bass player. God Hates Us All I find tiring to listen to especially with higher volume (which is something that should be expected with this kind of music) behause of the thin sound and distortion. It's also one of those things that gets worse and more audible the better your sound reproduction is.
When you compare God Hates Us All to some of Slayer's late 80s albums or just to Repentless, which is nearly as loud but mixed and mastered to reduce the audible distortion and avoid thin sound present in GHUA, and compare Californication to Blood Sugar Sex Magik, I think it becomes obvious how fucked those albums really are.
I don't mind the loudness itself - I do think it's stupid because on decent loudspeakers or headphones more dynamic recordings simply sound better, but I'm the minority and if lower DR albums sell better, that's what we get - but I do mind it if it brings audible distortion and stupid spectral balance just to make it louder. It was a conscious decision to make it sound more aggressive, but it was a stupid and unnecessary one.
For the record I don't mind loudness or distortion where it actually fits. I listen to breakcore, among other things. I just think that Rick Rubin sniffs his own farts too much and needs someone who tells him when his production ideas are stupid. I only tried to fix the recordings because I like the music but the sound annoyed me personally.
I just re-listened to californication (the song, not the whole album) and, holy shit, you're right about that clipping, "fucked" is a perfectly good way to describe the chorus. I don't think I agree about the reduced low end or overall "energy", it sounds good to me. I can't make out that much of a problem in comparing the two slayer examples - Repentless has more low end juice for sure, but I don't think that takes it out of "that's just how the album sounds" territory.
My understanding of the loudness wars comes from dubstep (brostep and riddim, if you're being specific about genre), where it seems the consensus is that mastering loud is part of the sound design process, so everyone just stopped caring. The difference in a clipped, mastered track and an unmastered one is pretty huge, and we've basically reached peak dynamic crushery at this point. If a track is totally fucked then it mostly gets criticized as "bad mixing" at this point because time has shown that there's definitely a way to get a super loud track sounding good.
On a broader note about distortion, it seems like crazy distortion is becoming more and more widespread, even in popular music where it wouldn't usually be the case. I'm not talking about loudness war distortion from trying to push the volume, I'm talking tracks being crushed in a way that could never happen accidentally. I noticed it in more fringe genres like hyperpop initially, but that makes sense because the musical decisions are often counter cultural (I quite like the linked track, for instance). Stuff like phonk and jersey club making rounds through tiktok and other social media seems to be pushing distortion way harder than before, or at least in a way I haven't heard be popular. I'm tempted to make a new thread about this, because it's not really fit for a technical projects thread.
To return to the original point, I was reminded of this video, which takes it to a whole new level. I was completely wrong about RHCP though, that's brutal.
EDIT: By the way, I'd love to hear your remastered versions of the tracks you mentioned, for the sake of comparison/
Without meaning to offend, saying "that's just how the album sounds" is meaningless. When mixing and mastering Californication, the producers involved no doubt had good enough gear and hearing to know very well about the distortion they're creating and kept it there as a creative decision because they wanted it to sound like that. This is made even more obvious with Death Magnetic, listen to a bit of this if you don't know it. It sounds ridiculous, and when you look at the waveform, it's obvious that Rick Rubin made a conscious effort to distort it as much as possible, because it uses methods that create distortion without necessarily making it louder: there are some channels that were heavily hardclipped and then highpass filtered, and as a result parts of the waveform are cut off despite being at like -6 dB, so their clipping did not bring any more loudness, objectively.
The Slayer album also has audible clipping in places, and on top of that it contains other creative decisions that I consider simply wrong. In general the idea when mixing and mastering it seems to have been that the music itself is not aggressive enough and it's necessary to make the sound itself aggressive as well, which I think is understandable when taken at face value but nonsensical when seen in a broader context of the genre.
Specifically what I see as mistakes, apart from clipping, is: overall spectral balance, sounding thin, which is often done in loudness wars because the less bass there is the louder you can push the rest. Fixable. Heavy compression without automation, which means that in a breakdown with no drums or vocals, where there's just guitar or bass playing for a bit, the guitar/bass suddenly sounds 2x louder, reducing the impact of the breakdown and the full band coming back. Fixable. Guitar solos being equalized to have a peak around 2 kHz, making them slightly painful to listen to when played loud. Fixable. Vocals compressed to seven hells, reducing their impact because when everything is exactly equal loudness, it doesn't matter how loud or how aggressively he's shouting and screaming, everything is the same. This one I cannot fix.
Regarding my remasters, I just did a couple songs so far and I'm not sure if I consider them finished because I later moved on to the programming part, but I'll send you (or others who ask) links in a DM. Can't share them publicly for obvious reasons.
edit: I love Dan Worall btw, brilliant source. I don't listen to hyperpop or phonk or brostep (in fact as a fan of "classic" dubstep i fucking hate what brostep did to the genre :D ), but I used to listen to some less hyperpopy postclub and more ghetto baile funk, which also tend to sometimes be a bit extreme sound-wise.
I've moved on from developing my self hostable personal library catalog (for now) and started a react portfolio website, I tried it once before but got stuck somewhere, and after working on the catalog application I figured I should circle back and try to learn react again, and it's going well!
Although I just deployed to my Linux web server and it's live on the web, it didn't go 100% and right now I'm troubleshooting the blogs backend component. I'll figure it out though.
My diskprices-on-eBay site was fairly successful (made $80 over the last 30 days!)--there's still a bit of maintenance work to be done to continue to improve it but I'm also thinking of what to do next. Some things that passed through my mind were iPad covers / charging cables (actually, these were the wife's ideas--a site that makes it easy to find compatible things), LEGO sets, luxury bags, and real estate.
There are already quite a number of websites to compare LEGO sets to insanely detailed levels (rebrickable, yottabrick, but also quite a few more!) and my wife pointed out that luxury bags have more complicated methods of appraisal and valuation.
So I'm leaning more towards the real estate idea because I'm also interested in that and I need an excuse to help me finish up a relative poverty index idea that I've been sitting on for a while--more on that next week!
I am finally starting on the idea I had for about a year of creating my own digital audio player (think modern MP3 player for audiophiles). The philosophy behind this is I have been moving more of my life away from using a smartphone as a do-it-all device to having individual purpose built devices. There are a few things that make consumer available digital audio players the wrong fit for me, so I decided to do a DIY implementation. The basic components is I am going to use a Raspberry Pi 3a+ as the primary device, and then attaching a USB dac/amp combo to drive the audio. As I get further along into making it portable, I will use a touch e-ink display, but I still need to source one as I have not done research into that. For batteries I am considering running it on AA batteries, but I will need to do some testing on that first. But that is getting a bit ahead of myself on the hardware. Right now I want to make good progress with the Raspberry Pi on the software side of things on my desk before I start buying screens and batteries