14 votes

What programming/technical projects have you been working on?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

20 comments

  1. [3]
    Artemix
    Link
    I spend a lot of time using e-mail. E-mail is, quite literally, my favourite format to organize myself for all kinds of purposes. I note my todo-lists by e-mail, same goes for "to do later on". I...

    I spend a lot of time using e-mail.

    E-mail is, quite literally, my favourite format to organize myself for all kinds of purposes.

    • I note my todo-lists by e-mail, same goes for "to do later on".
    • I write my blog article drafts by e-mail
    • I write my stories' chapters by e-mail
    • I prepare meetings, slideshows, and such, by e-mail (thanks to racket/slideshow and slideshow-simple)
    • I send myself reminders by e-mail, which I wish sometimes should go by SMS
    • My contact forms all go by e-mail, to a special folder, which allows me to build a small dashboard based on this folder's content

    Until now, I had a mix of my "main" inbox and my personal data inbox, which was messy and hard to track once I received lots of comms.

    So I decided to build a small setup where a cron-based script would check my mailbox by IMAP, extract every e-mail matching a "known" format, and enqueue the tasks on my job queue broker.

    Behind that comes a few workers, each performing a different task, e.g. creating a draft on one of my blogs, publishing a new chapter on my writings website, sending me a SMS through a SMS gateway, creating a todo list through my todo app, etc.

    I'm about done with the pre-broker part, and am planning the worker's core structure now.

    The way I built it means I can simply do something like the following, and I'll have a new worker ready and running.

    $ python -m pip -i https://my-pypi/ install hook-worker-orders
    $ systemctl enable hook-worker@orders
    $ systemctl start hook-worker@orders
    

    And boom, everything's automatically ready (systemd's config specialization is perfect for this need, I love it) and I have a new task that can be processed without caring much for the rest of the infrastructure.

    9 votes
    1. [2]
      archwizard
      Link Parent
      Wow, that sounds like a super great system. How'd you figure out the systemd aspect? It's always been a bit of a black box for me.

      Wow, that sounds like a super great system. How'd you figure out the systemd aspect? It's always been a bit of a black box for me.

      1 vote
      1. Artemix
        Link Parent
        SystemD unit template files. Basically, the part to the right of the @ is a parameter that'll be used in the template file. Makes deploying and managing new workers dead easy.

        SystemD unit template files.

        Basically, the part to the right of the @ is a parameter that'll be used in the template file.

        Makes deploying and managing new workers dead easy.

        2 votes
  2. archwizard
    Link
    Recently, I decided that I wanted to move as many of my projects out of the Amazon cloud as possible. It'll probably take a while, and I'll spread it out, but I figure it's probably for the...

    Recently, I decided that I wanted to move as many of my projects out of the Amazon cloud as possible. It'll probably take a while, and I'll spread it out, but I figure it's probably for the better.

    This week I moved my VPS and everything hosted there to another provider, and I set up an Ansible playbook to take care of setting up the machine. It sets up Fail2Ban, enables SELinux, does a few security tweaks, and installs a few Docker containers.

    I'd like to move some Lambda functions out of AWS, but I'm not sure how to convert those to Flask (or something else) applications.

    I've also been making steady progress on getting my OSCP. I'm planning on passing first try (fingers crossed).

    4 votes
  3. [6]
    krg
    Link
    I started an idea of modeling musical chords in Rust. Mostly, I've thought about the logic that applies to musical pitches and note names. A chord is just a collection of pitches, so it's not that...

    I started an idea of modeling musical chords in Rust. Mostly, I've thought about the logic that applies to musical pitches and note names. A chord is just a collection of pitches, so it's not that hard to reason about. (unless I'm being naïve).

    Here's what I got, so far:
    // 12-tone music
    const TONES: usize = 12;
    
    // note letters
    const NOTES: [(Midi, NoteName); 7] = [
    	(Midi { n: 0, }, NoteName { letter: 'C', accidental: Accidental::Natural, }),
    	(Midi { n: 2, }, NoteName { letter: 'D', accidental: Accidental::Natural, }),
    	(Midi { n: 4, }, NoteName { letter: 'E', accidental: Accidental::Natural, }),
    	(Midi { n: 5, }, NoteName { letter: 'F', accidental: Accidental::Natural, }),
    	(Midi { n: 7, }, NoteName { letter: 'G', accidental: Accidental::Natural, }),
    	(Midi { n: 9, }, NoteName { letter: 'A', accidental: Accidental::Natural, }),
    	(Midi { n: 11, }, NoteName { letter: 'B', accidental: Accidental::Natural, })
    ];
    
    // reference pitch: A4, in Hz
    const A4: Pitch  = Pitch { 
    	freq: { 
    		Frequency { 
    			hz: 440.0 
    		} 
    	}, 
    	midi: { 
    		Midi {
    			n: 69 
    		}
    	}
    };
    
    // accidentals
    // NOTE: they are encoded in UTF-8 and
    //   many fonts won't display them properly.
    const NATURAL: char = '♮';
    const SHARP: char = '♯';
    const FLAT: char = '♭';
    const DOUBLE_SHARP: char = '𝄪';
    const DOUBLE_FLAT: char = '𝄫';
    
    #[derive(Debug, Clone)]
    struct Chord {
    	pitches: Vec<Pitch>,
    }
    
    impl Chord {
    	fn new(pitches: Vec<Pitch>) -> Chord {
    		Chord { pitches }
    	}
    }
    
    #[derive(Debug, Clone, Copy)]
    struct Pitch {
    	freq: Frequency,
    	midi: Midi,
    }
    
    impl Pitch {
    	pub fn from_freq(freq: Frequency) -> Pitch {
    		Pitch { 
    			freq,
    			midi: freq.to_midi(),
    		}
    	}
    
    	pub fn from_midi(midi: Midi) -> Pitch {
    		Pitch {
    			freq: midi.to_freq(),
    			midi,
    		}
    	}
    
    	fn note_name(&self) -> NoteName {
    		NoteName { letter: NOTES[self.midi.n / TONES].1.letter, accidental: Accidental::Natural }
    	}
    
    	// see: https://en.wikipedia.org/wiki/Scientific_pitch_notation
    	pub fn octave(&self) -> isize {
    		(self.freq.hz / A4.freq.hz).log2() as isize + 4
    	}
    }
    
    #[derive(Debug, Clone, Copy)]
    struct Frequency {
    	hz: f64,
    }
    
    impl Frequency {
    	fn new(hz: f64) -> Frequency {
    		Frequency { hz }
    	}
    
    	// see: https://newt.phys.unsw.edu.au/jw/notes.html
    	fn to_midi(&self) -> Midi {
    		Midi {
    			n: ((TONES as f64) * (self.hz / A4.freq.hz).log2() + A4.midi.n as f64) as usize,
    		}
    	}
    }
    
    #[derive(Debug, Clone, Copy)]
    struct Midi {
    	n: usize,
    }
    
    impl Midi {
    	fn new(n: usize) -> Midi {
    		Midi { n }
    	}
    
    	// see: https://newt.phys.unsw.edu.au/jw/notes.html
    	fn to_freq(&self) -> Frequency {
    		Frequency {
    			hz: 2_f64.powf(((self.n - A4.midi.n) / 12) as f64) *  A4.freq.hz,
    		}
    	}
    }
    
    #[derive(Debug, Clone, Copy)]
    struct NoteName {
    	letter: char,
    	accidental: Accidental, 	
    }
    
    #[derive(Debug, Clone, Copy)]
    enum Accidental {
    	// the `bool` determines whether or 
    	// not we show the natural accidental.
    	Natural(bool), 
    	// the `u8` indicates how many flats
    	// or sharps this accidental has.
    	Flat(u8),
    	Sharp(u8),
    }
    
    fn main() {
    	let freq_0 = Frequency::new(196.0);
    	let freq_1 = Frequency::new(freq_0.hz * (5.0/4.0));
    	let pitch_0 = Pitch::from_freq(freq_0);
    	let pitch_1 = Pitch::from_freq(freq_1);
    	
    	println!("A4.octave() = {}", A4.octave());
    	println!("pitch_0 = {:?}", pitch_0);
    	println!("pitch_0.octave() = {}", pitch_0.octave());
    	println!("pitch_0.note_name() = {:?}", pitch_0.note_name());
    	println!("pitch_1 = {:?}", pitch_1);
    	println!("pitch_1.octave() = {}", pitch_1.octave());
    	println!("pitch_1.note_name() = {:?}", pitch_1.note_name());
    	
    }
    

    Dunno if I de-coupled things too much...or not enough. My main conundrum is deciding how to map a Pitch to NoteName. Pitch is kinda an absolute thing. 440hz is 440hz, after all. But NoteName is a human construct, and can change. That is, we've settled on the idea that 440z = A4, but I can map 440hz to 'cat' if I so feel like and 660hz to 'dog' and a chord can consist of "cat, dog, squirrel" and be valid if that's the system I've decided on. Maybe I'm trying to abstract things away too much? Feels like the right way to move, though.

    4 votes
    1. [3]
      mono
      Link Parent
      Nice! Do you have a project in mind for this or are you just doing it as an exercise? The only change I think I would make is decoupling the Frequency implementation from Midi... only because note...

      Nice! Do you have a project in mind for this or are you just doing it as an exercise?

      The only change I think I would make is decoupling the Frequency implementation from Midi... only because note frequency, as a concept, doesn't have anything to do with Midi. Having the code to produce a Midi on the Frequency impl isn't where I would expect it to be. I think I would instead move it to a static from_freq methods on Midi and keep the to_freq where it is so everything Midi note conversion related is on the Midi interface.

      If you do do that, consider whether you need Frequency at all. If it's just a struct with a single f64 and no methods, it's completely redundant and you can get rid of it and just use a f64 alone.

      4 votes
      1. [2]
        krg
        Link Parent
        Ya, mostly an exercise! But, in the end, I'd like to be able to display some chord voicings and adapt those to chordal instruments (piano & guitar, to start). Kinda wanted to start at the atomic...

        Ya, mostly an exercise! But, in the end, I'd like to be able to display some chord voicings and adapt those to chordal instruments (piano & guitar, to start). Kinda wanted to start at the atomic level, though. I've come to realize that thinking of stuff that way makes everything come together a lot faster than thinking of the complete system from the get-go. I.e, "what's a chord? well, a chord is a collection of notes. ...what's a note? well, a note is a name mapped to a pitch. ...what's a pitch? well, a pitch is a specific frequency (or a MIDI note designation, in my current implementation)" , etc.

        It's def a rough first start. I was 3 glasses of wine deep and feeling inspired, and spit that stuff out...so I'm sure I overthought/underthought some stuff! To me, the absolute fundamental of a sound is its frequency, of course. I suppose coupling that with a MIDI note number is automatically tying myself to a 12-tone equal temperament system. 🤔 But, I thought it'd be more convenient/easier in the long run. And, yea...certainly not everything needs to be stored in its own Struct, and that'd probably be how I'd usually handle stuff. Trying to force myself think "Struct first", though, as I think it allows for better generalization in the long run. Basically, trying to get used to creating types for things.

        2 votes
        1. mono
          Link Parent
          Totally! Thinking about complex concepts in terms of structured data and the relations between them is such a such a great way to gain insight and develop comprehension, and code is the perfect...

          Totally! Thinking about complex concepts in terms of structured data and the relations between them is such a such a great way to gain insight and develop comprehension, and code is the perfect way to do that.

          For off-the-cuff, it's great. I do a similar thing all the time, where I'll define some structure or abstraction when initially laying things out, eventually decide it's redundant and refactor it out, and often, I'll later change my mind after playing around with it and finding a reason to have it. It's all part of the process.

          3 votes
    2. [2]
      ShroudedMouse
      (edited )
      Link Parent
      I've toyed with modelling music in the past and it can quickly get overwhelming if you don't have concrete use cases. In general, whenever you're unsure if it's too abstract or not, I suggest...

      I've toyed with modelling music in the past and it can quickly get overwhelming if you don't have concrete use cases. In general, whenever you're unsure if it's too abstract or not, I suggest asking yourself, 'too abstract for who and for what purpose?'

      Great way to learn music theory though! Check out the Tonnetz system for another interesting way to look at chordal relations.

      2 votes
      1. krg
        Link Parent
        Good advice! My basic end goal is to collect notes and find out how to display those notes as spellings of a chord. Just a toy project at the moment, designed to get me to think about how...

        Good advice! My basic end goal is to collect notes and find out how to display those notes as spellings of a chord. Just a toy project at the moment, designed to get me to think about how to..uh..design these things.

        2 votes
  4. [3]
    3d12
    Link
    I've started learning electronics recently, and I'm hoping someone can help with what I'm sure is a fundamental misunderstanding of mine that I'm for some reason having a real bear attempting to...

    I've started learning electronics recently, and I'm hoping someone can help with what I'm sure is a fundamental misunderstanding of mine that I'm for some reason having a real bear attempting to google around to figure out.

    So, I've learned about Ohm's Law, and Kirchhoff's Voltage Law for closed DC circuits. All the practice problems work out real nicely, where all the resistors used in series drop exactly the amount of voltages required so the energy potential at the negative terminal of the battery ends up at 0v.

    In reality though, for a novice like me, if I were to miscalculate the resistance needed for a circuit, logic tells me that it wouldn't simply "evaporate" and end up with a final potential of 0v anyway. Energy converts to light, sound, and most commonly in electronics I think, heat. So I expect for example, if I had a circuit with only one 10 Ohm resistor in series from the positive terminal of a 9v battery to the negative terminal, this would generate more "wasted" energy to heat as opposed to, say, a 10,000 Ohm resistor right? So is there a way to determine that conversion of "wasted" energy?

    As I was trying to get my numbers right for the preceding paragraph I realized that the 10 Ohm resistor in that example does provide a -9v drop. Is that because the current being drawn is so comparatively high due to the low resistance? (0.9A)

    Taking this example to its extreme, I surmise that connecting a wire of negligible resistance (compared to a resistor) and connecting it directly from the positive terminal to the negative terminal of the aforementioned 9v battery would mean that wire would simply heat until the material tension of the wire couldn't conduct the provided current anymore. So is there an equation or theory that models this heat transference from "wasted" energy?

    Thanks in advance for any assistance, and sorry if this is a really rookie question. I'm trying to make sure I understand this all inside and out, as my end goal is eventually to work on an amplifier I have. (But I have a long way to go before I meet 120V AC!)

    4 votes
    1. [2]
      tindall
      Link Parent
      Ohm's Law is that equation! Well, that plus the formula for power = voltage * current. You connect a 9V source to ground with a 1,000 ohm resistor. I = V/R tells us that the current drawn is 0.009...

      Ohm's Law is that equation! Well, that plus the formula for power = voltage * current.

      You connect a 9V source to ground with a 1,000 ohm resistor. I = V/R tells us that the current drawn is 0.009 amps. Since there's just one resistor in the circuit, it's dropping the full 9 volts, so P = VI tells us that the power is 0.081 watts - basically nothing.

      Resistors are rated for a specific maximum wattage. The common THT resistors you see on a lot of hobby projects are 1/4W (0.25W). Above that, they can't dissipate enough waste power through just convective and radiative heating, and will burn unless externally cooled.

      2 votes
      1. 3d12
        Link Parent
        Thank you! That P = VI is the equation I was missing, I think. I'll have a look around. It seems I have a lot more to learn about how resistors work, too! :) Thanks again.

        Thank you! That P = VI is the equation I was missing, I think. I'll have a look around. It seems I have a lot more to learn about how resistors work, too! :) Thanks again.

        2 votes
  5. thistle
    Link
    Work continues on my Python-only soundfont synthesizer, Wiske. It's unbelievable how long I've spent trying to wrap my head around decibels and attenuation (not to mention the tricky model for...

    Work continues on my Python-only soundfont synthesizer, Wiske.

    It's unbelievable how long I've spent trying to wrap my head around decibels and attenuation (not to mention the tricky model for 'generators' and 'modulators' which the sf2 spec has).

    No matter what I tried, I was having problems with notes getting too quiet based on their velocity. A note at MIDI 127 was loud enough, but 100 was barely audible.

    Problem 1: the modulator setup was fucked, causing there to be two modulators having the same effect, effectively squaring the attenuation drop-off. After fixing this, there was still too much of a drop-off.

    Problem 2: turns out, all the little details matter. In the soundfont spec, it is specified that a drop-off in volume due to velocity can be linear, convex, or concave. I only implemented linear initially, thinking that it wouldn't be too far from concave - the default method of drop-off for volume.

    How wrong I was.

    Concave mappings use a logarithmic function, as specified in the spec. Decibels are also logarithmic. Suddenly, after wrangling some logarithmic functions into the right place, the volume drop-off became almost entirely linear!

    Moral of the story? Even the things you might see as unimportant or superficial have a purpose.

    3 votes
  6. Apos
    Link
    I've been working on a music composition software I call Texo. My goal is to make something that I can use to quickly try new music ideas, have an infinite canvas where I can input different...

    I've been working on a music composition software I call Texo.

    My goal is to make something that I can use to quickly try new music ideas, have an infinite canvas where I can input different melodies, try variations side by side, etc. I'm testing the idea where instead of having a piano roll with absolute notes, I can start groups of notes anywhere and have them be relative to any key.

    As a mid term goal, it would be nice to be able to generate the music sheets. (Using LilyPond, it shouldn't be too much of a challenge.)

    Long term goal, it could be cool to make an API where I can generate music programmatically.

    3 votes
  7. mxuribe
    Link
    In order to learn more about the [matrix APIs] (https://matrix.org/docs/spec/), i've been playing around with an alerting system, whereby alerts are packaged up (from a local system), and sent up...

    In order to learn more about the [matrix APIs] (https://matrix.org/docs/spec/), i've been playing around with an alerting system, whereby alerts are packaged up (from a local system), and sent up to a matrix room which i've designated as receiving all of my system alerts. THen, i simply receive the alerts per my mobile matrix client's notifications, etc. Clearly there are plenty of other established systems that exist using other protocols and platforms...but i like that all my system alerts can be centralized into a single public or private matrix room (or alternatively sent to different, numerous matrix rooms - one for each type of system alert, etc.). This little python project is by no means sophisticated. But just for me to leran. I'm calling it alertrix; a combo of the words alert and matrix...but also because in my mind it also is a bag of tricks for alerting, n other words alert tricks...but adjusted the spelling to fit matrix. And, no, it has nothing to do with a very alert dominatrix, which is what i've heard someone else note about the name, which yes, yes, i'll admit is cheesy. But that's the working name of my project, and again, something that i'm just toying around with.

    3 votes
  8. acdw
    Link
    I've been working on my emacs config .... like, a lot.

    I've been working on my emacs config .... like, a lot.

    2 votes
  9. SkewedSideburn
    Link
    For reasons unknown I've made a Telegram bot that makes Heroes of Might & Magic III memes about astrologers, the ones with the window and a "Astrologers announced a week of..." Works with English...

    For reasons unknown I've made a Telegram bot that makes Heroes of Might & Magic III memes about astrologers, the ones with the window and a "Astrologers announced a week of..."

    Works with English and Russian messages (the game used Times New Roman I think, so I took the Russian version of that as a font), in inline mode (though it's severly limis max message length by Telegram API) and in chats (full length messages are supported there)

    Here it is: https://t.me/astrologers_bot

    2 votes
  10. DaveJarvis
    Link
    Finished working on a fork of JMathTeX, which can parse and render into SVG 1,000 simple plain TeX math formula in ~500 milliseconds on modern hardware. Will be integrating with my text editor...

    Finished working on a fork of JMathTeX, which can parse and render into SVG 1,000 simple plain TeX math formula in ~500 milliseconds on modern hardware. Will be integrating with my text editor very soon.

    1 vote
  11. Emerald_Knight
    Link
    I haven't really dug into code for it yet, but I've been spending a lot of time thinking over the sources of friction in the code I've written and have been planning on building out a revamped...

    I haven't really dug into code for it yet, but I've been spending a lot of time thinking over the sources of friction in the code I've written and have been planning on building out a revamped version of a backend framework that I use for my work (I won't be mentioning what it is because it's obscure enough that simply knowing I use it would probably be enough to identify me). I plan on doing a full architectural overhaul to fix some of its underlying design problems and make it do what it was actually designed to do but currently does poorly, and then I plan on adding some wishlist items that I've been wanting for a while now but haven't had the time to get around to adding into production code.

    It could turn out really well, or it could turn out to be a complete disaster. Either way it'll be a fun exercise.

    1 vote