16
votes
What programming/technical projects have you been working on?
This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?
I just finished up my Quick Server Panels project. It lets me configure a (mobile-friendly) button-panel UI. Each button can be configured to either make an arbitrary HTTP(S) call or run a CLI command (with arguments provided by browser form) on my home server, optionally displaying/streaming the output back to the browser.
It's great for when I'm away from a computer and want to run some simple commands with my phone without having to SSH in and type precise commands on a phone keyboard.
I also use it as a universal remote - I have several HTTP-based IR emitters, and with this I can make a mobile-friendly panel of buttons whose HTTP calls tell the devices to emit IR signals for things like TV-on, volume-up, etc.
The project took way longer than I expected, because I consistently choose the harder/more "future-proof" option when given a choice (to the point of irrationality). For instance, the server has lots of safeguards for limiting which commands (and even what set of arguments) can be run. That choice, while "good" design, doesn't make a lot of sense given the threat model (the server is not meant to be internet-exposed, and operates on a trusted network by a trusted client). But I find it's more fun to build things to be needlessly extensible, and this was a hobby project after all - fun is the point!
Some other highlights:
README.md
https://example.com/${route}
.ffmpeg
jobs.CURL
button lets me type in a URL to download with a simple HTML input form.Hey, that’s cool! It’s almost like a more universal version of Apple Shortcuts, which (among other things) also let you do fun web and/or ssh things.
Yep! My target use case was on my iPhone/iPad, and I have a lot of shortcuts experience so I strongly considered shortcuts to solve this problem. I eventually built this instead because:
tmux
orzellij
to achieve this, but it might not be seamless.Plus, like you said, it’s nice that it’s more universally accessible if I’m on a windows/linux/android box!
The past couple days the "UTF-8 fairy" bit me and after going to the text encoding hospital I decided I was going to spend more time learning about segmentation.
Enter Unicode Standard Annex #29. Ah yes. Number. Twenty. Nine. I need not repeat this sequence of syllables... but I will paste some links:
Some ramblings:
🤦🏻♂️ <-- this emoji is 17 bytes long but it is only 2 spaces wide
⸻ this is the triple em-dash. Fear no longer that AGI has stolen your sword. It is only three bytes long. It's pretty long in the browser but only prints one-space wide in the terminal. kinda lame
So from these examples we can see that doing
len(text.encode("utf-8"))
is not enough if you want to determine if something will fit on the screen.In the terminal, emojis and certain languages have letters that show up as more than one space. In Python you can use
wcwidth.wcswidth
to calculate how much space something will take up. That's great.(Although now that I know more I wonder if there are edge cases like ﷽. That's a single character. It shows up really long in my desktop browser, but quite small on my phone's mobile browser, and even smaller in the Terminal--wcswidth says it is one-width wide and indeed it prints one-width in the terminal)
But if you want to ensure that a string will not print longer than one line in the terminal, your function needs to be aware of the graphemes to decide where to truncate the text. Otherwise your US flag 🇺🇸 can turn into 🇺, etc. This is because a lot of emoji and other text is built from other emoji/characters. This is not only emoji but some languages like Korean are particularly dependent on this behavior.
For text segmentation you really need to use the raw data of the UTF-8 spec to make intelligent decisions about where to cut text... but I was hesitant to add a dependency with 9 GitHub stars so I made an approximation.
While writing this up I also came across the icu library again... a familiar name but I didn't think to check that it could work here. It is an optional dependency of
natsort
so I'm already kind of using it... Maybe someday I'll switch to using that but for now my grapheme_iter seems to work well enough!While there was a dire need for “emoji standardization” back around ≈10 or so years ago, today, I’m not sure if throwing them all into the Unicode body forever was a good idea. Maybe as a sort of mutable “appendix”, perhaps? I could see that. In principle though, Unicode should standardize the base “scripts” of the world, and I’m not sure if emojis should count for that.*
But now, we’re stuck with a bajillion different ones, and their number will only ever go up, never down. What happens if we decide the star trek greeting isn’t culturally relevant anymore in a couple hundred years? What about the skin tone variants? The one with dollar signs in its eyes if the dollar were to disappear? And so on – guess what, computers and applications and programmers everywhere will have to take the hit and support those fun quirks, except for the unlikely event of abandoning Unicode support/standard conformance altogether.
*And that’s me arguing from the standpoint of someone who isn’t even necessarily against the emoji “premise” which, if you look at it realistically, is a regression for human communication (back to pictograms over consonants and vowels)…
The meaning of some emoji will change over time. Some emoji like this one 🔣 already don't seem to have a clear use case. Oh, what is 〒 you ask? It's the Japanese postal mark of course! It's a strange symbol to choose for representing symbols because I don't think many people recognize it so they don't really know what 🔣 means.
The floppy 💾 is already becoming disconnected from its physical origin. Most young people can easily recognize it as the save icon but they have no idea what it is. Still, it's more recognizable than if developers started using File Cabinet 🗄️, Folder 📂, CD 💽, or Hard Drive 🖴 icons. Did that last one not show up? If people become too upset with an emoji the fonts people could just boycott it... Although that's probably not what happened here.
It's good that there are a lot of emoji. As long as people keep using the zero-width joiner character there isn't a lot of extra processing that the computer has to do to support more emoji combinations. It just means that font packages will get larger in size. NotoColorEmoji is about 10MB. Pretty big but it can easily fit in memory.
Forgotten emoji will still have a chance at being rediscovered and reinterpreted.
Also, certain patterns of emoji are memetic and will go through periods of use and disuse: 👁️👄👁️
I've been bashing together a script that uses a number of tools to accomplish a task:
It's still very much a work in progress, but I've separately gotten all the steps to work, now I just need to combine it into one fluid step. Unfortunately there's no way for me to automate the audio coming from Craig to my Debian host, that I can see anyways, so that will have to be manual. Otherwise I can fully automate weekly AI powered recaps.
I just got a delivery notification for a bunch of electronic components. I'm going to attempt to build my first oscillator circuits so I can route them into my effect loop chain. I was a noisician years ago and have wanted to get back into it. I realized lately that I want to get more into the DIY aspect, so this should be a good starting point.
I've been machine-translating all the online help for my app into other languages. I'm up to 8 languages so far.
https://akkartik.itch.io/carousel
I'm working on a hobby computer game with trains, so I spent some time the last week to procedurally generate railway with crossties from splines so I don't have to build them in blender and export for every single configuration. It works pretty decent already and looks good, but there's still some cleanup to do. Getting it set up has been simple and fun - Godot keeps delivering :).
I finished the synthesizer portion of the software groovebox I described here. The hardest part was keeping the number of controls limited without sacrificing too much flexibility.
The sound turned out quite TB303-like but with a very distinct quality because it's all faked using phase distortion synthesis. I made the "filter" slider continuously morph from a sawtooth to a square via a sine instead of using a switch to change waveforms. The "resonance" control just mixes in a sine wave at the guesstimated resonant frequency, which is synchronized with the oscillator that provides the body of the sound and also fades to zero over the duration of a single cycle. This is probably illustrated more clearly in figures 18 and 19 of Casio's patent. One notable "improvement" on the patent is that I introduce feedback modulation of the phase, like in Yamaha's FM synths. That becomes a really nasty fuzz/distortion effect in this design.
For now it sits in my music prototyping/hacks repo is a JACK application that accepts MIDI notes and pitch bend, with simple on-screen sliders for the timbre controls. I will use MIDI messages internally for sequencer control once I build that, so it should be plug-and-play. If anyone wants to give it a go and are comfortable with JACK, the source is available here. I'm using Zig, but I haven't updated that in a while so I only know it builds with version 0.14.0-dev.2605+136c5a916.
I’m currently working on a local-first, bookmarking, archive, catch all, thing. It’s an Electron app + a browser extension designed to be a single, reliable solution for saving anything I find online, since I work across a few different computers and operating systems.
The extension automatically detects what you're saving and scrapes accordingly. It can grab any image, products with their price and images, save articles in a clean reader format to read later in the app, and capture entire webpages as full-page screenshots. Everything gets saved directly to a folder on my own machine as standard markdown and image files. This means all the data is fully accessible and editable from outside of the app, no proprietary lock-in. It also means you can add notes and local files to use in the app by simply moving it to that folder. I’ve quite a lot more features planned but really want to nail the frictionless capture side of things first.
Which leads me to the main challenge I'm wrestling with now: the path to mobile access and sync. Currently I see a few options:
The purist approach is to remain fully local, which would mean building a native mobile app and handling sync with CRDT (ideal) or third party cloud services (easier). However, as someone with no mobile dev experience I worry this could add a chunk of time to the project, perhaps worth it though. This has led me to consider:
a companion web app, which would sync all the local
files to its database periodically and be far more convenient to build and access. However this would perhaps undermine the whole "local-first" philosophy. Finally:
A potential middle ground is to offer a self-hosted option, which aligns with the user-sovereignty ethos while achieving the convenience of a webapp. I’m big into self hosting so this is an enticing option, but I would like to make it publicly available at some point and worry this would introduce a higher technical barrier for entry.
Curious to hear if others have faced any similar quandaries in their own projects.
I would say SQLite is probably a good solution here for the mobile side. CRDTs are no panacea... ie. you can still do conflict resolution in advance without using a CRDT data type
That's a good point. Since real-time collaboration isn't a necessary feature CRDTs might be overkill. Really, fast and reliable sync is the main goal, and given the use case custom conflict resolution should be fairly straightforward
You can use Electron or Tauri to build apps for mobile with web tech as well. Compiling them for mobile (especially IOS) is a bit of a hassle but doable.
Yeah I had looked into this briefly but have heard competing opinions on the practicality of it. My understanding is that it would be a bit of a square peg, round hole situation (though I guess you could say that about Electron as a concept anyway!)
You mentioned Tauri, do you have experience developing with it at all? I had considered it for this project for of the smaller footprint and the chance to finally learn some Rust, but Electron's consistent cross-platform experience ultimately felt like a safer bet
If you are using react, another option is react native. It actually renders to the native UI toolkit, unlike electron and flutter. It is just as much a native app as one written in swift or kotlin. The only difference is the main code uses JS instead of native code.
I don't have experience with it personally, but I have a friend who is developing an app with it, and they're very happy with it. The cross-platform support is probably as good as electron now (maybe a bit less mature bit I'm not sure), so that's not a problem.
I recently bought some 520 byte block size drives (HITACHI HUS72604 NEO4000 NAM4). They show up as state "Ready" in my r730xd iDRAC but they don't show up in
sg_scan -i
... I'd love to use them for cold storage but it looks like I'll need an alternative HBA to be able to reformat them. Kinda annoying.I guess I could try to switch my H730 Mini to RAID mode temporarily and see if I could somehow reformat them... I wonder if that will work... If not, what would adding another HBA involve? Does it have to replace the H730 Mini? Can it work together with the H730 Mini? Those are the questions on my mind this evening.
maybe I'll just buy a SAS docking station or something... kinda expensive tho... actually I have another PC with a LSI 9207-8i SAS2.1 HBA which I've been using for only SATA drives with a Mini SAS to SATA breakout cable. I guess the only thing I'm missing is an actual Mini SAS+Molex (or SATA Power) to SAS cable.
I've been working on a shape library using SDF math. (Essentially math that point towards the edge of the shape, allowing you to know when a pixel is inside a shape, a certain distance from the border, etc.)
I finally added gradient fills which was a bit of a rabbit hole.
https://github.com/Apostolique/Apos.Shapes
This allows creating games without any sprites, or it's really useful for creating debug tools or various visualizations.
Had a crazy idea for my research and decided to go for it, and learning Julia along the way!
I have this movie calendar thing that I never shut up about, it seems. Anyway, I can improve this, but here's one quick workflow that is handy
ezpz -- works really well, too.
I've been having an issue with my Linux install on an SSD connected to a hotswap dock in my case rarely failing, so I wanted to relocate my Linux install to an internal NVME drive (new mobo has two slots).
I also decided that, since I'm reinstalling, I should just switch distros. I've loved Aurora, but small fiddly things I like/need are extra work on Fedora Atomic. I enjoyed the stability and seamless updates, but decided Debian's the way to go since I have two major workflows now:
So Debian made sense. It won't change too much over time, will have stable support for up to five years, and updates every two. I won't suffer Shiny New Stuff Syndrome since Plasma 6.3 is a solid version, and anything I need I can get via flatpak, brew, or distrobox. I just got tired of not being in full control of system components (UBlue made some changes that weren't bad, but I didn't want), and dealing with OS abstraction via bootc/ostree.
Released a new version of KeenWrite, a desktop text editor that converts Markdown files into PDF files using ConTeXt. See the user manual for sample output.