13
votes
What programming/technical projects have you been working on?
This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?
I noticed that the data structures I used to define weapons in my twin stick shooter game resembled a procedural language and have been thinking for weeks that I'd make a scripting language for them some day so that I can tweak them without relatively slow recompilations. The other day I realized that I could just reuse the language and VM I've already implemented for defining attack waves for this purpose. My game now runs two instances of the same VM type, one for the wave progression and one for the weapons.
The language is very basic: parameterless subroutines, a single branching construct (
repeat <n>),sleep <n>to yield to the caller and pause execution until the given time has passed and a bunch of hard coded keywords that are used to affect the game state like spawning bullets or enemies, set the spawn area, set the current weapon, trigger sound effects or display messages/wave vignettes. Nearly every token in the language corresponds directly to an instruction in the VM, aside from braces which mean slightly different things depending on whether they're used for a loop or a function.Weapons in this system are plain subroutines (though with an alias for the subroutine keyword for organizational purposes). The player data structure has a pointer to the subroutine that corresponds to their current weapon, and for as long as they hold fire and no subroutine is already running, the VM executes the subroutine to completion.
For example, My definition of a shotgun looks like this:
This plays the shotgun firing sound effect, fires 12 "redbullet" projectiles at an angle offset of -0.15 to 0.15 and at a speed of 350 to 450 pixels per second, sleeps for 0.3 seconds before playing the shotgun cocking sound effect and then another 0.4 seconds before the player can fire again.
The effect on code was nice. It removed a whole bunch of complexity by getting rid of the previous weapon system altogether, and the compiler and VM changes were pretty small, so overall a net negative LOC change in the engine code, while also leaving weapon definitions much shorter because the previous weapon system lacked a looping construct. I anticipate that this will only get better with more complex weapons that could benefit from being able to call subroutines. The result is also much more flexible, because I don't strictly need to use the keywords I added just for weapons in a weapon definition. The weapon definitions now can cause any of the game state effects the wave definitions can.
I am curious how you handle continuations. Do you save the managed stack along the instruction pointer or is loop nesting forbidden? Or do you do something fancy like CPS transformation?
Also, what's your host language?
Nothing clever. The VM instance has a program counter and a 32-deep stack that is used both for return addresses and (loop address, loop counter) tuples. So in terms of VM instructions something like this
ends up being (address numbered)
So I can interrupt this process at any point so long as I keep the stack and program counter intact. There is some other state maintained in the VM which I keep as well. In practice, I only interrupt the VM when it sleeps. In pseudo code:
It's written in Zig, which I thoroughly enjoy. It's perfect for what I've previously used C for yet avoids many of the footguns, without having too much of a learning curve. Its seamless compile time execution with type parameters and introspection is a great solution to meta programming. I actually started out with C but a couple of weeks into the project when I had memory management and graphics engine in place I switched to Zig, which I never regret. Even something as simple as enumerated unions have saved me so much boiler plate.
Is that applied with uniform distribution, or triangular (
random()-random()) or somesuch?It's uniform!
Then I highly recommend trying triangular, even if you keep velocity uniform. I find it looks so much better. https://i.imgur.com/juJqTJ5.png
Everything is gaussian!
I've made a very simple blog for fun. I'd link to the post where I wrote about how I did things, but the site is still... A bit scuffed in terms of features, and I haven't actually made posts have their own pages yet. But there's only 2 for now, so just scrolling is still an option. I am hoping to find time to implement some things soon to make it a bit more functional
I'm impressed you've got content! I've been working on my own blog, and it was fairly late in the process that I realized that, no matter how smooth my build pipeline was, I still had to, like, write every single post.
kudos! It looks good.
How did you put it together? I've kind of found myself bother under- and overwhelmed by choices.
I've been trying to replace the ReMarkable e-ink tablet's stack with my own, open stack - there are plenty of options to run a desktop on an e-ink tablet, but if you want to actually draw on it then you're generally stuck with Xournal++, which seems to have more buttons than LibreOffice for some reason, basically none of which are named. So I installed Parabola-RM on my RM tablet, and I've written a stylus-drawing app in PyQt using QtQuick.
Problem is, QtQuick doesn't support stylus input - tablet events (which are nessecary for accessing pressure and tilt) aren't passed through, due to some several-year-old bug that's only just now getting merged for Qt6.6. On the plus side, the one advantage of Parabola is that the packages should be updated fairly quickly, once 6.6 is released.
I still have stylus-as-mouse input, so it's not that big of a deal and I should probably focus on other stuff like making it take less than 10 seconds to render on the tablet. I suspect that's due to it being GPU-accelerated on a device that doesn't have a GPU (it renders normally on my laptop), but there's probably some O(N^2) stuff so I should probably profile it anyway. I was sort of hoping to run it on the PineNote, which does have a GPU, but I wrote this app ("app", lol) a year and a half ago and the PineNote still isn't ready for the general public.
I have some vague plans for how this thing will be better than the ReMarkable's default stack, but they're all really far away. Like, the RM forces you to tap out names for new notebooks on a virtual keyboard, when there could just be a lasso tool to select the title to be OCR'd/HWR'd. This requires a functioning HWR library, which doesn't exist in the open-source world. I could hack up an OCR lib like Tesseract, but they're both less-accurate and more processing-intensive, and the RM has an (IIRC) 800MHz ARM processor with no GPU.
HWR is different from normal OCR in that you pass in more than just raw bitmap data, which improves accuracy - for instance, if you wrote the letter "e" but the top two lines are overlapping, then the resulting bitmap might be basically impossible to distinguish from a bitmap of a "c" letter. HWR can solve this as its vector input gives you the stroke origin point which makes it basically trivial to detect - if the stroke started on the left then looped around it's an e, and if it started on the right and is one near-circular anticlockwise stroke then it's a c. This is a very fascinating topic that I hope someone else will tackle.
I also would like to have the device auto-sync and also integrate with my desktop when I want to take notes, but sadly Parabola does not endorse having functional WiFi, due to being FSF-approved. I could probably compile it with the WiFi, but 1) setting up Parabola-RM was a nightmare that took a month and still isn't quite working, and 2) the Parabola-RM manual recommends using the RM toolchain to compile the kernel, and I basically started this project so I wouldn't have to think about the RM company or their toolchain or OS.
To be frank, I believe first getting to where you want to be functionality-wise, and in later iterations trying to “de-RM” as much as possible by potentially using another compiler and finding out what is not working out of the box is still a sensible approach.
As a somewhat related question to you as the “expert” on this, since I’ve been considering getting a Remarkable e-ink myself lately: would you say your project to openify the tablet is more based in practical or philosophical reasons? I.e., is there missing functionality you would like to have, or be able to tinker with its features more; or is it rather a thing of principle to use open/custom software for you – which to me would be an equally valid, if not better! reason for doing what you’re doing, but as a potential customer, more of a relief to hear than the first “category.” If it can even be pinned down to either reason of course.
BTW, what exactly is the stock OS on the thing? Is it some smaller Linux or (I’m hoping not?) completely custom, in addition to probably proprietary?
A bit of both. Initially, I was writing a drawing app for the PineNote, not the RM. This was 1-2 years ago, and the PineNote still isn't available, so I'm using the RM1 as a stopgap. Given that the PineNote will run a linux desktop distro, Parabola-RM is the most practical way to set up a similar environment (or at least, I thought so a month ago and sunk costs into it until it genuinely is now).
Choosing not to rely on RM mods is more about principles, although I see [not relying on proprietary stacks] as sort of a practical thing because I'm too small for RM Corp to care if they crush my setup by accident in any given update. Also, the RM2 is not repairable and clearly doesn't align with my values anyway, so I expect the PineNote to be my preferred option in the long term.
I want/need to tinker with the device until it suits my workflow - if I didn't, then I'd drop the issue to the bottom of my priority list for now, and just use the stock OS while I get the rest of my life in order - but frankly, proprietary gonna proprietary no e-note company is willing to open up their closed garden to let users build their own workflow properly; they want consumers dependent on their proprietary feature drip-feed. I don't fit in that pigeonhole so the pragmatic choice is the principled choice here.
I think there's a huge untapped potential for e-notes, both as a standalone device and as a desktop-extension device for writing notes and annotating on - most of the notes I write are about something, and usually that something is on my desktop/laptop. In the long term, I want to integrate my e-note into my WM and IDE and project system and todo-list system, and in the long-term HWR is obviously a first-class input system (literally, this thing is a writing tablet, how is writing not a first class input method for at least the title/tagging??). I expect RM Corp to satisfactorally achieve none of these, and I expect RM Corp to support my hypothetical future custom WM never. I don't even blame them, although again: proprietary gonna proprietary.
E-notes will become even more important if/when Gallery 3-style color tech matures and becomes practical enough to be ubiquitous, although that's well out of scope, right now.
I'm kind of horrified you'd call me a ReMarkable expert, by the way. I appreciate the quotes around the word - I let DavisR's RCU do the heavy lifting and fed it precompiled binaries to flash onto my RM.
The stock OS is a fairly normal linux distro, and they publish a fair bit of their source code AFAICT, but the RM2 is impossible to update the screen on without their proprietary binary (the binary also contains a GUI, so people erroneously claim that RM Corp "open-sourced everything except the GUI") and I suspect their open-source is a pragmatic thing that they only accidentally supported on the RM1.
I have successfully gotten ES8312 to work and sample an infrared phototransistor. Since it has 30 dB PGA, 24 bit depth and 100 kHz sample rate, the results are quite encouraging.
Next I am having some infrared beacons made. A set of four placed outside corners of a screen will hopefully do the trick.
The goal is to estimate where on a screen surrounded by the beacons is the phototransistor pointing to.
I will be using Goertzel algorithm to extract amplitudes of four different frequencies from the ES8312 stream. One for every corner. It works quite nicely in my small testing setup with 2 LEDs.
I am a bit worried about the math. Worst case it will work only when I am facing the screen. Best case I will be able to find a set of equations to solve in order for this to work from an arbitrary angle.
Some might ask why not use a camera. Well, this is way more fun. Also a lot cheaper. And definitely better (to me) than painting a weird white rectangle around the screen and tracking that.
I have already promised my wife a game of House Of The Dead once I finish it. :-)
PS: If you'd like to collaborate on this project, just let me know. If you chip in, I'll send you portion of the dev boards.
I'm currently working on the concept of "creating a hardware product" except packaging and financials.
So basically:
I'm a software guy and I have very limited experience with hardware of any kind.
I have worked with Arduino, ESPs, Teensy and alike before and I know generic electrical and microcontroller concepts, but overall it's pretty limited so I wanted to learn something new.
For now I chose a simple project: a fan and water pump controller that is configurable via USB + software.
I definitely want to make it open source, so the schematic/3D model/PCB design/code/usb protocol is intended to be open.
It must be simple enough for beginners to recreate, and cheap enough to be affordable, even for school kids.
That's why I'm trying to rather use THT components instead of SMDs to make it easier to solder, and use a Raspberry Pico because it has a lot of hardware PWM pins, is fast, is super cheap and allows to register itself as a dedicated HID device which I can use to allow its configuration.
The controller must allow the attachment of external thermistors that you can use as a target for a fan curve, additionally you should be able to simply set a static percentage value.
A dedicated lightweight software should be able to fully control and update it.
Honestly, understanding the USB hardware descriptor documentation and handling of the reports was by far the most difficult part of it yet.
Initially I tried micropython but was not very fond of it. Then I tried TinyGo but I was simply unable to make the USB connection work properly.
Now I just use C++ and deal with it. I use the arduino-pico library instead of the official pico-arm lib, because it's more widely known.
The HID connection now works fine and I can send data back and forth! :)
Doing the PWM part of the fan and water cooling pump was the easiest and that works very well already.
For the serialization I have used protobuf for now, although I'm currently thinking about switching back to manual byte encoding as it brings another dependency into it for a rather simple use case.
[a] Several weeks ago I built my first NAS server. It uses a pair of 2-GB drives set up as RAID 1. I was floored to see transfer rates as high as 92 MB/second from one of the Linux clients to the
Linux NAS server!
[b] Once I completed the NAS server, I needed a way to back it up, as it already holds 35 GB and I'm nowhere near having it completely mirror the contents of the three Linux clients. I'll be backing up the NAS server to one of four USB-connected 2 GB drives hosting an ext4 file system. The next step is to create a schedule in which an external drive is the target of a weekly NAS server backup, then once a month move the most recent backup to a secure off-site location.
[c] Earlier this year I bought a tool to crimp connectors onto cat 5/6 cable. It works well but it's painful to keep eight tiny wires in place in the connector prior to using the crimping tool; the
wires want to wiggle and squirm and reform into the original wire twist.
https://tildes.net/~life.home_improvement/19bg/advice_on_setting_up_home_ethernet_with_unused_cable_already_in_the_walls introduced me to "pass through connectors" which turn connector chores into a piece of cake! When I bought the pass through connectors, I also bought a cable jacket stripper and a cable tester. Both are useful and inexpensive tools.
You probably meant 2-TB.
Anyway, it's nice to see another person into self-hosting. I am personally running a 1U server in a small (almost community) colocation space to have in available from anywhere. The combination of slow HDDs with a LVM cache using NVMe is incredible.
Yes, 2-TB drives in the RAID pair and four 2-TB USB backup drives. Thanks for the good catch!
So I have been wondering...
A couple years back an acquaintance of mine needed a backup solution. We decided on USB drives but with a twist. The backup job would run nightly, but they would unplug the backup drive every week and swap it with a second one.
That way when the lightning hits, they would have at most a week old backup still available.
Do you do something similar?
I have an Open Office spreadsheet showing backup dates vs. USB serial numbers. Every Monday I attach the appropriate USB disk, back up the NAS server, and unmount and detach the USB disk. This keeps the disk away from lightning or other power-line disturbances and I don't have to worry about bumping or knocking over the disk while it's powered up.
That's pretty involved. Most probably wouldn't have the discipline.
I posted a few months ago about a visualization program for music I was working on.
Well it turns out the FFT code I used was returning absolute garbage so I need to replace it.
It's my fault that I relied on ChatGPT for it because I really struggle to understand the math behind Fourier transforms.
I resumed reading "The Scientist and Engineer's Guide to Digital Signal Processing" by Steven W. Smith, in the hopes that it will walk me through all the math and make me understand.
Besides that, I feel the urge to make a Pokemon Red clone. I started playing it and I'm 3 badges in, and for some reason it hit me that making a similar game would tickle all three of my interests - programming, pixel art and electronic music.
I do want to avoid starting new projects when I have unfinished one in the works though, but maybe this time I can manage 2 projects in tandem? One is basically stuck in math limbo, so while I read through that book, I could draw some fakemons and animate them at least? It could give me a chance to try out Bevy, it's a Rust game engine I've been eyeing for some time.
For FFT, I recommend FFTW3. Very fast and easy to compile. I haven't used it myself, a couple of scientific codes I use use it as a library.
Since I'm using Python I've been using scipy which is trusted and reliable, I just don't know how to correctly use it, haha. FFTW3 is awesome for C/C++ from what I've heard (there are probably bindings for Python too though).
i am currently building a custom wordpress theme for a pet project i will eventually use in my portfolio.
i have occasionally checked a website about a mildly obscure musician since at least 2004 and the website is still be run as if it were 1998 (vanilla html (4?) and updated by hand). i’m reworking it into a modern wordpress website with all of the existing content. once i finish, i'm going to then create an instructional video on how to use and maintain the website.
i'm going to offer both to the website owner if they want it, for free. i’ve gotten so much joy from their website, it feels like the least i could do. even if they don’t want it, it is nice to have a project that’s not just dummy text or something built from tutorial examples.
As someone who maintains a static HTML site, please be prepared to be turned down. The zero maintenance costs of static sites are difficult to beat, especially for a hobby site. No database, no software/plugin upgrade headaches -- just open a text file, edit, save. WordPress maintenance would turn the hobby into actual work, personally
This is a great point that I did not think about, thanks!
After a summer largely taken off from working on the project, I'm back to working on komorebi and whkd.
This week I started working on making both the window manager and the hotkey daemon more portable by allowing users to specify configuration file locations using flags (so that everything can be on a USB drive instead of in the system config paths), and I'm going to carry on integrating this so that AutoHotKey users can also do the same thing for their window manager key bindings as well.
After Google Music fell over for YouTube Music, Google gave me a small window to download all my songs I have purchased with them. However they didn't come with proper metadata, just the files with the song title.
After having tinkered with Python over the past month, I put together a small script that goes through a folder of .mp3 files, search the title of the song and pull the artist and album information and write it as metadata.
Spotify API has a limit of 100 requests every 30 seconds and Genius API only accounts for artist and lyrics.
After messing around with those, I discovered YouTube Music API (which should've been where I started first) was able to pull from their search results.
After running this through couple of songs, I noticed some songs had incorrect metadata. I had not accounted for other songs with the same name as the song title I had searched.
I do not have a solution for this problem, I may just let it run and fix any anomalies manually.
Ignore me if part of your motivation is to practice and improve your skills.
Programs like this are relatively popular for people who manage their own music collections, so you might be able to shortcut this and use an existing solution. Like https://github.com/beetbox/beets
welp, guess a microblog here may help give me some motivation:
I'm working on a "small" rendereder project in Vulkan. my main milestone for it will be to more or less be able to port some animation samples from this mini-series, and profile my implementation hoping for better performance. Main divergance (outside of OpenGL vs. Vulkan) is that I'll load the model via GLTF instead of as an md5mesh. It's a project that knocks out many birds at once so I want to try it out while I still have time.
In theory, this should be something I crack out in a few weeks (as I've made renderers and worked in the graphics pipeline under OpenGL), but as mentioned above, motivation can be a bit low. So as is I last touched my code 2 months ago and was maybe 70% of the way to a triangle. I want to at least show off a triangle by the next post,.
I'm continuing to work on this cross-platform app for programming LÖVE apps on both keyboard and touchscreen devices. In the last week I improved its graphics isolation (keeping programs you write from interfering with the IDE; it's all happening in a single app to keep iOS happy). And a menu option to manually rotate the device (because LÖVE doesn't seem to respond to rotate events on Android). And scrollbars. Man, scrollbars are hard.
I've also been building fun/silly little apps for my kids on it. Like this.
Nothing too major but earlier this afternoon I dabbled a bit with high availability in Proxmox. I set up three proxmox VM on my proxmox host, set up a cluster with these three VMs then spun up a bunch of VMs on those virtual hosts and it all just worked.
(1) RSS Reader: now that I am queuing up social media submission for real and developing some really long queues the mechanism I have for changing the order is inconvenient, right now I have buttons to bump an item up or down one step in the queue, I really need buttons that move things up or down one place and the quick way to improve things is to add "top" and "bottom" as well as "-5" and "+5"; I'm still making little changes like that here and there as opposed to doing any major development.
(2) Three-sided cards. I've got an art project that I'm procrastinating on but I made some anime cards just to keep the printer from drying out, as a consequence I made the first anime "I card", that is "individual card" where each individual card in a series has its own unique QR code
https://mastodon.social/@UP8/111013706271196029
right now the system puts in redirects so all the cards of the same design point to the same page but later I can add services that take advantage of this, one idea I have is something that's a bit of a parody of an NFT where somebody who has a card can register it to their email address.
(3) Blog. I picked out a theme for Pelican that I can live with, added a plugin for typesetting math, and did a lot of the DNS and cloud setup to publish the blog. Still gotta set up the CDN and finish the first blog post which is about the design rules for QR codes used in "I cards" (see (2)) to maximize the number of possible identifiers in a "version 2" QR code that is almost as simple as a QR code can be.
Work has a small automation division for me to mostly deal with nuisances with paperwork. Currently using UIPath to automate timesheets. Downside is instead of a straightforward spreadsheet, there's a lot of calls being made when you select a job title. I'll figure out a way to work around it; maybe lots of delays.