17
votes
What programming/technical projects have you been working on?
This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?
I have a small homelab that's constantly growing and evolving. Right now it's a FreeNAS server and a Proxmox VM host, but I have another potential VM host that I'm working on. FreeNAS is mainly to provide storage for various VMs via Samba or NFS - each VM gets a small slice of local storage on the Proxmox SSD, then a larger chunk of storage mounted on /home for applications to use. I discovered that I can't connect via http/https from outside my network despite my ISP leaving them open (or so they claim), so I'm planning on setting up a VPS to act as a reverse proxy.
On the programming side, I started a project to teach myself cryptography. It's the field that got me into computers in the first place, and I've done almost nothing with it for several years now. I'm just working my way through implementations of common hashes and encryption algorithms in Lua, which is adding its own unique challenges. I picked Lua because something about writing code with it just feels good, there's no logical reason behind my choice. Progress is slow and I'm still figuring out MD2 which is relatively simple, but the RFC has been the only guidance I've used so far.
I'm interested to hear about where you land for your reverse proxy setup. I've been trying to set one up myself and am somewhat paralyzed by the choices available with their various pros/cons.
I responded to the parent comment with the suggestion of cloudflare access. Is that something you've considered?
I've been avoiding anything proprietary as much as I can, so I didn't look too deeply into cloudflare offerings.
Here are the ones I was considering:
Nginx Proxy Manager: I like this option because I'm already familiar with using nginx in and out of containers. It does seem to be one of the older options, though, so I'm not sure if I'm limiting myself by sticking to outdated tech.
Caddy: This one seems super popular, so I think I'll get similar amounts of support for issues as with Nginx. But with that comes a lot of potential misinformation, and I'm entirely new to the ecosystem here (unlike with Nginx)
Traefik: This one feels like overkill for my small use case. But maybe I'm biased because it was used by the infrastructure team at my last corporate job... so perhaps it just feels too heavy because it was being used in a heavy environment there. I also have a set of services that are all over the place (in and out of containers, spread across devices, etc), and I've seen comments about how Traefik works best if you are using it in a more orderly and containerized environment.
So right now I'm thinking I might try both Caddy and Nginx and see which one sticks. But I'm still fairly overwhelmed and there isn't a super clear winner.
Traefik is nice for containers because it can utilize the docker (or podman) API so you can just add labels to your compose files to tell it what to do.
I also used caddy for a while and it's pretty good. I found having to refresh it after modifying the caddyfile to be a bit tedious, but that's my only complaint.
And as you said, nginx is tried and true. Nothing wrong with it, other than perhaps being a little heavier than caddy. (I'd review actual benchmarks if you're seriously performance tuning, though.)
If it's just yourself that you need the reverse proxy for, a VPN is another option.
I have a Wireguard VPS on ice in the cloud for a few cents a month. I spin it up for when I'm traveling to get access to my home network on the road, it's super convenient and cost efficient.
If you're ok with the proprietary nature of it, cloudflare access (previously called tunnels) is pretty sweet. I run it in a container and have it connected to a traefik container, which in turn is linked to my other containers that get dns names. (I can explain that part of it better if you're interested.)
There's other ways to expose services without open ports now as well, I've heard a lot of talk about tailscale but I've never used it personally.
Cool. I built a ProxMox with TrueNAS Core (formerly FreeNAS) for storage out of a 2U bought off Enay for $400. Such a steal considering a similar retail device would've cost 1k and only use 5 drives to my 2U's max of 13.
For context, I'm working on an expense tracking and splitting app. The plan is to be a 100% open-source, self-hosted, ad-free alternative to Splitwise. (Written entirely in Rust and Tailwind CSS, by the way.)
I realized Settlemint is a name already taken by some crypto thing, so I'll need a new name. Thank god I resisted the urge to work on branding or buying a domain.
Anyway, the app now supports separate expense groups, so right now you could theoretically use it as an expense tracker, but you'd have no statistics (those are coming, but later!).
I spent some work ensuring my database schema is normalized and coherent (I can finally say my college classes were useful for something, lmao) so I can easily query for, say, expenses made by a specific user in a specific group. I'm concerned about performance (Splitwise is really snappy, so I need to match or even beat that, which will be a challenge) so I expect I will need to flex the design a bit, but we'll see.
Now I want to let multiple people join a group and add expenses, then split those expenses between the group members. I'm not 100% sure on the best way to implement that, but we'll see how it goes.
I also made the UI not suck. It's not pretty, but it's not ugly either, it's quite literally the bare minimum with a little bit of polish. My partner has been pushing a lot for a UI redesign though, which I'm willing to do (provided he actually wants to spend a few days in Figma to design it), but that will happen only after the MVP is ready. Right now I want to have the functionality nailed down, so I can import my Splitwise data (oh god, I'll need to figure that out too) and start dogfooding ASAP.
What does that entail?
This sounds really interesting. Will you be integrating any banking provider APIs?
Database normalization is a set of steps and rules to make sure your tables don't have duplicate/redundant data, that each table only contains data relevant for the object/relation/relationship it models, and that they have a structure that is easy to work with.
I don't know anything about banking APIs, so no - it's not something I would be using, and AFAIK my bank doesn't have any public APIs to play around with anyway.
Thanks for the link and explanation.
I only asked about APIs as that seems to be something that's becoming popular with UK targetted tools at the moment, made much easier by Open Banking. But most of all, we have modern "challenger" banks, which are as much tech companies as they are finance, and provide excellent APIs for balance checkings and creating transactions, etc, that I could imagine being integrated effectively into a payment splitting application.
Yeah, unfortunately I don't follow fintech at all. Somehow I managed to not have a Revolut account despite it being very popular with European techbros.
All my finances are still with a very traditional bank, and the only reason I'm developing my app is that my partner and I are absolutely tired with Splitwise's ads, and believe their Pro plan is a travesty, pricing-wise.
We just want to split groceries and bills without constantly moving money from our accounts. With Splitwise we can see that I owe 200€ right now, so that means I'm the one paying for the next grocery trip (and the one after that).
What normal form did you go with for your model?
Third normal form, since that's what I was taught in college. I now know there are a handful more forms but I'm unsure if they're worth doing. We were told that the potential benefits are small, and that third normal form is adequate for most use cases. I'm still very much a database noob.
I wouldn't say the performance benefits are small per se, but they do only become evident at a scale unlikely with the application you describe. I find that it's usually worth decomposing to 5NF to eliminate the need for certain classes of constraints (multivalued dependencies); it's only 6NF that should be ignored as it lacks support in basically every RDBMS, and usually implies you want a columnar db anyway.
Hm, I can give 5NF a try and see how it looks. I'd need to document myself a little bit more to understand how to do it though, do you have any resources I could read?
I started (re)learning Rust. It's been fun, it's been difficult at times, but I like that. Originally I started out with the intention of using gpui to make an apple music client for linux, but after reviewing the API documentation this morning I realized it's most likely impossible. MusicKit is for Swift, and MusicKit.js is for web apps. I don't believe there's any way for me to integrate Apple's js library into a Rust application, if anyone has intel that suggests otherwise I'm all ears!
Have you looked into Tauri?
I hadn't, thanks for the link!
As mentioned, Tauri is something you should look into, but you can also use Dioxus for the UI and logic, they have a
use_eval()
hook which can run JS and return the result as needed.It'll probably be extremely hacky and ugly but it'll work.
rejiggering my 3d printer. Im moving all the electronics to an external enclosure, and adding cable chains. RN im modifying my electronics enclosure to fit my specific power supply and control boards, as well as learning how to use buck converters to power a RPI off of 24v. after that, the cable chains, and after that, I need to cut, strip, and crimp over 50 cables.
Crimping 50 cables?! Have you done enough cable work for it to not be nightmare anymore?
Is moving electronics to a separate enclosure a common mod for 3D printers? What's the advantage?
The reason for moving the electronics is that previous mods Ive done havent left enough room for the built in enclosure, so prior to this current round of mods, the electronics were sprawled out on a table behind the printer, with no enclosure.
Ive done enough crimping that its not hard anymore, but it will be tedious as hell, so plan is to cut all the wires, set it all out on a table in the living room, and put a show ive already watched on. My wife is also learning how to crimp things, so its not going to be that bad all things considered.
I read your other post on this system and found it interesting. Do you have any sort of background in that area, or just something you found interesting and started researching and trying?
I would be really intrigued if you had a long list of reward ideas in your app. Coming up with potential rewards feels like a chore in itself that I'd need a reward for! :)
I've been creating a tacacs server as part of my NMS platform for network devices with authentication happening against Azure Entra. User SSHes to a switch and gets a URL and code, user goes to URL and enters code and logs in to Microsoft Entra. User presses "Enter" on router SSH prompt and gets in with list of commands/priv associated. Password never goes through the network switch =D but validates the user authns through MS. In addition, I now collect accounting information from said devices, normalize it and ship it off to Azure log workspace table (and trigger auto backups/diffs). All of this along with authorizations gets controlled through the web UI. Next part I need to do is setup token forwarding (or similar) from the web front end to the switch as the 'password' then validate the short lived token is valid, that way a user logs into my NMS UI and clicks a device and automatically gets SSOed against their Azure creds and straight to the switch. Obv, there's a lot more to this like offline 'break glass' accounts, caching, etc, but it's coming along! Very satisfying feeling to see the pieces come together and work with real Cisco, Fortinet, Aruba, etc devices after so much effort.
I started writing my own property-based testing framework using Typescript and I think I'm making good progress. The API's are looking good. The next thing will be to start on the shrinker.
Working with Arbitraries is looking quite similar to logic programming. I should write an n-queens solver demo.
welp, I decided to take my little calendar thing a step further.
Here's how it was:
wget -O calendar.ics https://docs.googe...
Not the worst system, but it would be nice to not need the VPS involved. I decided to go with github actions...
YAML
Runs every other hour. Update the sheet and the calendars follow suit. ezpz.
Working on a personal project for a MUD I play, basically taking color codes that are in the MUD and displaying the correct colors in browser using spans.
Everything is working correctly. I mostly wanted to play with Astro and islands, which has been fun, and now I'm just trying to make things look pretty.
I picked up an e-ink panel and I'm playing with making it show home management information ("it's trash night, take the trash cans out to the curb", "laundry is done", "you left the garage door open", a calendar, maybe package tracking?). I've been referring to it as trying to automate away our executive dysfunction.
The Home Assistant API is... kind of asinine. Let's say you have some devices with battery level sensors, reporting that data to HA, and you want to show an "X has a low battery" alert for each one. So you go and ask it for a list of entity states, find the ones with class "battery" and id starting with "binary_sensor.", and then... uh. You can't get the device name from there. There literally isn't an endpoint that returns device information (an "entity" is an individual attribute of a device, devices are the physical objects and have multiple entities each; these are not the names I would have chosen). The
friendly_name
of the entity is something like "Device Name: Low battery level" by default, so you can parse it out of that as long as nobody renames anything. Or you can use/api/template
to ask the HA server to render a string template containing the value you need, because that has an entity-id-to-device-id function.I've been playing with some geospatial AI stuff recently, since I did a training course on it a month or so back. I've got a lot of geospatial experience (I work in the field), and quite a bit of AI knowledge but hadn't really put them together much before.
I spent a while putting together a simple app that can search an aerial photo using text queries like "roundabout" or "school playground". I got it working on my computer a while back, but working out how to deploy it properly on the internet, with a server that was powerful enough to deal with the AI model, HTTPS and all that jazz took a while.
The current version of it is working at https://server1.rtwilson.com/aerial - feel free to have a play, comments welcome. I definitely have some things to improve for the next version, but it's a reasonable tech demo at the moment, I think.
Cool! Not at all familiar with the area, but searched for "five way junction" and got this. Some of the boxes seem right, but others look like regular three way junction.
Nifty either way!
I started looking at migrating an web app written in asp.net with ef core to use postgres instead of sql server. I quickly ran into a weird issue that updates failed (expected 1 row affected but 0 rows affected). Still haven't figured out why. I thought ef core would be largely agnostic when it came to this type of basi operation! The updates essentially maps a domain object to an entity, then calls something like:
It's weird to me that the updates fail now, but worked when the db was sqlserver... have to look over the migrations to see if there's something weird there that is causing issues... inserts seem to work 😕 🤷