17 votes

What programming/technical projects have you been working on?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

35 comments

  1. [8]
    Banazir
    Link
    I have a small homelab that's constantly growing and evolving. Right now it's a FreeNAS server and a Proxmox VM host, but I have another potential VM host that I'm working on. FreeNAS is mainly to...

    I have a small homelab that's constantly growing and evolving. Right now it's a FreeNAS server and a Proxmox VM host, but I have another potential VM host that I'm working on. FreeNAS is mainly to provide storage for various VMs via Samba or NFS - each VM gets a small slice of local storage on the Proxmox SSD, then a larger chunk of storage mounted on /home for applications to use. I discovered that I can't connect via http/https from outside my network despite my ISP leaving them open (or so they claim), so I'm planning on setting up a VPS to act as a reverse proxy.

    On the programming side, I started a project to teach myself cryptography. It's the field that got me into computers in the first place, and I've done almost nothing with it for several years now. I'm just working my way through implementations of common hashes and encryption algorithms in Lua, which is adding its own unique challenges. I picked Lua because something about writing code with it just feels good, there's no logical reason behind my choice. Progress is slow and I'm still figuring out MD2 which is relatively simple, but the RFC has been the only guidance I've used so far.

    9 votes
    1. [5]
      Mendanbar
      Link Parent
      I'm interested to hear about where you land for your reverse proxy setup. I've been trying to set one up myself and am somewhat paralyzed by the choices available with their various pros/cons.

      I'm interested to hear about where you land for your reverse proxy setup. I've been trying to set one up myself and am somewhat paralyzed by the choices available with their various pros/cons.

      2 votes
      1. [4]
        ShroudedScribe
        Link Parent
        I responded to the parent comment with the suggestion of cloudflare access. Is that something you've considered?

        I responded to the parent comment with the suggestion of cloudflare access. Is that something you've considered?

        1 vote
        1. [3]
          Mendanbar
          Link Parent
          I've been avoiding anything proprietary as much as I can, so I didn't look too deeply into cloudflare offerings. Here are the ones I was considering: Nginx Proxy Manager: I like this option...

          I've been avoiding anything proprietary as much as I can, so I didn't look too deeply into cloudflare offerings.

          Here are the ones I was considering:

          Nginx Proxy Manager: I like this option because I'm already familiar with using nginx in and out of containers. It does seem to be one of the older options, though, so I'm not sure if I'm limiting myself by sticking to outdated tech.

          Caddy: This one seems super popular, so I think I'll get similar amounts of support for issues as with Nginx. But with that comes a lot of potential misinformation, and I'm entirely new to the ecosystem here (unlike with Nginx)

          Traefik: This one feels like overkill for my small use case. But maybe I'm biased because it was used by the infrastructure team at my last corporate job... so perhaps it just feels too heavy because it was being used in a heavy environment there. I also have a set of services that are all over the place (in and out of containers, spread across devices, etc), and I've seen comments about how Traefik works best if you are using it in a more orderly and containerized environment.

          So right now I'm thinking I might try both Caddy and Nginx and see which one sticks. But I'm still fairly overwhelmed and there isn't a super clear winner.

          1. ShroudedScribe
            Link Parent
            Traefik is nice for containers because it can utilize the docker (or podman) API so you can just add labels to your compose files to tell it what to do. I also used caddy for a while and it's...

            Traefik is nice for containers because it can utilize the docker (or podman) API so you can just add labels to your compose files to tell it what to do.

            I also used caddy for a while and it's pretty good. I found having to refresh it after modifying the caddyfile to be a bit tedious, but that's my only complaint.

            And as you said, nginx is tried and true. Nothing wrong with it, other than perhaps being a little heavier than caddy. (I'd review actual benchmarks if you're seriously performance tuning, though.)

            1 vote
          2. streblo
            Link Parent
            If it's just yourself that you need the reverse proxy for, a VPN is another option. I have a Wireguard VPS on ice in the cloud for a few cents a month. I spin it up for when I'm traveling to get...

            If it's just yourself that you need the reverse proxy for, a VPN is another option.

            I have a Wireguard VPS on ice in the cloud for a few cents a month. I spin it up for when I'm traveling to get access to my home network on the road, it's super convenient and cost efficient.

            1 vote
    2. ShroudedScribe
      Link Parent
      If you're ok with the proprietary nature of it, cloudflare access (previously called tunnels) is pretty sweet. I run it in a container and have it connected to a traefik container, which in turn...

      If you're ok with the proprietary nature of it, cloudflare access (previously called tunnels) is pretty sweet. I run it in a container and have it connected to a traefik container, which in turn is linked to my other containers that get dns names. (I can explain that part of it better if you're interested.)

      There's other ways to expose services without open ports now as well, I've heard a lot of talk about tailscale but I've never used it personally.

      2 votes
    3. elight
      Link Parent
      Cool. I built a ProxMox with TrueNAS Core (formerly FreeNAS) for storage out of a 2U bought off Enay for $400. Such a steal considering a similar retail device would've cost 1k and only use 5...

      Cool. I built a ProxMox with TrueNAS Core (formerly FreeNAS) for storage out of a 2U bought off Enay for $400. Such a steal considering a similar retail device would've cost 1k and only use 5 drives to my 2U's max of 13.

      1 vote
  2. [9]
    arqalite
    Link
    For context, I'm working on an expense tracking and splitting app. The plan is to be a 100% open-source, self-hosted, ad-free alternative to Splitwise. (Written entirely in Rust and Tailwind CSS,...

    For context, I'm working on an expense tracking and splitting app. The plan is to be a 100% open-source, self-hosted, ad-free alternative to Splitwise. (Written entirely in Rust and Tailwind CSS, by the way.)

    I realized Settlemint is a name already taken by some crypto thing, so I'll need a new name. Thank god I resisted the urge to work on branding or buying a domain.

    Anyway, the app now supports separate expense groups, so right now you could theoretically use it as an expense tracker, but you'd have no statistics (those are coming, but later!).

    I spent some work ensuring my database schema is normalized and coherent (I can finally say my college classes were useful for something, lmao) so I can easily query for, say, expenses made by a specific user in a specific group. I'm concerned about performance (Splitwise is really snappy, so I need to match or even beat that, which will be a challenge) so I expect I will need to flex the design a bit, but we'll see.

    Now I want to let multiple people join a group and add expenses, then split those expenses between the group members. I'm not 100% sure on the best way to implement that, but we'll see how it goes.

    I also made the UI not suck. It's not pretty, but it's not ugly either, it's quite literally the bare minimum with a little bit of polish. My partner has been pushing a lot for a UI redesign though, which I'm willing to do (provided he actually wants to spend a few days in Figma to design it), but that will happen only after the MVP is ready. Right now I want to have the functionality nailed down, so I can import my Splitwise data (oh god, I'll need to figure that out too) and start dogfooding ASAP.

    8 votes
    1. [8]
      bugsmith
      Link Parent
      What does that entail? This sounds really interesting. Will you be integrating any banking provider APIs?

      I spent some work ensuring my database schema is normalized

      What does that entail?

      This sounds really interesting. Will you be integrating any banking provider APIs?

      1 vote
      1. [7]
        arqalite
        Link Parent
        Database normalization is a set of steps and rules to make sure your tables don't have duplicate/redundant data, that each table only contains data relevant for the object/relation/relationship it...

        Database normalization is a set of steps and rules to make sure your tables don't have duplicate/redundant data, that each table only contains data relevant for the object/relation/relationship it models, and that they have a structure that is easy to work with.

        I don't know anything about banking APIs, so no - it's not something I would be using, and AFAIK my bank doesn't have any public APIs to play around with anyway.

        4 votes
        1. [2]
          bugsmith
          Link Parent
          Thanks for the link and explanation. I only asked about APIs as that seems to be something that's becoming popular with UK targetted tools at the moment, made much easier by Open Banking. But most...

          Thanks for the link and explanation.

          I only asked about APIs as that seems to be something that's becoming popular with UK targetted tools at the moment, made much easier by Open Banking. But most of all, we have modern "challenger" banks, which are as much tech companies as they are finance, and provide excellent APIs for balance checkings and creating transactions, etc, that I could imagine being integrated effectively into a payment splitting application.

          1 vote
          1. arqalite
            Link Parent
            Yeah, unfortunately I don't follow fintech at all. Somehow I managed to not have a Revolut account despite it being very popular with European techbros. All my finances are still with a very...

            Yeah, unfortunately I don't follow fintech at all. Somehow I managed to not have a Revolut account despite it being very popular with European techbros.

            All my finances are still with a very traditional bank, and the only reason I'm developing my app is that my partner and I are absolutely tired with Splitwise's ads, and believe their Pro plan is a travesty, pricing-wise.

            We just want to split groceries and bills without constantly moving money from our accounts. With Splitwise we can see that I owe 200€ right now, so that means I'm the one paying for the next grocery trip (and the one after that).

            2 votes
        2. [4]
          danke
          Link Parent
          What normal form did you go with for your model?

          What normal form did you go with for your model?

          1. [3]
            arqalite
            Link Parent
            Third normal form, since that's what I was taught in college. I now know there are a handful more forms but I'm unsure if they're worth doing. We were told that the potential benefits are small,...

            Third normal form, since that's what I was taught in college. I now know there are a handful more forms but I'm unsure if they're worth doing. We were told that the potential benefits are small, and that third normal form is adequate for most use cases. I'm still very much a database noob.

            1 vote
            1. [2]
              danke
              Link Parent
              I wouldn't say the performance benefits are small per se, but they do only become evident at a scale unlikely with the application you describe. I find that it's usually worth decomposing to 5NF...

              I wouldn't say the performance benefits are small per se, but they do only become evident at a scale unlikely with the application you describe. I find that it's usually worth decomposing to 5NF to eliminate the need for certain classes of constraints (multivalued dependencies); it's only 6NF that should be ignored as it lacks support in basically every RDBMS, and usually implies you want a columnar db anyway.

              1. arqalite
                Link Parent
                Hm, I can give 5NF a try and see how it looks. I'd need to document myself a little bit more to understand how to do it though, do you have any resources I could read?

                Hm, I can give 5NF a try and see how it looks. I'd need to document myself a little bit more to understand how to do it though, do you have any resources I could read?

  3. [4]
    ogre
    Link
    I started (re)learning Rust. It's been fun, it's been difficult at times, but I like that. Originally I started out with the intention of using gpui to make an apple music client for linux, but...

    I started (re)learning Rust. It's been fun, it's been difficult at times, but I like that. Originally I started out with the intention of using gpui to make an apple music client for linux, but after reviewing the API documentation this morning I realized it's most likely impossible. MusicKit is for Swift, and MusicKit.js is for web apps. I don't believe there's any way for me to integrate Apple's js library into a Rust application, if anyone has intel that suggests otherwise I'm all ears!

    5 votes
    1. [2]
      bugsmith
      Link Parent
      Have you looked into Tauri?

      Have you looked into Tauri?

      6 votes
      1. ogre
        Link Parent
        I hadn't, thanks for the link!

        I hadn't, thanks for the link!

        3 votes
    2. arqalite
      Link Parent
      As mentioned, Tauri is something you should look into, but you can also use Dioxus for the UI and logic, they have a use_eval() hook which can run JS and return the result as needed. It'll...

      As mentioned, Tauri is something you should look into, but you can also use Dioxus for the UI and logic, they have a use_eval() hook which can run JS and return the result as needed.

      It'll probably be extremely hacky and ugly but it'll work.

      2 votes
  4. [3]
    Toric
    Link
    rejiggering my 3d printer. Im moving all the electronics to an external enclosure, and adding cable chains. RN im modifying my electronics enclosure to fit my specific power supply and control...

    rejiggering my 3d printer. Im moving all the electronics to an external enclosure, and adding cable chains. RN im modifying my electronics enclosure to fit my specific power supply and control boards, as well as learning how to use buck converters to power a RPI off of 24v. after that, the cable chains, and after that, I need to cut, strip, and crimp over 50 cables.

    4 votes
    1. [2]
      ogre
      Link Parent
      Crimping 50 cables?! Have you done enough cable work for it to not be nightmare anymore? Is moving electronics to a separate enclosure a common mod for 3D printers? What's the advantage?

      Crimping 50 cables?! Have you done enough cable work for it to not be nightmare anymore?

      Is moving electronics to a separate enclosure a common mod for 3D printers? What's the advantage?

      1. Toric
        Link Parent
        The reason for moving the electronics is that previous mods Ive done havent left enough room for the built in enclosure, so prior to this current round of mods, the electronics were sprawled out...

        The reason for moving the electronics is that previous mods Ive done havent left enough room for the built in enclosure, so prior to this current round of mods, the electronics were sprawled out on a table behind the printer, with no enclosure.

        Ive done enough crimping that its not hard anymore, but it will be tedious as hell, so plan is to cut all the wires, set it all out on a table in the living room, and put a show ive already watched on. My wife is also learning how to crimp things, so its not going to be that bad all things considered.

        1 vote
  5. [3]
    Trev
    Link
    I've been working on an app for managing Random Intermittent Reinforcement (also called Variable Ratio Reward Schedule) for personal benefit. I posted about it a while back, but now there's just a...

    I've been working on an app for managing Random Intermittent Reinforcement (also called Variable Ratio Reward Schedule) for personal benefit. I posted about it a while back, but now there's just a little bit of red tape left before getting something out to other users. I'm really excited for the release because figuring out how to use this concept in an active, intentional way has had a lot of impact on my life.

    On the technical side, I've been figuring out how to increase the back-end's resistance to scripting so I don't have to collect user information as early in the on-boarding process. I always hate when I install a new app, and instantly hit a wall where I have to fill out forms and check emails and enter codes in order to do anything. By contrast, in my current prototype, the app is usable without an account and a user will unlock a good amount of features before deciding if they want to create an account which also does not require an email/phone-number. The app will need some form of identification for account recovery most likely, but that can come later when it is not blocking the user from doing basic things. All the niceties for the user does mean I have to spend some time to make the API a little less easy to script (but also to debug 😢), but I'm betting it pays off.

    If you want to take a look at the landing page and get notified when it goes live, you can take a peak at https://www.tantaluspath.com/ and fill out the relevant google form. I'm always looking for feedback, so I hope if you see something improvable you let me know!

    3 votes
    1. [2]
      ShroudedScribe
      Link Parent
      I read your other post on this system and found it interesting. Do you have any sort of background in that area, or just something you found interesting and started researching and trying? I would...

      I read your other post on this system and found it interesting. Do you have any sort of background in that area, or just something you found interesting and started researching and trying?

      I would be really intrigued if you had a long list of reward ideas in your app. Coming up with potential rewards feels like a chore in itself that I'd need a reward for! :)

      1. Trev
        Link Parent
        My background is technical so I had to do a bunch of research, a mixture of academic literature as well as studying how this is actually being used on unwitting people by places like large game...

        My background is technical so I had to do a bunch of research, a mixture of academic literature as well as studying how this is actually being used on unwitting people by places like large game companies. Later when there was a fledgling prototype I added in self-experimentation.

        Good observation! There's a starter pack of basics as well as a longer list of easy suggestions to get people to the start line without requiring much effort. Eventually you start to notice what your brain wants, and you can use it as a reward either as a one-off or added to your personal list.

  6. DVNO42
    Link
    I've been creating a tacacs server as part of my NMS platform for network devices with authentication happening against Azure Entra. User SSHes to a switch and gets a URL and code, user goes to...

    I've been creating a tacacs server as part of my NMS platform for network devices with authentication happening against Azure Entra. User SSHes to a switch and gets a URL and code, user goes to URL and enters code and logs in to Microsoft Entra. User presses "Enter" on router SSH prompt and gets in with list of commands/priv associated. Password never goes through the network switch =D but validates the user authns through MS. In addition, I now collect accounting information from said devices, normalize it and ship it off to Azure log workspace table (and trigger auto backups/diffs). All of this along with authorizations gets controlled through the web UI. Next part I need to do is setup token forwarding (or similar) from the web front end to the switch as the 'password' then validate the short lived token is valid, that way a user logs into my NMS UI and clicks a device and automatically gets SSOed against their Azure creds and straight to the switch. Obv, there's a lot more to this like offline 'break glass' accounts, caching, etc, but it's coming along! Very satisfying feeling to see the pieces come together and work with real Cisco, Fortinet, Aruba, etc devices after so much effort.

    3 votes
  7. skybrian
    (edited )
    Link
    I started writing my own property-based testing framework using Typescript and I think I'm making good progress. The API's are looking good. The next thing will be to start on the shrinker....

    I started writing my own property-based testing framework using Typescript and I think I'm making good progress. The API's are looking good. The next thing will be to start on the shrinker.

    Working with Arbitraries is looking quite similar to logic programming. I should write an n-queens solver demo.

    3 votes
  8. tomf
    (edited )
    Link
    welp, I decided to take my little calendar thing a step further. Here's how it was: google sheet with date in one column and the event title etc in another in another column I've got a formula to...

    welp, I decided to take my little calendar thing a step further.

    Here's how it was:

    1. google sheet with date in one column and the event title etc in another
    2. in another column I've got a formula to generate the .ics file
    3. I publish the spreadsheet as a .tsv and limit it to the range of the generated .ics file. the tsv is only so it'll be plaintext. There won't be any tabs separating anything.
    4. on my VPS I set a cron job to wget -O calendar.ics https://docs.googe...
    5. I subscribe to the output

    Not the worst system, but it would be nice to not need the VPS involved. I decided to go with github actions...

    YAML
    name: Calendar - Television
    
    on:
      schedule:
        - cron: '* */2 * * *'
      workflow_dispatch: 
    
    env:
      FILE_NAME: tv
      SHEET_URL: https://docs.google.com/spreadsheets/d/e/2PACX-..._/pub?gid=1234567890&single=true&output=tsv&range=G1:G
    
    jobs:
      download-and-commit:
        runs-on: ubuntu-latest
    
        steps:
        - name: Checkout repository
          uses: actions/checkout@v4
    
        - name: Download file
          run: wget -O ${{ env.FILE_NAME }}.ics --accept tsv "${{ env.SHEET_URL }}"
    
        - name: Configure Git
          run: |
            git config --global user.name 'github-actions[bot]'
            git config --global user.email 'github-actions[bot]@users.noreply.github.com'
            
        - name: Check for changes
          id: changes
          run: |
            git add ${{ env.FILE_NAME }}.ics
            git diff-index --quiet HEAD || echo "has_changes=true" >> $GITHUB_ENV
    
        - name: Commit changes
          if: env.has_changes == 'true'
          run: |
            git commit -m 'Add downloaded file'
            git push origin main
          env:
            GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    

    Runs every other hour. Update the sheet and the calendars follow suit. ezpz.

    3 votes
  9. Sage
    Link
    Working on a personal project for a MUD I play, basically taking color codes that are in the MUD and displaying the correct colors in browser using spans. Everything is working correctly. I mostly...

    Working on a personal project for a MUD I play, basically taking color codes that are in the MUD and displaying the correct colors in browser using spans.

    Everything is working correctly. I mostly wanted to play with Astro and islands, which has been fun, and now I'm just trying to make things look pretty.

    1 vote
  10. em-dash
    Link
    I picked up an e-ink panel and I'm playing with making it show home management information ("it's trash night, take the trash cans out to the curb", "laundry is done", "you left the garage door...

    I picked up an e-ink panel and I'm playing with making it show home management information ("it's trash night, take the trash cans out to the curb", "laundry is done", "you left the garage door open", a calendar, maybe package tracking?). I've been referring to it as trying to automate away our executive dysfunction.

    The Home Assistant API is... kind of asinine. Let's say you have some devices with battery level sensors, reporting that data to HA, and you want to show an "X has a low battery" alert for each one. So you go and ask it for a list of entity states, find the ones with class "battery" and id starting with "binary_sensor.", and then... uh. You can't get the device name from there. There literally isn't an endpoint that returns device information (an "entity" is an individual attribute of a device, devices are the physical objects and have multiple entities each; these are not the names I would have chosen). The friendly_name of the entity is something like "Device Name: Low battery level" by default, so you can parse it out of that as long as nobody renames anything. Or you can use /api/template to ask the HA server to render a string template containing the value you need, because that has an entity-id-to-device-id function.

    1 vote
  11. [2]
    robintw
    Link
    I've been playing with some geospatial AI stuff recently, since I did a training course on it a month or so back. I've got a lot of geospatial experience (I work in the field), and quite a bit of...

    I've been playing with some geospatial AI stuff recently, since I did a training course on it a month or so back. I've got a lot of geospatial experience (I work in the field), and quite a bit of AI knowledge but hadn't really put them together much before.

    I spent a while putting together a simple app that can search an aerial photo using text queries like "roundabout" or "school playground". I got it working on my computer a while back, but working out how to deploy it properly on the internet, with a server that was powerful enough to deal with the AI model, HTTPS and all that jazz took a while.

    The current version of it is working at https://server1.rtwilson.com/aerial - feel free to have a play, comments welcome. I definitely have some things to improve for the next version, but it's a reasonable tech demo at the moment, I think.

    1 vote
    1. archevel
      Link Parent
      Cool! Not at all familiar with the area, but searched for "five way junction" and got this. Some of the boxes seem right, but others look like regular three way junction. Nifty either way!

      Cool! Not at all familiar with the area, but searched for "five way junction" and got this. Some of the boxes seem right, but others look like regular three way junction.

      Nifty either way!

  12. archevel
    Link
    I started looking at migrating an web app written in asp.net with ef core to use postgres instead of sql server. I quickly ran into a weird issue that updates failed (expected 1 row affected but 0...

    I started looking at migrating an web app written in asp.net with ef core to use postgres instead of sql server. I quickly ran into a weird issue that updates failed (expected 1 row affected but 0 rows affected). Still haven't figured out why. I thought ef core would be largely agnostic when it came to this type of basi operation! The updates essentially maps a domain object to an entity, then calls something like:

    var entry = _context.Entry(entity);
    if(entry != null) 
    {
        entry.CurrentValues.SetValues(entity);
        entry.State = EntityState.Modified;
    }
    
    await _context.SaveChangesAsync();
    

    It's weird to me that the updates fail now, but worked when the db was sqlserver... have to look over the migrations to see if there's something weird there that is causing issues... inserts seem to work 😕 🤷