19 votes

What programming/technical projects have you been working on?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

41 comments

  1. [12]
    acdw
    Link
    I'm slowly starting to try FreeBSD in a VM, hopefully to switch over to it soonish. I'm also thinking about how to restyle my personal website to be more ... presentatonal ? I was going to try...

    I'm slowly starting to try FreeBSD in a VM, hopefully to switch over to it soonish.

    I'm also thinking about how to restyle my personal website to be more ... presentatonal ?
    I was going to try writing an ssg using POSIX tools, but I'm not sure if that's the best use of my time.

    7 votes
    1. [6]
      vord
      Link Parent
      There's a lot to love about FreeBSD (Top-notch ZFS being one of those), and it's totally worth exploring. I actually just decided to migrate off FreeBSD at home (except OPNSense for my router). I...

      There's a lot to love about FreeBSD (Top-notch ZFS being one of those), and it's totally worth exploring.

      I actually just decided to migrate off FreeBSD at home (except OPNSense for my router). I enjoyed it, but here are the big reasons I jumped ship:

      • Containers have a much bigger community backing and deploying them is easier than using jails.
      • Neat developments from openSUSE I've been meaning to test.
      • For all the hate systemd gets, after getting used to it, older init systems are harder to use and I find myself missing great systemd features.
      • BTRFS is sufficient for my uses.
      • In light of everything else, wasn't worth the added hassle of having to context switch between Linux and FreeBSD. Same reason I try to consolidate on one Linux distro at home, rather than diverse ecosystem with different nuances.
      5 votes
      1. [5]
        acdw
        Link Parent
        Neat! Can I ask what you use containers for? I don't really get the purpose, at least for my one-laptop setup at home. I'm guessing you have more machines, most (now all) of which run Linux.

        Neat! Can I ask what you use containers for? I don't really get the purpose, at least for my one-laptop setup at home. I'm guessing you have more machines, most (now all) of which run Linux.

        1 vote
        1. [4]
          vord
          (edited )
          Link Parent
          So containers are one of those things that don't make sense, until suddenly they do and you rarely if ever want to use anything else. Here's a quick and dirty explanation: They are like miniature...

          So containers are one of those things that don't make sense, until suddenly they do and you rarely if ever want to use anything else. Here's a quick and dirty explanation:

          They are like miniature VMs that have standardized scripts for building them from scratch. There are public repositories of these scripts, and anybody can include a definition for how to build a container for their program, either in their code base or added to a public repository.

          They can also be linked together using an orchestration system, so it becomes possible to deploy multiple full stack applications in almost complete isolation from the rest of the system(s). In general, they make the most sense for development and running server programs.

          One other caveat: Due to the underlying tech stack, it's a Linux-exclusive technology. Microsoft has recently developed some sort of incompatible windows-based containers, but otherwise any containers running on non-Linux systems are concealing some sort of Linux VM in the background, including Docker for Desktop on Windows.

          I turned the rest of my post into its own top-level, as it got too big.

          4 votes
          1. [3]
            acdw
            Link Parent
            Cool, thanks! -- I'll give it a read over lunch. It looks like (from a cursory glance) that you use containers in your personal life? Like I said, I'll read your full post, but I'm just talking...

            Cool, thanks! -- I'll give it a read over lunch. It looks like (from a cursory glance) that you use containers in your personal life? Like I said, I'll read your full post, but I'm just talking out loud here.

            1 vote
            1. [2]
              vord
              Link Parent
              It's a virtuous cycle for me. I use containers at work, so using them for home projects teaches me to use them better at work, and since I use them at work I get better at using them at home, and...

              It's a virtuous cycle for me. I use containers at work, so using them for home projects teaches me to use them better at work, and since I use them at work I get better at using them at home, and so on.

              2 votes
              1. acdw
                Link Parent
                Ah. That's party of the problem for me...I don't know anyone else who uses Linux, at all, much less containers. I still need to read your post by the way.

                Ah. That's party of the problem for me...I don't know anyone else who uses Linux, at all, much less containers. I still need to read your post by the way.

                1 vote
    2. [3]
      jcdl
      (edited )
      Link Parent
      Related, I just switched my personal hosting over to FreeBSD without much fuss. It's nice that Digital Ocean supports it now. As for an SSG, I use Hugo, but that's because of my infatuation with Go.

      Related, I just switched my personal hosting over to FreeBSD without much fuss. It's nice that Digital Ocean supports it now. As for an SSG, I use Hugo, but that's because of my infatuation with Go.

      4 votes
      1. [2]
        acdw
        Link Parent
        I've just ordered the C language by K&R, so I'll learn that first, but I've been eyeing Go as a hobby. The problem is I just don't have any projects.

        I've just ordered the C language by K&R, so I'll learn that first, but I've been eyeing Go as a hobby. The problem is I just don't have any projects.

        2 votes
        1. jcdl
          Link Parent
          My favourite textbook on Go is The Go Programming Language, which also happens to be coauthored by Brian Kernighan.

          My favourite textbook on Go is The Go Programming Language, which also happens to be coauthored by Brian Kernighan.

          2 votes
    3. [2]
      Silbern
      Link Parent
      I used to use FreeBSD all the time as my main workstation OS, it's really lovely. I'd still be using it on my ThinkPad today, but unfortunately it doesn't have working suspend on this particular...

      I used to use FreeBSD all the time as my main workstation OS, it's really lovely. I'd still be using it on my ThinkPad today, but unfortunately it doesn't have working suspend on this particular model, which is a deal breaker for a laptop. Good luck with the switch! If you run into any problems, while I can't promise to being able to help, I still remember a few tricks that might be helpful.

      3 votes
      1. acdw
        Link Parent
        Awesome, thank you! I'm still meaning to play with it in a VM for a little bit, but I'll keep you in mind :)

        Awesome, thank you! I'm still meaning to play with it in a VM for a little bit, but I'll keep you in mind :)

        2 votes
  2. [2]
    Anwyl
    Link
    Working on a simulation of special relativity with some low accuracy general relativity stuff, hoping to turn it into edutainment giving people an intuitive feel for relativity. I've got the...

    Working on a simulation of special relativity with some low accuracy general relativity stuff, hoping to turn it into edutainment giving people an intuitive feel for relativity. I've got the basics of the positions of each object, but currently each object uses its own internal timeframe to calculate how far to move in a frame, so you can end up with fast objects being too far ahead in time.

    What I'd like to do is have each object run until it hits the observer's "present" plane, with a restriction that nothing hits the future light cone of the observer, but I'm running all this on the graphics card, and detecting that would be tricky with how I currently have it running without killing performance.

    At least I got a bit of a confidence boost when I plotted the solar probe and watched the simulation send it right by venus.

    7 votes
    1. ThatFanficGuy
      Link Parent
      That sounds really fucking cool. Like a useful tool of education, too.

      That sounds really fucking cool. Like a useful tool of education, too.

      4 votes
  3. [4]
    Deimos
    Link
    I haven't been working actively on this, but I've spent some time thinking about it lately and am curious if anyone has any thoughts or knows any projects doing something similar that I could look...

    I haven't been working actively on this, but I've spent some time thinking about it lately and am curious if anyone has any thoughts or knows any projects doing something similar that I could look at:

    There are a couple pieces of Tildes where I'd like to be including data in the open-source project instead of code. For example, I currently have this dumb "site info" file. I did that in code so that anyone else running a copy of the site could easily update and merge any new sites by just updating the code, but it should really be in a database table.

    I think eventually I'm probably just going to switch to including a dump of a table made with PostgreSQL's pg_dump tool, but then it will be the user's problem to figure out if and when they want to update their table (by loading that dump) and how to merge it with any changes they've made locally (if any). I think that'll practically always be fine, but it does feel a little weird to me.

    Does anyone know of open-source projects that ship data as part of the project, and if they have any special process or handling around it? Like I said, I don't think it's really worth worrying about, but I'm interested in knowing how other projects have approached it.

    7 votes
    1. vord
      (edited )
      Link Parent
      Yay, a subject I feel qualified to give advice on! Wikipedia does provide full data dumps, so they would be the first option that comes to mind. It would be cool for Tildes to do something similar...

      Yay, a subject I feel qualified to give advice on!

      Wikipedia does provide full data dumps, so they would be the first option that comes to mind. It would be cool for Tildes to do something similar (provide dumps of all data that could be crawled publicly anyway), and depending on choice comes with some huge side benefits. These options I'm tossing out work best if there are two separate databases: one for secret info that probably shouldn't be shared (password/email hashes and any other private data that isn't encrypted), and one for the public data.

      pg_dump is a fantastic option for database structure exports (like you'd want to do for a secrets database), and decent for one-time data imports. It's a logical text-based export, so comes with all the pros and cons of that. A well-designed database could mitigate issues with subsequent data imports, which also facilitates upstream and downstream data flow using typical CRUD operations, which could be very useful if Tildes were to ever consider implementing some sort of federated or semi-federated network. It would be neat to have self-hosted communities that 'bubble up' highly active topics to upstream servers.

      If designing a federated system is out of scope, my favorite option would be making full backups taken with pg_basebackup and WAL publicly available. Anybody could then setup perpetual downstream replicas, which enables all sorts of other interesting possibilities.

      If bandwidth is a concern, torrents and IPFS could mitigate those costs.

      3 votes
    2. ThatFanficGuy
      Link Parent
      Cataclysm ships much of its game settings as JSON. This proves easily moddable on local installations.

      Cataclysm ships much of its game settings as JSON. This proves easily moddable on local installations.

      1 vote
    3. spacecowboy
      Link Parent
      If this data is not meant to be updated at runtime then it feels confusing to load it into a database - which stores the mutable state. If you don't like that it is in Python code, then maybe load...

      If this data is not meant to be updated at runtime then it feels confusing to load it into a database -
      which stores the mutable state. If you don't like that it is in Python code, then maybe load it from a data format like JSON/yaml/protobuf?

  4. [4]
    ffmike
    Link
    I've been working on getting a Raspberry Pi Zero W set up to be a portable Pihole that I can use for DNS with my laptop. The idea is that the Pi has a connection in the 10.0.7.x subnet that...

    I've been working on getting a Raspberry Pi Zero W set up to be a portable Pihole that I can use for DNS with my laptop. The idea is that the Pi has a connection in the 10.0.7.x subnet that proxies through the USB network driver to the laptop's wireless connection, which it can use to contact upstream DNS servers. Then it supplies DNS on the 192.1.168.x shared wireless network to the laptop.

    I've almost got this working after much fiddling. Which is to say it works at home, but didn't on the road. I suspect the USB cable I used was not passing data...

    6 votes
    1. [3]
      pseudolobster
      Link Parent
      That sounds like a fun project, and maybe it'd be nice if the pi could also somehow be a wireless hotspot. That way you could use it as a condom for your cellphone and other wireless clients. As...

      That sounds like a fun project, and maybe it'd be nice if the pi could also somehow be a wireless hotspot. That way you could use it as a condom for your cellphone and other wireless clients.

      As for your laptop though, why not just edit your host file directly? Correct me if I'm mistaken but a pihole is nothing more than a DNS server that has a custom host file, no?

      The main advantage of a pihole is that you can use it on devices where you can't change the host file, like unrooted cellphones, smart tvs, etc. If you're only using it on one device, and that device is a laptop, you ought to just be able to edit the host file directly.

      I'm not sure which host files pihole uses, but MPVS is probably the biggest single list out there.

      3 votes
      1. [2]
        ffmike
        Link Parent
        The PiHole is a custom DNS server with auto-updating lists from a variety of sources. It also adds a management UI with the ability to see what's going on and adjust whitelist & blacklist. So...

        The PiHole is a custom DNS server with auto-updating lists from a variety of sources. It also adds a management UI with the ability to see what's going on and adjust whitelist & blacklist. So yeah, I could do everything directly in the hosts file, but I'm willing to fuss a bit to get the extras.

        2 votes
        1. pseudolobster
          Link Parent
          Fair enough. The auto-updating alone is probably worth the effort. I suppose an equally kludgy, but slightly neater way would be to run PiHole in a virtual machine and pass your wifi card to the...

          Fair enough. The auto-updating alone is probably worth the effort.

          I suppose an equally kludgy, but slightly neater way would be to run PiHole in a virtual machine and pass your wifi card to the VM. That way whenever you start the VM your wifi card disappears and a bridged virtual ethernet adapter appears. No configuration needed on the host. No need to carry extra hardware.

          I mean, your way's fine too. I can appreciate the desire to make cool, semi-useful gadgets out of raspis. I have a small but growing collection of weather displays, portable emulators systems, computer vision experiments etc piling up. My next project is probably going to be hooking one up to my garage door opener and writing an android app (or just a web interface) that'll let me control the garage door over the internet.

          1 vote
  5. gpl
    Link
    I'm still working on the same cosmological simulations that I've mentioned in previous posts. There's still a few bugs to work out, but things are "looking" more and more realistic by the day. In...

    I'm still working on the same cosmological simulations that I've mentioned in previous posts. There's still a few bugs to work out, but things are "looking" more and more realistic by the day. In order to test the remaining issues I'm having to run larger simulations which slows things down since I have to wait for time on bigger machines. I'm just relieved I've worked out the big issues that were plaguing me for the last few months.

    5 votes
  6. [3]
    kemosabe
    Link
    My mother-in-law is helping me translate my website to Croatian one evening per week. It's a good excuse for in-law family time.

    My mother-in-law is helping me translate my website to Croatian one evening per week. It's a good excuse for in-law family time.

    5 votes
    1. [2]
      DonkeySlingshot
      Link Parent
      Do you speak any Croatian? I think everybody should learn a foreign language, it's incredibly challenging but feels so rewarding once you get proficient. Good for your brain!

      Do you speak any Croatian?

      I think everybody should learn a foreign language, it's incredibly challenging but feels so rewarding once you get proficient. Good for your brain!

      3 votes
      1. kemosabe
        Link Parent
        Yeah, I speak a bit and have done a beginner course (A1 level). Can communicate but still not really conversational. As a native English speaker it's really hard to get into. Just everything is...

        Yeah, I speak a bit and have done a beginner course (A1 level). Can communicate but still not really conversational. As a native English speaker it's really hard to get into. Just everything is super weird. Grammar, vocab, genders, everything. German was much, much easier.

        Finished the website translations last night, have to finish a few other things, and then I'll be ready to deploy. Really looking forward to it.

        3 votes
  7. [4]
    vord
    Link
    I'm trying to reduce my computer footprint, as I've definitely allowed my home setup to sprawl over the years, and is consuming way more electricity than I want. As of last week, here is an...

    I'm trying to reduce my computer footprint, as I've definitely allowed my home setup to sprawl over the years, and is consuming way more electricity than I want. As of last week, here is an inventory of all my actively running computers, with some of the more interesting notes:

    • Wife's laptop - Windows 10
    • My laptop - openSUSE Tumbleweed
    • Headless gaming PC - Windows 10
      • Need one of these to use a consumer GPU headless
      • I use Parsec, Steam, and VNC to access remotely. Parsec works best overall, Steam best for mobile phones, VNC as a worst-case option if one of the others needs fixed.
      • To save power, system goes to sleep after 10 minutes of inactivity. Wake On Lan packets are sent to bring it back up when needed.
      • Going to setup a smart plug for remote hard-booting for hung UI situations.
    • Dedicated OPNSense router
      • Better performance than a typical home router for fiber internet
    • Raspberry Pi 4 running Home Assistant
    • Raspberry Pi 3 running LibreELEC
    • FreeBSD file server
    • openSUSE Leap hosting server

    Those last two servers are getting merged together, and the OS being replaced with openSUSE Tumbleweed - Transactional Server.

    Future plans include:

    • Merging in Home Assistant to free up a Pi for other projects.
    • Finding a suitable Linux-based replacement for OPNSense to consolidate that as well.
    • Transferring the OEM Windows 10 license on the gaming rig to a VM to learn how to do GPU passthrough, then migrate everything into that system.
    • Upgrade gaming rig with additional GPU or two, more RAM, and an 8 core or higher CPU to host a more gaming VMs for family and guests. Did a cost/benefit on this and it ends up being way cheaper than building separate systems.
    • Use leftover parts from all this merging to build an offsite backup server to setup at a friend's house.
    • Experiment with using that as a fail over system for my services in event of extended power outage or internet loss.
    5 votes
    1. [3]
      DonkeySlingshot
      Link Parent
      I don't understand what those Dummy HDMI plugs do, could you explain?

      I don't understand what those Dummy HDMI plugs do, could you explain?

      2 votes
      1. [2]
        vord
        Link Parent
        So essentially they emulate a monitor being connected to your computer. Without one of those (or a monitor) hooked into your GPU, Windows won't use GPU acceleration for games. If you just want to...

        So essentially they emulate a monitor being connected to your computer. Without one of those (or a monitor) hooked into your GPU, Windows won't use GPU acceleration for games.

        If you just want to be able to remotely access your normal computer, they're not needed. But I tossed all my server computers (including that one) in the basement to reduce noise and heat in other rooms, so having a monitor hooked up nobody would use seemed pointless.

        3 votes
  8. [7]
    ThatFanficGuy
    Link
    I had a lengthy response written on the subject of the game mods I'm planning on doing. Before sending the response, I opened two more tabs – and the browser crashed. Fuck you, Chromium. (If any...

    I had a lengthy response written on the subject of the game mods I'm planning on doing. Before sending the response, I opened two more tabs – and the browser crashed. Fuck you, Chromium. (If any one of you uses this as an opportunity to promote Firefox, I'm going to find you and delete all your unfinished projects and drafts.)

    Basically, there's a game called Cataclysm: Dark Days Ahead. There's a mod called Advanced Gear, which adds nanotech gear into the game. One of those is a nanosuit. I've decided, in the face of the mod fading into oblivion, to take matters into my own hands and start a new mod, taking Lorith's previous work as grounds for improvement. It's aiming to replace grind with a modest system of updates that keep the player engaged.

    The other mod is aiming to replace the current CBM system with an alternative system of hub-based self-installing bionics. CBMs don't give you the transhumanist feeling they're supposed to, being, you know, bionics. I'm aiming to ameliorate that by providing a comprehensive bionics experience.

    There. Fuck you, Chromium.

    4 votes
    1. [4]
      xstresedg
      Link Parent
      Something something Firefox something something better. But that sounds really cool, the modding aspect. I hope you enjoy the project. I've never heard of the game before though, so I may look...

      Something something Firefox something something better.

      But that sounds really cool, the modding aspect. I hope you enjoy the project. I've never heard of the game before though, so I may look into it.

      4 votes
      1. [3]
        ThatFanficGuy
        Link Parent
        how dare you One of the hurdles of modding C:DDA is the fact that some of its features are still hardcoded into the engine. The folks at CleverRaven are working hard to ameliorate that, but it's a...

        Something something Firefox something something better.

        how dare you

        One of the hurdles of modding C:DDA is the fact that some of its features are still hardcoded into the engine. The folks at CleverRaven are working hard to ameliorate that, but it's a big engine for a small team, so the progress is slow.

        Other than that, it's fixin' to be a fun challenge. My best modding so far has been editing others' mods for my preference. Compares quite well with how I started web dev. Now, I'm using others' projects as blueprints for my own. So very web dev.

        I am become Craft, the creator of things.

        2 votes
        1. [2]
          xstresedg
          Link Parent
          New word for me. Nice! But on the rest of it, fun challenges are good challenges. If I want to try my hand at modding, is there a particular avenue you recommend, or just picking something I like...

          ameliorate

          New word for me. Nice!

          But on the rest of it, fun challenges are good challenges. If I want to try my hand at modding, is there a particular avenue you recommend, or just picking something I like that is moddable (i.e. Minecraft) and throw my entire self-worth into the ring?

          1 vote
          1. ThatFanficGuy
            Link Parent
            I find modding as with any other project that mixes the technical and the creative: find a game you enjoy modding, and add something to it. Doesn't have to be big or deep or impressive: dipping...

            I find modding as with any other project that mixes the technical and the creative: find a game you enjoy modding, and add something to it. Doesn't have to be big or deep or impressive: dipping your feet into the pool should be enough of the first step.

            If you end up not liking it, you haven't yet committed to a major project that you'll feel guilty about not finishing. If you do, it'll boost your confidence and give you a little mor insight into the process to allow for bigger things down the road.

            Other than that, I'm not the person to ask. The most I can do is give you a bit of perspective from the other things I've done that are similar in nature. If you figure better, you might well be right. :)

            2 votes
    2. [2]
      mrbig
      Link Parent
      I write all my long posts on Emacs, it auto saves and is much safer. But any text editor will do.

      I write all my long posts on Emacs, it auto saves and is much safer.

      But any text editor will do.

      1 vote
      1. ThatFanficGuy
        Link Parent
        It happened exactly once in years, so it's not a problem that needs solving so far. Although a good solution should this become more annoying to deal with.

        It happened exactly once in years, so it's not a problem that needs solving so far.

        Although a good solution should this become more annoying to deal with.

        1 vote
  9. xstresedg
    Link
    I just started a new gamedev project, "Arpeedjee." As you may be able to tell from the project name, it's an RPG. Just wanting to try some new things so I'm sure it'll be interesting. I'll likely...

    I just started a new gamedev project, "Arpeedjee." As you may be able to tell from the project name, it's an RPG. Just wanting to try some new things so I'm sure it'll be interesting. I'll likely start with just building out a battle system a la Dragon Quest (NES). Relatively simple when it's one player, one monster. Then we'll expand from there.

    Also started fiddling with Python, and made a number guessing game. Gotta start somewhere haha.

    EDIT: I'm also going to be playing around with a Rapsberry Pi 2b at some point. Borrowed from a friend. I want to see if I can make something to have as an excuse to buy a RP4.

    3 votes
  10. moocow1452
    (edited )
    Link
    I finally got around to using GPT-2 to create fairy tales using Grimm's canon as a seed, and Max Woolf's Colabratory Notebook. GPT-2 is far better with articles then with narrative, since it tends...

    I finally got around to using GPT-2 to create fairy tales using Grimm's canon as a seed, and Max Woolf's Colabratory Notebook. GPT-2 is far better with articles then with narrative, since it tends to just sort of wander off at the end rather than coming to a conclusion, which is a little more acceptable in a news report over a story. Would be nice if you could force it to end with "The End" and it would have to work its way backwards as well as forward.

    2 votes
  11. tnkflx
    Link
    Raspberry Pi Kubernetes cluster.

    Raspberry Pi Kubernetes cluster.

    1 vote
  12. joplin
    Link
    A few years ago I read about The Hungarian Algorithm for solving the Assignment Problem. It looks easy enough, but there's one step (step 3 in the Wikipedia article) that is never fully explained...

    A few years ago I read about The Hungarian Algorithm for solving the Assignment Problem. It looks easy enough, but there's one step (step 3 in the Wikipedia article) that is never fully explained in a way that makes it obvious how to program it. Looking on various coding sites like Stack Overflow, I see that others have hit the same hurdle as me. There's another way to solve it by creating a bipartite graph, but all of the explanations of it use a lot of mathematical symbology that I don't understand, so it's exhausting trying to understand it. Anyway, I put in a little time this week revisiting it.

    1 vote