14 votes

What programming/technical projects have you been working on?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

21 comments

  1. ShroudedScribe
    Link
    I've been working on my backup solution for my homelab (and other files) and have finally figured it out (for now). Almost all of my containerized apps have an associated database container, so...

    I've been working on my backup solution for my homelab (and other files) and have finally figured it out (for now).

    Almost all of my containerized apps have an associated database container, so I'm including a db-backup container in each app's pod (using podman, not docker). This scheduled backup places a db backup in a directory on the local server, then runs rsync in a post-script to copy the application's app files to the same backup directory. (Apps without a db will have a similar rsync script run, still finalizing logistics for that one.) I can use an excludes file to ensure the rsync backup doesn't include SQLite database files and other temp objects that don't need backup.

    Now that everything will be in a single backup directory, I will be using rdiff-backup to copy from the local server to my NAS, in a synchronized way that leverages snapshots.

    On my NAS, I will be running an instance of Plakar to periodically snapshot this backup. Then, once every so often (perhaps every 3 days?) I will sync to a plakar instance that uses Cloudflare R2 as storage, so I have an encrypted cloud backup that I can selectively restore files from, if needed.

    9 votes
  2. [7]
    ali
    Link
    I have been really digging into a couple of tools that I was curious about recently. One is a moodboard I wanted to create to gather ideas for creative projects, but instead of deciding for one, I...

    I have been really digging into a couple of tools that I was curious about recently.

    One is a moodboard I wanted to create to gather ideas for creative projects, but instead of deciding for one, I just built my own (still looking for a name)

    And one is a space simulation because space engine wasn't available on mac (I later realized I wanted Universe Sandbox, which is Available on mac)
    But this runs in the browser. It's free if anyone wants to try it, it's available at:

    https://luna.watermelonson.com
    Still working on it intensely. But I am really happy with it so far. Any suggestions for further features are welcome. I'm still thinking about the direction, whether I want to add planetwide exploration, world generation (e.g. start with some gas and let gravity do it's work and see what we get out of it) Or maybe flying around different planets - I am in Cape Town right now and I thought about the natural beauty and how that might be the same in other planets, where no one sees it.
    Other things are something like black hole simulations
    I definitely want to go for more Astrobiology, so atmosphere simulation, but also giving the planets some more than just a color. This is the perfect/worst ADHD project because I really keep digging in and learning

    The Whiteboard is also available for anyone that wants to try (I'd love feedback) - it's the first product I might try and monetize at some point, would also love feedback on it. Let me know if you just want to try it, there's no signup required. I just am still working on both projects quite a lot

    7 votes
    1. [5]
      ShroudedScribe
      Link Parent
      Perhaps a dumb question, but what does a moodboard look like for you? If it's just throwing words and paragraphs into a single spot with the ability to move them around, would a diagramming...

      Perhaps a dumb question, but what does a moodboard look like for you?

      If it's just throwing words and paragraphs into a single spot with the ability to move them around, would a diagramming software like draw.io or excalidraw fill this need?

      Or is it more like a mind map (for which there's a lot of existing software)?

      Not trying to burst your bubble, just curious of how you might be trying to differentiate your product if it will one day become paid.

      1 vote
      1. ali
        Link Parent
        No I totally get it, and I'm wondering the same. I am right now just building something that I feel like works for me. Which is basically just the right amount of features that I need. Since I...

        No I totally get it, and I'm wondering the same. I am right now just building something that I feel like works for me. Which is basically just the right amount of features that I need. Since I don't like friction when I wanna do something.

        And I'm just learning with the tool too. So I don't really expect anyone to use it anyways, but I want to polish it, to see how I can build good products.
        I learned a lot while building it, so it was worth it in that regard.

        1 vote
      2. [3]
        CrypticCuriosity629
        (edited )
        Link Parent
        Not op, but to me creative mood boards are far more than words. It's everything from images and videos of things I can improve upon to textures, or things that inspire creativity. I do not think...

        Not op, but to me creative mood boards are far more than words. It's everything from images and videos of things I can improve upon to textures, or things that inspire creativity.

        I do not think in words, I think in like holographic simulations, so like ideally outside of a digital mood board I'd have like tactile samples of things and textures and items.

        I often struggle finding something I can use as a mood board. Pinterest was good until it kind of imploded into some kind of snootzy Goop like site. I regularly get emails that things on my Pulp Horror Art board keeps getting taken down.

        1 vote
        1. [2]
          ali
          Link Parent
          If you like, I can send you a link to my moodboard platform- it's free anyways, and I'd love your feedback. I would keep it free for as long as I can (and if the cost of hosting it would be too...

          If you like, I can send you a link to my moodboard platform- it's free anyways, and I'd love your feedback. I would keep it free for as long as I can (and if the cost of hosting it would be too much, I was thinking of either just open sourcing it if there's no market for paid users anyways)

          I've also been using Pinterest before but the AI-Art-slop kind of ruined it for me. I have built this site now to basically brainstorm creative ideas for myself for my next trips. It's been 2 weeks and I have barely built my own moodboards yet - but I'm pretty happy with it so far.

          1. CrypticCuriosity629
            Link Parent
            Sure! I wouldn't mind taking a look. I'm actually very curious about it. I've been using Obsidian for the most part recently for my "mood" boards, but it's not great at the visualization and...

            Sure! I wouldn't mind taking a look. I'm actually very curious about it.

            I've been using Obsidian for the most part recently for my "mood" boards, but it's not great at the visualization and imagery part, it's basically where I've just been saving links and images and taking notes on thoughts.

    2. ali
      Link Parent
      I've added some super fun new feature now. Random solar system generation. It's really fun to see how it plays out. And adding rings to stars and watching them fly away is also really fun. Not to...

      I've added some super fun new feature now. Random solar system generation. It's really fun to see how it plays out. And adding rings to stars and watching them fly away is also really fun. Not to toot my own horn, but this is the most fun I've had building anything, so I just want to share the joy :D

      It's built in a way that lets you go super detailed with simulations, I've added some basic atmosphere's and ideas for habitability, but I'm not sure how much deeper I'll pursue this particular project. But yeah it was great fun. Been thinking of giving the planets some mountains and whatever, but I feel like maybe it's better suited as it's own exploration "minigame"

  3. Zorind
    Link
    I’m currently working on creating a web app for budgeting! I’ve been using hledger (similar to ledger, or beancount) which is a plaintext accounting tool. Basically, transactions are added to a...

    I’m currently working on creating a web app for budgeting!

    I’ve been using hledger (similar to ledger, or beancount) which is a plaintext accounting tool. Basically, transactions are added to a plain text file, which hledger parses and computes that I use to track my expenses and income, and I do a pseudo-envelope style budget with. Since it’s stored as a text file, I can easily back it up to my iCloud.

    Currently, I add the transactions all manually every weekend by writing them into the text file (with vim & macros that auto populate most fields based on the bank account and card used) so it isn’t that painful), but I’m working on the web app so I can get nicer graphs out of it. There is a web ui it provides, but since I do some weird things with the accounts for the envelope style I’m using, the built in web ui doesn’t work for what I want it to.

    I’m ultimately hoping to create a web app that still uses hledger as the “back end” via cli commands, but one where I can more easily view graphs. Stretch goals for the project are for me to be able to add transactions to it via the web app, and ideally host it with tailscale or something so I can access it from my phone and add transactions when I make them, rather than recording them at the end of each week.

    4 votes
  4. Apos
    Link
    I've been working on my shape rendering library. snip It's a library that can render shapes in MonoGame projects along with other XNA derivatives like KNI and FNA. Recently I added gradients fills...

    I've been working on my shape rendering library. snip

    It's a library that can render shapes in MonoGame projects along with other XNA derivatives like KNI and FNA. Recently I added gradients fills which took quite a bit of research. Behind the scene, it uses signed distance fields.

    I'm adding new features so that I can use it next month in a local gamejam that I'm participating in.

    2 votes
  5. CrypticCuriosity629
    Link
    I have been trying to get TiviMate to work on Waydroid using a custom AndroidTV image compiled for Waydroid, so I can watch a personal IPTV network I host on my PC. So in the very middle of trying...

    I have been trying to get TiviMate to work on Waydroid using a custom AndroidTV image compiled for Waydroid, so I can watch a personal IPTV network I host on my PC.

    So in the very middle of trying to get this to work and thinking I finally compiled it correctly, I update my fedora that updates my python to 3.14. Which breaks Waydroid.

    That's where I'm at.

    I also created a python script that helps me update music video .nfo files in a spreadsheet like way and export all the .nfo files at once. I really want to get classic MTV style opening side credits for my 90s/00s MTV channel re-creation, as well as to be able to sort the music videos by year and genre so I can have different "Themed Music Video Hours" play throughout the day between regular programming. I've got the basic spreadsheet UI working and .nfo file gen working, but I'm still working on csv import/export.

    1 vote
  6. [3]
    Akir
    Link
    This week my digital electronics class has gone up to learning about the higher level stuff, which has brought us to learning about assembly code. The class is rediculous for reasons I am not...

    This week my digital electronics class has gone up to learning about the higher level stuff, which has brought us to learning about assembly code. The class is rediculous for reasons I am not going to go over for the time being, but it’s got me interested in trying to learn an assembly language. I thought it might be interesting to try to learn assembly on a retro computer or console because that seems to be the most simple way, and I wanted to ask if anyone had any resources they would recommend. I’m already aware about the 8bitworkshop IDE and books but figured I’d ask in case there are better places to look or alternative paths anyone would suggest.

    1 vote
    1. [2]
      Zorind
      Link Parent
      So it’s not an actual assembly language as far as I know, and maybe not what you’re looking for at all, but the game Exapunks is a game where the main gameplay is writing assembly instructions for...

      So it’s not an actual assembly language as far as I know, and maybe not what you’re looking for at all, but the game Exapunks is a game where the main gameplay is writing assembly instructions for the EXAs to “hack” things. It’s a pretty fun game!

      Either way, I would highly recommend it, even if it’s not exactly what you’re looking for

      3 votes
      1. Akir
        Link Parent
        I played through Shenzhen I/O not too long ago, actually. Even bought the physical manual for it!

        I played through Shenzhen I/O not too long ago, actually. Even bought the physical manual for it!

        2 votes
  7. tlhunter
    Link
    I just migrated my RFC Hub project from RFCHub.app to RFCHub.com. It took longer as expected since I ran into a couple DigitalOcean bugs: Their S3 clone apparently only answers to one custom...

    I just migrated my RFC Hub project from RFCHub.app to RFCHub.com. It took longer as expected since I ran into a couple DigitalOcean bugs:

    • Their S3 clone apparently only answers to one custom domain at a time
    • Their DNS editor has a display bug about which hostname is being set

    Things were then confounded by waiting for DNS updates to resolve. I'll submit a pair of bug reports to hopefully save the next engineer.

    That said the migration was a success. I'm using ForwardEmail so that I can send and receive emails and Mailgun so that the app can send transactional emails. I'm always happy when a lot of my engineering work results in a smooth experience for users.

    1 vote
  8. secret_online
    Link
    My continued rewrite on my Tiny Flowers mod for Minecraft is going well. I have all of the main functionality done, the rendering is mostly ok (there are some slight lighting differences, but it's...

    My continued rewrite on my Tiny Flowers mod for Minecraft is going well. I have all of the main functionality done, the rendering is mostly ok (there are some slight lighting differences, but it's ok for now). I'm now on to the optional extras stage.

    • Adding all of the tiny flower types to the Creative menu was actually much harder than I thought it was going to be. Mostly because I wanted to have the tiny flowers appear in the order the original ones do, which means collecting all of the original blocks before adding them. The way this works is by listening to an event that is called for each tab in the menu and modifying that one tab.
      • The first issue was that my mod's tab might come before other mods, in which case anything after won't be added to the menu because my tab has already been. I fixed this by adding in a mixin that sorts my tab to the end of the list when running the event. Of course it's possible some other mod does the same, so let's just hope they don't add any flowers.
      • The second issue is that I didn't know what order my event callback would run in relation to other mods, since this same event is used by other mods to add items to the menu in the first place. Luckily there was an intended solution for this one, which is to add a new phase to the event. There's a method you can call to make certain phases go after others, so I just made mine go after the default phase.
    • Adding a new flower type can be done entirely in JSON now, but it's a bit of a pain to make them all (there's quite a few to define models for rendering). To test this, during which I found a few bugs, I made a mod that contains just JSON that defines a new flower made from dirt. It works quite nicely.
    • I've added a new crafting recipe type for Suspicious Stew that allows you to combine many Tiny Flowers together, so you can have up to 6 different potion effects in a single bowl, or 1 potion effect for a longer duration, or something in-between.
    • One of my pet peeves from v1 of the mod was that the Tiny Wither Roses couldn't be placed on Soul Sand/Soil. So I added a field to the definition to have a list of block IDs and tags that a Tiny Flower can be placed on. So now Tiny Wither Roses can be placed on nether blocks, Tiny Cactus Flowers can be placed on more things (though not as much as the full block), and any flowers types added by other mods can have their own placement restrictions.
      • This required far more code than I thought it would, and had so many little edge cases. At one stage, I had the restriction in place for most tiny flowers, except for anything like Pink Petals or Wildflowers that were already segmentable. They couldn't be placed on their own, but if some flowers had already been placed then they'd slot right in. You could then remove that original flower to be left with something that looks illegal.

    My current side task is making a website where you can select the basic models, upload some textures, and it will spit out a .jar file that you can either directly use as a mod, or extract to get the files out to include somewhere else. I'd like to have a couple of integration mods ready to go when I release v2.0.0, in part to show how it can be done. I want to lower the barrier of entry to adding integration for my mod, and I think this will be the highest value thing I can do. Plus I have done a little web project for a while.

    I don't know whether I'll be releasing v2 for 1.21.11 or wait until 26.1. It is a breaking change and there isn't an easy upgrade path to take, so I'm tempted to wait since most modded worlds stay on a single version. On the other hand, I like it when people get to use the things I've made. It's not a now decision, anyway.

  9. xk3
    Link
    I've been getting more into multiline rewriting for simple scripts. In Rust or Go I'd import a library to do it because it seems complicated. Two months ago I discovered that one can call clear...

    I've been getting more into multiline rewriting for simple scripts. In Rust or Go I'd import a library to do it because it seems complicated. Two months ago I discovered that one can call clear via execve or something like os.system("clear") in Python and it actually works but there are some downsides like the scrollback being obliterated.

    Yesterday, I dug into this a bit more by searching à la LLM and discovered that it is actually quite simple to do. You only need to count the number of lines that you print and then move the cursor back up. I made these two commits:

    The code itself is not very interesting but I'm really happy with how it works and I want to write more scripts that make use of this.

  10. [3]
    TonesTones
    Link
    I’m just about finished my personal plain-text accounting project that I started last week. The biggest hurdle I ran into is that Beancount applies its options locally within each file when using...

    I’m just about finished my personal plain-text accounting project that I started last week. The biggest hurdle I ran into is that Beancount applies its options locally within each file when using its include directive, and I really wanted a specific separation of files. It wasn’t hard to spin up a Taskfile to collate all my inputs together into a single file to avoid that include directive entirely, but it really confused me when I first got those errors.

    I’ve got a few supporting scripts to make data entry fast. Luckily, my most-used accounts support exporting transactions in a CSV so I didn’t need to mess with a library for reading PDFs. I also cache previous transaction data so entry gets easier over time. Overall, it was a well-scoped side project for my week of holiday; still got to spend plenty of time with family and relaxing.

    1. [2]
      Zorind
      Link Parent
      Nice! I use hledger, but I did consider beancount. I don’t really remember why I ended up going with hledger instead. I do all my transaction importing “manually” since I like keeping an eye on it...

      Nice! I use hledger, but I did consider beancount. I don’t really remember why I ended up going with hledger instead.

      I do all my transaction importing “manually” since I like keeping an eye on it directly, and because I’m doing weird things with my accounts for envelope budgeting

      1 vote
      1. TonesTones
        Link Parent
        I looked at both, and both seem like good pieces of software. I ended up choosing Beancount because it is more opinionated and less customizable, which was a good choice to avoid getting lost in...

        I looked at both, and both seem like good pieces of software. I ended up choosing Beancount because it is more opinionated and less customizable, which was a good choice to avoid getting lost in the configuration (I customize everything I am able to).

        For your webapp project and envelope budgeting, the alternative makes a lot of sense. I really struggle to use a budget; it’s too much maintenance. I try to leep a regular monitor on where my money is going and decide if I need to adjust my habits (cancel a subscription, eat more cheaply, etc.). It helps that I very rarely spend money outside of my habits.

        I also still verify the transactions as I import them manually, but it helps to not have to type anything out.

        1 vote
  11. tomf
    Link
    quick and dirty ios shortcut > google form > script to parse JSON. I was scraping at first, but its pretty sloppy.. then I found this tidy chunk of JSON in the <head> -- its perfect!...

    quick and dirty ios shortcut > google form > script to parse JSON. I was scraping at first, but its pretty sloppy.. then I found this tidy chunk of JSON in the <head> -- its perfect!

    {"@context":"https://schema.org","@type":"Book","name":"The Last Yakuza: Life and Death in the Japanese Underworld","image":"https://m.media-amazon.com/images/S/compressed.photo.goodreads.com/books/1687153561i/75660873.jpg","bookFormat":"Paperback","numberOfPages":416,"inLanguage":"English","isbn":"9781957363578","author":[{"@type":"Person","name":"Jake Adelstein","url":"https://www.goodreads.com/author/show/3001096.Jake_Adelstein"}],"aggregateRating":{"@type":"AggregateRating","ratingValue":3.87,"ratingCount":1077,"reviewCount":98}}
    

    That covers me for the author, title, series (if applicable), and pages. Then I do a normal scrape for the published date and plot. A little clean up with the script and I end up with

    Author Title Series Pages Date Plot
    Michael Connelly The Black Ice Harry Bosch 2; Harry Bosch Universe 2 439 1993-12-01 plot
    Jake Adelstein The Last Yakuza: Life and Death in the Japanese Underworld 416 2023-10-17 plot

    I've got a few things like this. I've struggled with a book tracker over the years because I always wanted to use APIs, relied on ISBNs, etc -- but they're either unreliable or simply terrible. The ISBN system is good if you care about editions, but all I want to know is if I read a book six years ago or not.

    All of this is a total breeze, though. For the form

    https://docs.google.com/forms/d/e/--formID--/formResponse?usp=pp_url&entry.--fieldID--=--inputValue--&submit=Submit
    

    you can get the URL by going to the three dots > prefill form -- copy the URL, change it to formResponse and add &submit=Submit and you're set for this piece.

    The full process from the user side is

    1. on iPhone, go to the book's goodreads page
    2. share > goodreads > google sheets

    done.