16 votes

What programming/technical projects have you been working on?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

27 comments

  1. [5]
    jmpavlec
    (edited )
    Link
    I recently quit both my paying jobs to focus on my side-project full time. It has been a while since I have posted on one of these threads as I haven't had much time to work on the project. For...

    I recently quit both my paying jobs to focus on my side-project full time. It has been a while since I have posted on one of these threads as I haven't had much time to work on the project. For those unaware, my side project is called gametje.com and you can try it out right now! For some additional context, you can check out my previous Tildes post which has a lot more detail. The TLDR is that it is a site with games similar to Jackbox Games. It is playable in 9 different languages without needing to install anything (no Steam or Consoles etc). The idea is create it more like a platform with new games being added on a regular basis.

    I've been busy in August working on a major UI overhaul for the main pages and an integration with Discord. The UI overhaul is a result of working with a UX designer earlier this year. The new design is not yet live. The current one is functional but a bit uninspired so don't judge it too much yet :) I'll probably post again next thread when the new design is live. I've also been working on what I am calling "game sessions" which are sort of a lobby before an actual game starts. This allows players to join the session once and switch games without having to enter new game codes. This mainly came about as a result of integrating with Discord as their embedded activities are tied to an instanceId. I needed to give the hosting player the ability to switch/start games from that centralized point. It also needed to be able to figure out which game session to join when clicking the button in Discord itself. Game sessions in general is also a big win as you don't need to fiddle around giving out a new game code every time you want to change games.

    I've also been working on adding Discord as an Identity Provider for logging in with OAuth. This is related to the integration but also a bit separate as you can log into the site using the OAuth flow outside of Discord's app. Needed to do a bit of refactoring around the user providers as previously I only had my own username/password provider. This should also make it much easier to add new Idps like Google etc later.

    As a part of the Discord integration, I've also been tightening up the websocket code so it can properly detect when clients disconnect and remove them from the game/game session. That's been fun reading through some of that code and playing with it.

    Overall, very happy with my progress the last 3 weeks.

    Next up:

    • Change my hosting provider (probably to Hetzner) for my backend/database.
    • Finish the Android TV app to replace the Amazon Fire Store version as the store is being shut down on August 25th and the web-wrapped app it created was never the best experience.
    9 votes
    1. [2]
      xk3
      Link Parent
      This is a great sound effect lol: https://gametje.com/audio/oh-no-the-car-exploded.mp3 I like that you can choose different sounds on the score screen. The site is pretty fun!

      This is a great sound effect lol:

      https://gametje.com/audio/oh-no-the-car-exploded.mp3

      I like that you can choose different sounds on the score screen. The site is pretty fun!

      3 votes
      1. jmpavlec
        Link Parent
        Thanks for giving it a look. I've been meaning to add more sound effects... The whole site should look/feel a bit more polished next week when I get my last 3 weeks of changes live.

        Thanks for giving it a look. I've been meaning to add more sound effects...

        The whole site should look/feel a bit more polished next week when I get my last 3 weeks of changes live.

        2 votes
    2. [2]
      hobblyhoy
      Link Parent
      That's awesome! I remember the first time playing Jackbox thinking it was a great idea but also had so much room for improvements. I hope you keep us up to date on this and if you're comfortable...

      That's awesome! I remember the first time playing Jackbox thinking it was a great idea but also had so much room for improvements. I hope you keep us up to date on this and if you're comfortable share things like revenue over time. Building out a big side project into a business is a dream of mine, I'm stoked you're doing it! What's your stack for this project?

      2 votes
      1. jmpavlec
        (edited )
        Link Parent
        Thanks for checking it out! I had the exact same feeling the first time I played Jackbox games and actually started on a prototype back at the end of 2017... It was playable and fun but looked...

        Thanks for checking it out!

        I had the exact same feeling the first time I played Jackbox games and actually started on a prototype back at the end of 2017... It was playable and fun but looked terrible and would never scale. You can read a bit more about it in my devblog if you are interested. I decided to take another job at the time rather than pursuing that (wasn't the right time in my life with young kids etc).

        In Jan 2023, I started part-time (12-16hrs/week) on this project. I have people playing it each week but I need to tighten up the business model. The original idea was to get it in front of as many people as possible to get feedback rather than focusing on getting paying customers. I added guest accounts so people could try it out without signing up. Now I have people playing but nobody paying :-D. To be fair, the site is still a bit rough around the edges so I wasn't expecting paying customers yet. Now that I am working on it full-time, it should hopefully improve considerably.

        Tech stack is React/typescript on the frontend, java + spring boot on the backend. Using postgres db and redis for distributed cache. Currently running with the integrated websockets server with Spring but have plans to move to RabbitMQ if necessary for scaling. So far the load has been well within my small VPS's limits. I've done some performance tests and should be able to handle a decent amount without having to change the architecture too much.

        I've built something similar to https://joinplayroom.com/ which didn't exist when I started the project (or I didn't find it). I think that's an avenue someone could take to build something a bit quicker than I did. It does tie you to their eco system and pricing though.

        I plan to share more on my devblog, but haven't had much time (until now) to focus on writing posts. Hopefully will write one in the next week or two discussing all the Discord integration challenges.

        2 votes
  2. [5]
    shrike
    Link
    I've been moving back to physical media for the stuff I know I want to rewatch in 10 years. Movies and TV-shows just keep jumping from provider to provider on streaming and some just completely...

    I've been moving back to physical media for the stuff I know I want to rewatch in 10 years. Movies and TV-shows just keep jumping from provider to provider on streaming and some just completely disappear because of corporate bullshit.

    There is a local site that sells Blurays and 4k Blurays and has pretty good sales at times, BUT their site is a horseshit mismash of slow-ass javascript and infuriating to browse as they don't give proper filtering options.

    So a few days ago I started Claude Code and created a system that uses Python+Playwright to browse through both categories, grab the links to the movie product pages and store the prices to a local sqlite database.

    Next up is adding a watchlist + price tracking for each so I can hop on the sales when they happen.

    This is my first personal project I might actually release to public use if I can get it polished enough and can be arsed to add some kind of user accounts. ...or just store the watchlist in the URL so that no accounts are needed... 🤔

    5 votes
    1. xk3
      Link Parent
      If it's all client side filtering you can use the URI Fragment to store and load watchlists. The URI Fragment is never sent to the server so you don't need to implement any GDPR type stuff related...

      just store the watchlist in the URL so that no accounts are needed

      If it's all client side filtering you can use the URI Fragment to store and load watchlists. The URI Fragment is never sent to the server so you don't need to implement any GDPR type stuff related to user data

      3 votes
    2. [3]
      oracle17
      Link Parent
      Man, what a fantastic idea, I really wish I had the cash to indulge in 4k Blu-rays. A way you could extend your project, btw, is to rip your Blu-rays after you buy them and store them on a nas,...

      Man, what a fantastic idea, I really wish I had the cash to indulge in 4k Blu-rays. A way you could extend your project, btw, is to rip your Blu-rays after you buy them and store them on a nas, that way you'd be able to access and stream from your content pretty much anywhere, Louis Rossmann acc talks about his setup for doing so as a part of his FUTO wiki (btw, this is a fantastic read, highly recommend, if just for your self edification).

      Also, once you rip the Blu-rays, you also have the option of just storing them on your laptop - I personally use an ssd to keep some comfort movies and shows on hand (b/c the internet isn't so great where I am), so whenever I get that urge to rewatch that specific scene from that one movie, it's pretty accessible.

      I also realize this is far beyond the scope of what you're doing, so sorry about that, but I wish you the best of luck!

      2 votes
      1. [2]
        shrike
        Link Parent
        I don't have the cash either, that's why I'm building a tool to get the deals :D 30€+ for a 4k disc is way too much, but for 10-14€ it's definitely doable for movies I know I'll be rewatching 10...

        I don't have the cash either, that's why I'm building a tool to get the deals :D

        30€+ for a 4k disc is way too much, but for 10-14€ it's definitely doable for movies I know I'll be rewatching 10 years from now.

        For "normal" Blu-rays I can easily get them for under 10€ regularly by having a list of the ones I need/want and buying a few on sale.

        1 vote
        1. oracle17
          Link Parent
          That's a great point, I also would love to start building a repo of more physical media - keep us updated on how it goes and if you need any help!

          That's a great point, I also would love to start building a repo of more physical media - keep us updated on how it goes and if you need any help!

          1 vote
  3. xk3
    (edited )
    Link
    So I was wondering how to rewrite all the previous git messages in my journal from the time that I was using gitupdate. I mentioned a couple weeks ago that I wrote my own replacement: I had to...

    So I was wondering how to rewrite all the previous git messages in my journal from the time that I was using gitupdate. I mentioned a couple weeks ago that I wrote my own replacement:

    With some extra time on my hands I made something a little more descriptive. If you're just modifying one file it will include the edit that you made into the commit message, or if too long, count the number of added, deleted, and moved lines

    I had to modify my wip_message.py script to not use --staging only but the script is a lot more flexible now and I did this to rewrite all the messages in my history:

    git rebase -i (git log --reverse --pretty=%H | sed -n '2p') --exec '
          git commit --amend -m "$(wip_message.py HEAD)"
    '
    

    Pretty cool!

    Actually, I should also add some additional flags to also keep the author date original.


    Another git related thing that I do, and probably more universally useful than the above, is that I have short commands for working with files based on their git status. For example I have this fish shell function:

    function added
        git status --porcelain --short $argv | awk 'substr($0, 1, 1) == "A" && substr($0, 2, 1) == " " {print substr($0, 4)}' | string unescape --style=script
    end
    

    then I can run something like open (added) to open all the files that are new. It's really useful! (but only if you remember to use it lol--sometimes I forget).

    I revamped these to be more robust and put them all here:

    https://gist.github.com/chapmanjacobd/5ecb8be86f83dcfdcd043cf3b68b0b27


    Here are a couple useful tools I found this week:

    4 votes
  4. [9]
    lynxy
    Link
    While procrastinating revision, I drafted up a quick REST interface for the thermal printer that my partner grabbed second hand last week, and I've been letting online friends hit the endpoints...

    While procrastinating revision, I drafted up a quick REST interface for the thermal printer that my partner grabbed second hand last week, and I've been letting online friends hit the endpoints and print whatever (and we only had one short period of mostly smut).

    I'll get back to revising now, but I'm really struggling to force myself to cover the material on Greedy optimality proof by Induction & Contradiction, and Divide&Conquer isn't any more interesting to me. Maybe I'll skip ahead to Network Flow & NP-Completeness for now. Given the exam is a 45 minute oral examination, and considering the amount of material to cover, I'm guessing it won't be too proof-heavy and more general theory..

    4 votes
    1. [8]
      chundissimo
      Link Parent
      Wow I love that! How are you handling auth? Just a static password via a simple user interface? Or actual user logins? Woof I do not miss my days of algorithms courses. Interesting stuff in theory...

      Wow I love that! How are you handling auth? Just a static password via a simple user interface? Or actual user logins?

      Woof I do not miss my days of algorithms courses. Interesting stuff in theory but logical proofs were never my forte. Oral examination is interesting, I can’t picture how that would work with the subject matter; I don’t think I ever had anything like that.

      2 votes
      1. [7]
        lynxy
        Link Parent
        In all honesty, currently no auth! We have ~400 meters of thermal paper and messages are length limited, so I'm not all that worried! The most security it has is through obscurity, and the worst...

        In all honesty, currently no auth! We have ~400 meters of thermal paper and messages are length limited, so I'm not all that worried! The most security it has is through obscurity, and the worst anyone can do is print words or rude images.

        In reality it has been a surprisingly good way to encourage some of my non-techy friends to dabble with APIs! Maybe I'll stick up a front-end at some point, when I find the energy to mess with CSS.

        As for the exam, I'm really not sure either! It's what has me so anxious about it. As you say, interesting in theory! If I'm asked for mathematical proof I'll wither away on the spot ^^;

        2 votes
        1. [6]
          chundissimo
          (edited )
          Link Parent
          Well best of luck on the exam! You’ll do great! And if you don’t that’s alright too, I graduated despite many a bad exam grade 😜

          Well best of luck on the exam! You’ll do great! And if you don’t that’s alright too, I graduated despite many a bad exam grade 😜

          3 votes
          1. lynxy
            Link Parent
            Thank you! I'd be a little less terrified if my current Visa didn't hang on the outcome of these exams!

            Thank you! I'd be a little less terrified if my current Visa didn't hang on the outcome of these exams!

            2 votes
          2. [4]
            lynxy
            Link Parent
            Update: I passed! :D

            Update: I passed! :D

            2 votes
            1. [3]
              chundissimo
              Link Parent
              Ayyyy congratulations!! Hope you’re able to take at least a small amount of time to relax and celebrate the win! How did the oral aspect of it end up working?

              Ayyyy congratulations!! Hope you’re able to take at least a small amount of time to relax and celebrate the win!

              How did the oral aspect of it end up working?

              1 vote
              1. [2]
                lynxy
                Link Parent
                Thank you! The professor basically asked a number of questions- what is the topological ordering of this graph and why, perform a mergesort and describe the runtime in terms of Big-O- that sort of...

                Thank you! The professor basically asked a number of questions- what is the topological ordering of this graph and why, perform a mergesort and describe the runtime in terms of Big-O- that sort of thing. With each question there was chance for dialogue, so with certain areas in which I wasn't so certain he walked me through it, allowing me to connect the dots. It was actually quite a nice experience- I just wish I had known the scope of the subject material that I would actually be asked about, because in the end I ended up covering more material than I needed, but in not-quite enough depth (I was so ready to discuss reduction using gadgets and other stuff like that, but he laughed and said that was a bit specific for the examination). I think it was less of a comprehensive interrogation of the subject and more of a test to ensure I had actually put effort into learning the material, and not just turned in what an LLM spat out! More of a vibe check, which might be necessary in the times of AI.

                1 vote
                1. chundissimo
                  Link Parent
                  Oh that’s great! Glad to hear it was positive despite the unknowns going into it!

                  Oh that’s great! Glad to hear it was positive despite the unknowns going into it!

                  1 vote
  5. [6]
    marcus-aurelius
    Link
    Moved from ZSH to Fish, mostly because of the sluggish startup times of OMZ, I don't even have that many plugins (2), and following this guide did not improve it much. Another major contributor to...

    Moved from ZSH to Fish, mostly because of the sluggish startup times of OMZ, I don't even have that many plugins (2), and following this guide did not improve it much.

    Another major contributor to slow startup times was nvm detecting the correct node version on each directory, this was solved by fnm.

    The moving was easier than expected, the defaults of fish are better, and having web config is genius, the only issue I had was git completions, I was too used to the ones from omz, and even with this plugin they vary a bit, not sure if I will write my own aliases to match my muscle memory or simply adapt myself to the existing ones to keep it more "zero conf".

    Other than that, I got to a point on Rust where I understand all the syntax, ownership, borrowing, lifetimes, etc or at least it does not feel so alien anymore, I have to decide what to build with it yet, my background is mostly in frontend, so I look at Rust like a tool to build UI which is probably not the best use-case, I played with two interesting projects, Dioxus and egui (Their example page is incredible in terms of performance and detailed examples).

    I should build something outside my comfort zone, more backend or library oriented (I take ideas), I do prefer to build something of substance than random exercises.

    If you need convincing to use Rust, this guy does a wonderful job with his videos: @NoBoilerplate

    4 votes
    1. [3]
      tauon
      (edited )
      Link Parent
      Not a fish user, but avid zsh customizer… what’s “web config” about? Edit: I’ve taken a look at egui, and while it does look very neat, usually means accessibility has been killed by favoring...

      Not a fish user, but avid zsh customizer… what’s “web config” about?

      Edit: I’ve taken a look at egui, and while it does look very neat,

      Everything you see is rendered as textured triangles. There is no DOM, HTML, JS, or CSS.

      usually means accessibility has been killed by favoring portability, so keep that in mind depending on the use case/intended user base for your projects.

      3 votes
      1. streblo
        Link Parent
        Fish has a builtin fish_config which serves a configuration portal on localhost. I've been using fish for years but I've never actually used it but it seems useful.

        Fish has a builtin fish_config which serves a configuration portal on localhost. I've been using fish for years but I've never actually used it but it seems useful.

        2 votes
      2. marcus-aurelius
        Link Parent
        A very convenient way to configure more settings, specially if you are new and want to have an overview of what's customizable. More info: fish_config I agree that egui drawing directly to canvas...

        what’s “web config” about?

        A very convenient way to configure more settings, specially if you are new and want to have an overview of what's customizable.

        Without arguments or with the browse command it starts the web-based configuration interface. The web interface allows you to view your functions, variables and history, and to make changes to your prompt and color configuration. It starts a local web server and opens a browser window. When you are finished, close the browser window and press the Enter key to terminate the configuration session.

        More info: fish_config

        I agree that egui drawing directly to canvas it's a big issue and the reason I started to look into other UI frameworks, I would not use it for an app that might require heavy reading, but for other more niche UI it seems like a great solution.

        1 vote
    2. xk3
      (edited )
      Link Parent
      I like Rust too but I'm still pretty new to it. Fish shell was recently ported to Rust. Maybe you could add a PR to that. It is really easy to build and edit. I really have no idea what I'm doing...

      I take ideas

      I like Rust too but I'm still pretty new to it. Fish shell was recently ported to Rust. Maybe you could add a PR to that. It is really easy to build and edit. I really have no idea what I'm doing and I was able to add a feature to fish shell mostly by myself.

      I also ported my "cluster sort" idea to Rust. It works but it is slower than the python version. I'm sure it could potentially be faster, if there was more optimization done at the linfa layer... Python's numpy is heavily optimized.

      You might try creating a Rust version of "regex sort" and maybe change the name to something better at the same time... maybe "sentence sort"? idk.... But I think regex-sort could be faster in Rust without much work. A lot of it is very simple.

      I've also wanted to experiment with a Rust version of the media aspects of library with maybe a web interface since that is what most people seem to want. But I lack the time. Or even a CLI version that experiments with embedding postgres instead of using SQLite. Or maybe a rust version that compiles to wasm and also uses pglite and runs inside the browser...? Let me know if you proceed with any of those ideas and I'll be happy to be your Umarell.

      2 votes
    3. bme
      Link Parent
      Fish! Fish! Fish!

      Fish! Fish! Fish!

      2 votes
  6. tomf
    Link
    if you steal audiobooks from libby, they give you the mp3s and a metadata.json with offsets for chapters. i did up a script to split out the mp3’s into chapters (no transcoding) and then i ru it...

    if you steal audiobooks from libby, they give you the mp3s and a metadata.json with offsets for chapters. i did up a script to split out the mp3’s into chapters (no transcoding) and then i ru it through m4b-tool —- but i’m not sure this is transparent.

    all in all, it works well. i liked the old Overdrive system more, but it is what it is. i don’t know what kind of person hates themselves enough to listen to audiobooks in the libby app.

    ultimately, i’d like to have it do it all in one go. it won’t be far off, but i’ve got other similar scripts to redo first. when i got a new computer, i decided to do everything from scratch instead of bringing scripts and configs over. i’m happy i did, but there are some quirks with a few things that are beyond my comprehension. :)

    3 votes