13 votes

What programming/technical projects have you been working on?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

13 comments

  1. [3]
    cadentius_aurelius
    Link
    I have been creating a 3D game engine in Python for a bit over a year. It does not have any graphical capability, other than a sanity-checking module that creates static images with matplotlib to...

    I have been creating a 3D game engine in Python for a bit over a year.

    It does not have any graphical capability, other than a sanity-checking module that creates static images with matplotlib to check my geometry math, and it is not real-time; However, it is still "real time". What I mean by that is that it sleeps for a period (my current setting is five minutes), taking input and such. After sleeping for that length of time, it simulates that length of time all at once, and sleeps for input until the next tick.

    The end product I have in mind is a mostly-hard science fiction game/sandbox, played over SSH, which can be summed up as, "Your starship runs a Linux server; Fly it from the command line". This is approximately where I am:

    • [x] A Client-Server framework capable of transferring JSON data structures
    • [/] A Terminal-based "fullscreen" CLI with multiple panes
    • [x] A unified World, divided into discrete Domains, built on four NumPy NDArrays for Position, Velocity, Heading, and Rotation of every Object; Time is progressed by simply adding the master Velocity Array to the master Position Array
    • [x] A Coordinates class which provides Objects an interface to find their correct values in the master Arrays
    • [x] An Object class that stores intrinsic values such as mass and radius (all Objects are spherical cows)
    • [ ] Subclasses of Object for specific things such as asteroids, ships, weaponry rounds, etc.
    • [x] A high-precision collision detection system based on a method, which I invented (but highly doubt I am the first to invent), which I call "time-slicing" (and is completely unrelated to timeslicing in computer multitasking)
    • [/] Accurate-to-reality collision reaction math between Objects

    Unfortunately I cannot share it here right now because that VCS profile is associated with a different "realm" of usernames from the ones I use here and I try to keep them separate, but I can discuss it in abstract terms.

    10 votes
    1. [2]
      determinism
      Link Parent
      Can you elaborate on "time-slicing"?

      Can you elaborate on "time-slicing"?

      5 votes
      1. cadentius_aurelius
        (edited )
        Link Parent
        The movement of an Object over a period of time is a line segment, from the point of the Position Vector P of the object to the point of P plus the Velocity Vector V multiplied by the amount of...

        The movement of an Object over a period of time is a line segment, from the point of the Position Vector P of the object to the point of P plus the Velocity Vector V multiplied by the amount of time t; that is, P + Vt. So, at the beginning of the period, lets say 1s, the object is at P, and at the end of the period, it is at P + 1V. We compare the distance between the Objects D at 0s, and at 1s, and at the midpoint, 0.5s. The change in distance over the first half, between t=0 and t=0.5, is H1, and the change over the second half is H2. This is the "slice". Additionally we sum the radii of the Objects, for R, which is the distance at which the Objects touch.

        I wont go into too much specific detail on the logic, because it is very tedious and I needed to draw myself probably ten different diagrams to get my mind around it, and even then I wasnt even sure it works until I tried it (and Im still not certain there are no edge cases). But essentially, comparing H1 with H2, and checking D against R, its possible to conclude, if the Objects collide during this time period, whether the collision happens in the first half or the second half of the period. With that information, we then repeat this process across either the first half or the second half of the time period: (0s, 0.25s, 0.5s) or (0.5s, 0.75s, 1.0s).

        This process is iterated a number of times, and eventually, it homes in on the exact moment of collision. At that moment, the Velocities of the Objects are changed and they bounce apart, and then time continues from there. Initially, I used 7 iterations, but now I have it repeat indefinitely, until either:

        1. Collision is ruled impossible.
        2. The distance D at the time of collision is known to a precision smaller than 1cm.

        E: One thing to note is that, for the purposes of my math, I mentally redefined "collision" to mean "the last possible moment before the Objects intersect". So this is probably the best way to understand it when reading it.


        E (11 months later): In the intervening months I have learned that I just reinvented the binary search :D So yeah, that is what I used here. Binary search to find the first value of time where the distance between the spheres is less than the sum of their radii.

        3 votes
  2. unknown user
    Link
    I took a week off feature development to reset and do a general spring clean of our business applications. Even though we're still a few months away from generating any revenue from our products,...

    I took a week off feature development to reset and do a general spring clean of our business applications. Even though we're still a few months away from generating any revenue from our products, I take pride in having a tidy and well-developed software architecture. Thus, I've spent the last few days...

    • Massively expanding method and class documentation to ensure at a future point I can write or use a documentation generator to produce high quality output.
    • Expanding our test suite to cover more features. I also learnt how to finally mock time to write some correctness unit tests around JSON Web Token verification (how does one test a token is invalid when it expires without literally waiting for the token to expire? etc etc). Up until we've started this business, I've always ignored testing, and it was high time I got into it.
    • I also figured out how to publish and utilize a base Docker image for our applications that includes modern PHP 7.4, composer, phpunit, and xdebug. Got that published to DockerHub. Eventually I'll host it on a private registry but that isn't important at the moment.
    • Spent some time configuring my IDE to connect to my Docker configuration to run PHPUnit tests via the GUI. It's nice being able to just click a "run" button next to a unit test to execute it, instead of executing a CLI instruction.
    • Generally reducing the complexity and size of several API endpoints, and abstracting several portions of functionality to shared classes.

    As you can see, nothing feature-related. But these efficiency improvements will pay for themselves in terms of time and reducing my frustration down the road.

    For the coming week, I'm going to be focusing on writing a TOTP implementation.

    6 votes
  3. skybrian
    Link
    Accordion synthesizer project: drilled some holes in FR4 and one board is almost done. I'm using a Dremel with a drill press attachment which is certainly more accurate than the hand drill I was...

    Accordion synthesizer project: drilled some holes in FR4 and one board is almost done. I'm using a Dremel with a drill press attachment which is certainly more accurate than the hand drill I was using before, but on the other hand a Teensy 3.6 has a lot of pins so it's a lot of tiny holes. As well as five holes per keyswitch. I'm not accurate enough to avoid having to rout out holes that are in the wrong place.

    I should probably be learning how to use KiCAD and having someone else make the boards. I'm wondering if it would make sense to design some kind of protoboard that works with keyswitches?

    4 votes
  4. Deimos
    Link
    Over the last few days I've been thinking and reading a lot about how I can remove RabbitMQ from Tildes's stack. I don't need to—it works great and has almost always been reliable in my experience...

    Over the last few days I've been thinking and reading a lot about how I can remove RabbitMQ from Tildes's stack. I don't need to—it works great and has almost always been reliable in my experience with it on both Tildes and Reddit, but it's pretty unnecessary and my goal of keeping the complexity of Tildes low makes me want to eliminate it (and the various dependencies I need to install solely for it, like Erlang).

    Right now, the method of initiating background/asynchronous tasks through it is a little convoluted:

    1. When certain events happen in the database (such as a new topic being posted), a PostgreSQL trigger sends a NOTIFY signal with a type (like topic.created) and some data (like the topic's ID).
    2. A pg-amqp-bridge service that's listening for these signals picks it up and converts it into a RabbitMQ message.
    3. Based on the type of event, RabbitMQ puts this message into one or more queues. For example, when a topic is created it needs to be added for the queues for the different link scrapers.
    4. Python services running in the background pick up these messages, get the relevant rows from the database, and do whatever processing is necessary.

    Overall, I was mostly deciding between two different options for replacing this with systems that I'm already using for other purposes anyway:

    • Just build the task queue inside PostgreSQL - this works pretty well now with SKIP LOCKED in PostgreSQL, and tasks wouldn't need to leave the database at all. The Python consumers could probably still listen for NOTIFY signals as a way of knowing there's something new to be done, instead of needing to poll.
    • Use the new "Streams" capabilities added in Redis 5.0. This basically lets you put events into a linear "log", and then you can have consumers (and groups of consumers) go through the log (and keep up with new additions) to see what they need to do.

    I ended up deciding that I'm going to use Streams since I think they have some really nice capabilities, especially once you start adding in multiple consumers dealing with the same tasks (like if you need multiple scraper processes to keep up with new links being posted). So now I need to work on a good way to update the streams based on events happening in PostgreSQL (replacing steps 2 and 3 in the current process above).

    Initially I think I'm going to write a really simple Python service similar to pg-amqp-bridge that will listen for the NOTIFY signals and add the relevant stream entries. Another option I'm looking at is writing a custom C function in PostgreSQL that will just do the Redis operations itself. I haven't written any C in a long time though and I don't have any experience with extending PostgreSQL like that. Between that and the general weirdness of getting the database to update a separate database as part of its operations, I'm probably too nervous about that approach to go straight to it, but it might be interesting to look into more in the future.

    4 votes
  5. [2]
    unknown user
    Link
    I am actually fuckin' finishing a release-ready version of Intergrid. And when I say "release", I mean "early access" sort of release. You're already able to take notes with it. Recently it gained...

    I am actually fuckin' finishing a release-ready version of Intergrid.

    And when I say "release", I mean "early access" sort of release. You're already able to take notes with it. Recently it gained the ability to handle copying and pasting, so you could manually import and export your notes. It will be released publicly, though with little publicity, once it gains the ability to save and restore the notes between sessions.

    In a way, it's mirroring Deimos' approach towards Tildes: while it's not complete, it's perfectly usable as is. It serves its purpose well, and everything from here would be an upgrade.

    This early release will see some of the functions of the app still missing. The undo system would be one of those, crucial though it is to the whole idea. There will probably be bugs, especially on platforms I haven't tested for. On the other hand, it could still be used by those who can't use the Windows-only Indigrid. The Web is fairly cross-platform.

    3 votes
    1. pvik
      Link Parent
      That's really cool! I am assuming you are making a cross-platform version of this Indigrid. First few plain google searches were showing results for Infrastructure investments or power grid...

      That's really cool!

      I am assuming you are making a cross-platform version of this Indigrid. First few plain google searches were showing results for Infrastructure investments or power grid equipments, etc... Finally searching for "Indigrid Application" got me to that!

      It looks like an interesting tool and I have never heard of it. Emacs org-mode suffices all my note taking needs! However this looks like something I could suggest to a couple of my non-emacs using friends.

      Hope you announce your early-access release on tildes, would love to try out this tool.

      3 votes
  6. [3]
    ubergeek
    Link
    Working on getting ansible vault working so a pubnix I run can be fully ci/cd. Kinda technical, working on forming a worker owned IT services coop. Technical in the legal and governance way.

    Working on getting ansible vault working so a pubnix I run can be fully ci/cd.

    Kinda technical, working on forming a worker owned IT services coop. Technical in the legal and governance way.

    3 votes
    1. [2]
      cmccabe
      Link Parent
      Hey uber, are you writing about your thunix development steps anywhere? I would be interested to follow that.

      Hey uber, are you writing about your thunix development steps anywhere? I would be interested to follow that.

      2 votes
      1. ubergeek
        Link Parent
        I've not, but surely can start :) I'll make sure I brag about it in #meta and probably #gopher when I do, since it's certainly going to get posted to the gophersphere.

        I've not, but surely can start :) I'll make sure I brag about it in #meta and probably #gopher when I do, since it's certainly going to get posted to the gophersphere.

        3 votes
  7. admicos
    Link
    So, after asking about new browser features, I tried working on my Firefox fork/patchset, but I feel like I'm starting to lose interest on it, so I'm not entirely sure if it'll ever release, even...

    So, after asking about new browser features, I tried working on my Firefox fork/patchset, but I feel like I'm starting to lose interest on it, so I'm not entirely sure if it'll ever release, even as an alpha.

    Since I've improved the build system (from Markdown guide(s) to an semi-automatic build script), I feel like this is a good time to slowly share the code.

    So, if anyone here is interested in building a privacy-focused browser with me, take a look at the repo here. (Unlisted, though the link should work)

    It's still heavily in development, so there are missing, planned features, and is definitely rough around the edges.

    Also, I haven't tried building it outside of Linux, so if there's anyone that can help with building it under Windows and/or macOS (and documenting it), please let me know!

    1 vote
  8. aldian
    Link
    I'm working on a map making / editing utility in flutter. The general idea, you can create a map, populate it with terrain, furniture, buildings, etc and then put character tokens in and use it to...

    I'm working on a map making / editing utility in flutter. The general idea, you can create a map, populate it with terrain, furniture, buildings, etc and then put character tokens in and use it to run a session for D&D / Pathfinder / whatever tabletop system that uses a 2d grid.

    It's still really early days, but I'm excited for it. I have basic terrains and tokens working and now it's just building up the other layers.

    1 vote