7 votes

What programming/technical projects have you been working on?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

23 comments

  1. mahoukov
    Link
    I've been working on something for a while now. It's a combination of a decentralised identity system and a decentralised messaging system that makes use of the aforementioned new identity system....

    I've been working on something for a while now. It's a combination of a decentralised identity system and a decentralised messaging system that makes use of the aforementioned new identity system. I originally conceived them as a single system but then decided it's better to split them up because identity is the crux of the problem and messaging is just a layer on top that uses it.

    It's based on cryptographic keys with one major property: keys are paired with arbitrary mnemonic strings, e.g. your favourite username, so apps can send messages in a P2P fashion addressed to the strings directly without having to worry about IP addresses. These key-string pairs can be generated on the fly on any computer and there's no central database. When multiple people inevitably choose the same strings as their identifiers, e.g. five people choose "alice", the system has a consensus mechanism for resolving such conflicts across the entire network so "alice" will always point to one public key that all nodes will use at a given point in time for encryption and signature verification.

    I've written two papers describing both systems:

    I plan on releasing the source code and documentation some time later this month. It's all written in Python. Nomicle's code includes a Python library as well as a C library that developers can use to integrate it into their apps. I'm trying to make it as easy and simple as possible for anyone to pick up. These are the first projects in a pipeline of open source projects that build on this work and that I intend to work on and release in the near future. Messaging is one potential use of Nomicle, but I can already see more uses such as consensus-based packet routing and information publishing. In fact, the latter is the next project I'll be working on.
    --
    Ali A. Mahouk

    9 votes
  2. [9]
    joplin
    (edited )
    Link
    So I got something working with my genetic programming experiment. It's not pretty, but evolution rarely is. I should say that I started looking into this again (for like the 3rd time in 5 years)...

    So I got something working with my genetic programming experiment. It's not pretty, but evolution rarely is.

    I should say that I started looking into this again (for like the 3rd time in 5 years) because there was a link on HackerNews that pointed to this free eBook. It explains things relatively clearly and rekindled my interest in the subject. If you're interested, give it a look.

    So to start with I thought I'd try to have it generate a program that solves some simple equation. Thinking about it, I realized that I have no idea how to mechanically compute the cube root of something. (I know that I can call cbrt() or pow(x, 1.0 / 3.0), but I couldn't do it by hand.) So I thought that would be a neat test if I could get it to come up with a cube root function. I gave it the 4 basic operators (+, -, *, /) and threw in negation, and absolute value.

    I got something up and running without too much trouble. In testing it, I realized that I'd probably need to constrain the domain of the problem to get remotely useful answers. So I decided to have it calculate the cube root of values between -1,000 and 1,000. It's fascinating to watch it evolve!

    It works by generating a random tree of operators and values. It might do something like this:

    y = x + (5 * -x) / 4.258 - abs(-42.3)

    It's literally just a random string of operators and operands. (But a valid one.) I generate a couple hundred of these and then calculate their error from the proper value for a random set of x's between -1000 and 1000. I then rank them from smallest error to largest. I choose a handful of the top 10% of performers and a random selection of the rest. Then I cross breed them and mutate them. That means pulling part of one equation out and sticking it into another one to make a new "child". Or randomly changing an operator or operand. Then I test them again and repeat.

    For my initial tests, I made the stopping criteria be when the average error of all the tests is < 1.0. So over the range being tested, the values vary from the ideal by less than 1.0 (on average).

    So watching it run and plotting the best results every few generations, I see some fascinating trends. Initially, a constant in the right range gives decent error amounts. Between -1,000 and +1,000, cube root goes from -10.0 to +10.0, so the equation y = 0.0 has an average error < 10.0.

    Next it tends to find linear equations that are a decent fit. A line that goes from -10 at -1,000 to +10 at +1,000 has an even better average error than a constant.

    After that it gets kind of wacky finding various polynomials that are a really good fit for a short stretch of the number line, then diverge wildly and come back again later.

    Initially, the best fit equations after several generations were a very good fit on the ends of the range, but went to +/-infinity at 0. So I changed the test criteria so that I generate half the test values in the range -1.0 to 1.0 and half outside that range. That gave a number of interesting results, too. Often close in the -1 to +1 range and decent, but less close outside that range, but again with the occasional value shooting off to infinity at some point.

    Anyway, it was a fun diversion. I hope to be able to do something more serious with it going forward, but I have some more reading to do.

    6 votes
    1. [6]
      skybrian
      (edited )
      Link Parent
      How are you computing the error at a point? Is it the absolute difference? The square of the distance?

      How are you computing the error at a point? Is it the absolute difference? The square of the distance?

      2 votes
      1. [5]
        joplin
        Link Parent
        I'm using the square of the difference between expected value and calculated value. After averaging them I take the square root, so if I understand Wikipedia correctly, I'm calculating the root...

        I'm using the square of the difference between expected value and calculated value. After averaging them I take the square root, so if I understand Wikipedia correctly, I'm calculating the root mean square error. I don't know a lot about computing errors, and that was the first thing that popped into my head. If there's a better way to do it, I'd be interested to hear about it!

        2 votes
        1. [4]
          skybrian
          Link Parent
          Yeah, that's pretty standard. I was just wondering if big mistakes are being penalized enough. It seems like division is likely to put singularities in the evolved function, unless it's division...

          Yeah, that's pretty standard. I was just wondering if big mistakes are being penalized enough.

          It seems like division is likely to put singularities in the evolved function, unless it's division by a constant, in which case it could be replaced by multiplication. Maybe you could take out the division operator and see what you get?

          3 votes
          1. [3]
            joplin
            Link Parent
            Interesting thought! I'll give it a try and see what happens. Thanks!

            Interesting thought! I'll give it a try and see what happens. Thanks!

            3 votes
            1. [2]
              skybrian
              Link Parent
              Also it might be interesting to add a step function and see what it does with it.

              Also it might be interesting to add a step function and see what it does with it.

              2 votes
              1. joplin
                Link Parent
                Funny you should mention that! By coincidence I added a sign(x) function that returns -1 for less than 0, 0 for equal to 0 and +1 for greater than 0. So now we see functions that start out as 5.0...

                Funny you should mention that! By coincidence I added a sign(x) function that returns -1 for less than 0, 0 for equal to 0 and +1 for greater than 0. So now we see functions that start out as 5.0 * sign(x) for early passes, and then get refined to combinations of other variations on y * sign(x) or sign(x +/- y) (where y is some constant) and you get a stair-stepped approximation that gets closer and closer to the right shape as it's refined.

                3 votes
  3. [3]
    ahq
    Link
    I've been working on a new community called Gurlic. Or a platform for communities, if you will. It started as a way for me to write longform/articles. I built a minimal editor based on...

    I've been working on a new community called Gurlic. Or a platform for communities, if you will. It started as a way for me to write longform/articles. I built a minimal editor based on prosemirror, with options like custom-CSS, co-authors etc.

    Here are some examples of articles:
    https://gurlic.com/homer/the-iliad-book-i
    https://gurlic.com/plato/ion
    https://gurlic.com/raw/next-stop-immortality?theme=pudding
    https://gurlic.com/raw/the-priory-of-sion?theme=minimal
    https://gurlic.com/plato/protagoras?theme=naked

    There are quick posts too, which can contain text, images, attachments, questions, polls, reviews, and have some customization options.

    Here are some quick posts:
    https://gurlic.com/davinci/post/hjWoniPoYNDK
    https://gurlic.com/bob/post/322880636660482053
    https://gurlic.com/raw/post/321003922279890949
    https://gurlic.com/raw/post/319716669645677571

    All the articles and quick posts are brought together in profiles, like this:
    https://gurlic.com/root
    https://gurlic.com/haxor
    https://gurlic.com/omg

    Profiles can be at subdomains:
    https://root.gurlic.com

    Or at domains:
    https://bob.monster

    communities

    Gurlic has something called a "bread". A bread is a collection of posts. The most common type of bread is a community, which is something like a subreddit, subforum, or facebook group. Here are some examples:
    https://gurlic.com/programming
    https://gurlic.com/rust
    https://gurlic.com/interesting
    https://gurlic.com/space

    These can be at subdomains:
    https://programming.gurlic.com

    Or at domains:
    https://gurlic.space

    galleries

    Another type of bread is a gallery, which only accepts image posts.

    Here is a gallery:
    https://gurlic.com/art -> https://art.gurlic.com -> https://gurlic.art

    publications

    Another type of bread is a publication, which accepts articles. Here is an example:
    https://gurlic.com/classics --> https://classics.gurlic.com --> https://classics.wtf

    Posts from publications would be at:
    https://gurlic.com/classics/the-iliad-book-ii --> https://classics.gurlic.com/the-iliad-book-ii --> https://classics.wtf/the-iliad-book-ii

    nested breads

    Communities in Gurlic can be deeply and arbitrarily nested like this:
    https://gurlic.com/music
    https://gurlic.com/music/jazz -> https://music.gurlic.com/jazz
    https://gurlic.com/music/jazz/fusion -> https://music.gurlic.com/jazz/fusion

    Nested bread types don't have to match, so any kind of bread could have a 'slice' or another type, like this:
    https://gurlic.com/art/discussions
    https://gurlic.com/linux/memes

    matrix integration

    I'm working on better matrix integration. For now, new users get a @username:gurlic.com account. There is a hosted instance of element at https://gurlic.com/chat. Waiting for some progress in the matrix ecosystem so I can start integrating it into Gurlic's codebase.

    short-term goals

    Fix bugs, get feedback, open up the codebase

    medium-term goals

    Think about how to approach federation, without breaking gurlic's chaotic custom domain setup. Will probably need help from someone smarter than me.

    Thanks. Let me know if you have any feedback, questions, or advice.

    6 votes
    1. [2]
      krg
      Link Parent
      this goes for everyone else in this thread, but damn... y’all are talented as hell! This is so good, too! The aesthetic fuckin’ rules.

      this goes for everyone else in this thread, but damn... y’all are talented as hell!

      This is so good, too! The aesthetic fuckin’ rules.

      1 vote
  4. tindall
    Link
    I've been building a few small Python programs to run on the SAMD51 on the Winterbloom Sol. Started out with SeqPal, a sequencer helper which uses different MIDI channels for different outputs...

    I've been building a few small Python programs to run on the SAMD51 on the Winterbloom Sol. Started out with SeqPal, a sequencer helper which uses different MIDI channels for different outputs (ideal for melodic stuff), and just yesterday I published DrumBud, which works with more traditional drum sequencers on a single MIDI channel. The Sol is a wonderfully versatile piece of hardware but as I write more firmwares for it I realize there are a lot of holes in its libraries, so I've been working with the creator of the module to add more capabilities there.

    4 votes
  5. eledrave
    Link
    I have a new work project where we need to capture short video from customers. It needs to work on pc/mac and android/iphone. I'm not looking forward to it. It seems like the browsers have...

    I have a new work project where we need to capture short video from customers. It needs to work on pc/mac and android/iphone. I'm not looking forward to it. It seems like the browsers have advanced enough to have an api for media capture, but of course, Apple doesn't fully support it and the current support is experimental. I really don't want to build an iOS app (which still wouldn't resolve the Mac issue, though I think I could get away with mandating Chrome on Mac which might solve that issue).

    3 votes
  6. [6]
    WMWMWMWMWMWMWMWMWMWM
    Link
    I'm still working on distributed web-based message board. Recent efforts: proof-of-work puzzles to combat spam are working now. Fixed a JS error in Netscape 3.0 that was bugging me for days....

    I'm still working on distributed web-based message board.

    Recent efforts: proof-of-work puzzles to combat spam are working now. Fixed a JS error in Netscape 3.0 that was bugging me for days. Internal cleanup and bugfixing. Image preview before upload. Several new user studies and interviews. Dvorak and Russian keyboard adaptations. Three different on-screen keyboards for various devices. And more...

    I think the most interesting feature I'm still exploring is "dev_mode", which is to help me dogfood the system. When this mode is enabled, when certain keywords are present, the forum will append directly to the project's documentation files!

    3 votes
    1. [5]
      tindall
      Link Parent
      Sounds very cool! Is the source code available?

      Sounds very cool! Is the source code available?

      2 votes
      1. [4]
        WMWMWMWMWMWMWMWMWMWM
        Link Parent
        Yes, it's on github: https://www.github.com/qvtqht/shitmyself/
        4 votes
        1. [3]
          tindall
          Link Parent
          May I ask why the name was chosen?

          May I ask why the name was chosen?

          5 votes
          1. [2]
            WMWMWMWMWMWMWMWMWMWM
            Link Parent
            The origin of the name lies with the domain name... Years ago, someone I know offered to sell it to me for just 100 bucks, money I had to spend at the time. I hung on to it for years not knowing...

            The origin of the name lies with the domain name... Years ago, someone I know offered to sell it to me for just 100 bucks, money I had to spend at the time.

            I hung on to it for years not knowing what to do with it. I tried out a few toy sites, but nothing stuck for long. It's pretty memorable, and it's a .com, which is also nice.

            At some point I noticed that the letters HTML appear in order in the name, so it could be stylized as sHiTMyseLf, something which reminds me of the 90s Web, which fits my vibe.

            Finally, I realized that the offensive element is also part of the appeal for me, as well. It is hard to imagine corporate anyone quoting shitmyself.com as the source or deciding to integrate it into their stack.

            I've been told several times that it would benefit me to change it, and I may change it still, but that's where the name came from.

            3 votes
            1. tindall
              Link Parent
              That's a really interesting story, actually. Thanks for sharing it :)

              That's a really interesting story, actually. Thanks for sharing it :)

              3 votes
  7. helloworld
    Link
    I wipl be trying out Flakes, an upcoming feature of Nix package manager. I'll first be switching couple of my personal projects to it from previous shell.nix, and then will try to switch my whole...

    I wipl be trying out Flakes, an upcoming feature of Nix package manager. I'll first be switching couple of my personal projects to it from previous shell.nix, and then will try to switch my whole system to use flake

    1 vote
  8. Apos
    Link
    Last weekend I participated in the Ludum Dare 47 with a few friends. We made a time travel game using an engine made from scratch on top of MonoGame:...

    Last weekend I participated in the Ludum Dare 47 with a few friends. We made a time travel game using an engine made from scratch on top of MonoGame: https://ldjam.com/events/ludum-dare/47/justin-time.

    It was pretty nice to work with a pixel artist and a composer.

    I worked on the visual effects, writing shaders and setting up the visual pipeline. Also automated our release builds on itch.io.

    1 vote