22
votes
What programming/technical projects have you been working on?
This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?
I've been working on a bot for the StarCraft II Micro-Arena on sc2ai.net.
Since I'll likely never get a proper bot for the whole game done, I figured I'd try the micro league for now, though now that I've been working on it a couple of weeks, I have my doubts about my ability to even write something that works well on this small scale...
The MicroArena starts you off with a handful of units and the objective to destroy the enemy's pylon on the other side of the (small) map.
I'm currently developing my bot by pitting it against the simplest possible strategy, one that just attack-moves towards my pylon.
Unfortunately I just can't seem to get a win-rate higher than 62% even against this simple opponent. And 62% is a pretty bad winrate, since the attack-move bot will basically lose automatically if it spawns on the left side of the map. In that case, the built-in pathfinding will, probably due to an asymmetry in the map, go through the ravine in the center of the map where it can be shot at with impunity. So factoring out the guaranteed wins, that's an abysmal effective winrate of 24%.
Some of it comes down to randomness due to the way the units pathfind and bump into each other. Much of it is due to bad decision making though. I often see behavior that is objectively bad, but I can't really think of a way to teach the bot not to do that. Or if I find a way to teach it not to do that, something else breaks in a neverending cycle.
The bot itself is made up of a few components:
Have you considered trying a reinforcement learning based approach?
I would have liked to try that, but I don't think it is feasible when using a single computer only. Multi-agent reinforcement learning is supposedly even more difficult to get working than normal reinforcement learning, and I don't think I have enough compute to train a good policy anyway.
Running neural networks in Rust is also a big pain -- I've looked into using decision trees instead, but didn't get any useful results.
I have a few more ideas how to improve the win-rate, but getting the bot into a state that is ready for submission to the ladder will likely take two or three more weeks.
It's not super-technical, but I've been nerd-sniped by this project that came up on Hacker News a couple weeks ago: https://github.com/larryboyko/dimityjones
It's a story where each chapter is encrypted with clues from the previous chapter, so you have to solve a series of puzzles and write some code to progress through the story.
Sounds interesting, I did something similar when I was at school with the cipher challenge which is mostly archaic ciphers, but I've been recommended cryptopals as something to learn "real" encryption
The cipher challenge looks neat, I might use some of that for a Cub Scout activity about codes that's coming up soon. Have you done cryptopals yourself? It's been on my list for a while, and I started it a few years ago but got sidetracked early on looking at different sources of historical English letter frequency; I probably still have a few gigabytes of Google ngrams data in a folder somewhere.
I did set 1 of cryptopals pretty quickly but then got distracted with other things and never got bothered enough to resume it. If you're looking for a comprehensive source of n-gram frequencies without having to parse all the data I found peter norvig's blogpost helpful. Seeing how/if frequencies change over time would be interesting, because it's well known that different languages have different unigram frequencies, so you'd expect some sort of change over time as well. I think the issue with google books is that the data for older books are pretty low-quality, lots of OCR mistakes and low sample size, but better than nothing
edit: If you want a good source to start learning about how to approach the ciphers, practical cryptography is a good shout
I've been idly looking at the space of possible rules for 1-D cellular automata. Basically you have a line of cells that can be on one of two states ('alive' or 'dead') and rules that decide how a cell's state evolves based on the state of its neighbors. I think it's fun! You get some nice pretty pictures with time advancing from top to bottom.
My youngest child loves music, but is in that terrible space where digital mediums are best for durability but they can't read so navigating typical UIs is out.
So I'm building a system akin to Juuke. Got some old RFID cards and a simple reader, and rigged up a simple metadata system to associate the songs with the cards. I find it's easier than the Juuke method since I'm not gonna write to any cards, they're just dumb random 10-digit identifiers.
Currently, it's Typescript with a server hosting the music/metadata and a (fairly) simple web client. Lets them use any computer with a local network connection.
I'll probably keep the metadata model (a simple file named to the rfid card identifier containing json pointing to the song, formatted metadata, and album art. I'm probably gonna ditch the client/server and build everything into a small box using a pi, an audio hat, a power switch, and a few buttons.
Then build (or find) a small python script to accept key presses or gpio buttons for a fully self-contained system.
Nice idea! Check out https://github.com/maddox/magic-cards for some inspiration. Combined with Home Assistant you can do some awesome stuff.
You could make a card play their favorite show on the tv or a playlist on Spotify or do something with a Google home etc.
More inspiration
Oh wow, this is neat, as is Juuke. It's definitely got my brain brewing with some ideas. Love the thought of having a "physical component" to a digital music library.
Currently scrambling to work on an app to help me and my friends who will be going to Bonnaroo in June (it's a large multi-genre music festival in Tennessee for those unfamiliar). There is already an official app for the event, but some red-flag reviews regarding security concerns have prevented me from wanting to use it as a solution.
So far I've landed on a Progressive Web App (PWA) as a possible solution for the features I'm looking to implement:
I've never done any sort of app development before, so I'd love to hear suggestions as for the best way to go about this.
I am not sure a PWA will be good if there is actually no internet connection at all. In theory it is possible but as far as my understanding goes this is more intended for situations where a user might be offline sometimes. So people using the app would need to make sure they have visited the website once in order to have cached the data. There is no way to fix stuff by reinstalling the app on location, etc.
I mean, it might be possible and might be worth exploring. My main reservation is that with my experience of PWAs so far I wouldn't trust them to actually have all data available in offline operation. Granted, I only have made very limited PWAs myself and the ones I have used as a user might not have been designed that way. But I have used a fair use of them and the fact that I have never seen one that specifically works in offline mode would make me hesitant going for this solution. Not without properly testing and making sure it can work offline. More importantly, that it can work offline reliably and consistently on a variety of devices and browsers. Which is something that will cost a bunch of time when the possible outcome is that it does not work.
Am I correct in guessing that you are familiar with web development and that is why you landed on a PWA for now? If that is the case you might want to explore using something like React Native.
As far as your other requirements go:
The reason for landing on a PWA was due to a friend's suggestion who has a bit more coding experience than I. I've only really done basic python scripts and automations, so I wouldn't say I'm too familiar with web development besides the basics. My understanding is that we would be able to make the app and everything ahead of time, download everything to our phones (after testing and formatting for different phones/browsers of course), and then we would be good to go.
However, after looking at React Native, this seems to be a more natural and easier solution for what I'm trying to accomplish. The main advantage (from what I'm understanding) is that it'll operate as a dedicated app instead of having the main interface be through a web browser. I like the idea of having developed modules or packages to make implementing certain features easier.
I'm not sure about the OpenStreetMap component for what it currently requires. Given that the map feature list is pretty limited, it might be a bit overkill. Also, Bonnaroo is held on an operational farm, so the structures for the event are only up for a certain amount of days. It might be difficult to distinguish landmarks or have a visual reference if there are no roads or built landmarks to go off of in the underlying imagery.
I've actually been wanting to develop SQL skills, so I'll definitely look into that as a database solution for hosting the backend data.
Thanks for the suggestions!!
Well, PWAs are overall easiest to make as they are effectively websites with extra things added to them. So in that regard, I think your friend's suggestion is a reasonable one. The biggest concern to me would be the potential for it not working after you already put in a lot of time.
I do realize I did forget to ask about the sort of phones you will be dealing with. The most convenient situation would be for everyone to be on Android phones.
However, given you are located in the US I can only assume there is a healthy amount of iPhones in the mix? While React Native does allow for cross-platform development, I honestly don't know how easy or difficult it would be to distribute the resulting application. I also suspect you still need a macOS device for some aspects.
So this might have been one reason as well why your friend thought of using the PWA solution. The trouble there is that it is my understanding that PWA support on iOS is a bit lackluster. But, I also don't know the full details.
Totally fair, like I said, you might want to keep it simple and just pull maps from the organization website. Or if you feel up to it, go to google maps in satellite view mode, zoom in on the terrain, make a screenshot and draw in various locations yourself.
I'm still working on my server. My best friend is getting miffed with streaming services, that motivated me to revamp the server and make it more stable and consistently accessible outside my local network.
I've got it running as well as it was before, and now I've got my stack dockerized so the apps are fully portable, but I'd still like to flesh out my functionality. Currently I'm working through subtitles with Bazarr. Last time I used it, it downloaded a bunch of out of sync subtitles, but that was years ago. It seems like they've added a lot of functionality since. I was hoping there was some sort of function that could scan the current subs and report if it had typos or didn't meet the Netflix subtitles QC standards, but it looks like I'll need to rely on Subtitle Edit for that.
I've got a fair bit of anime, I'm curious how helpful Bazarr will be for that. Near all anime subs I find are translations of the JP rather than actual EN subs, so I'm on the look out for a provider that gives preference to those. I reckon this may be where the Whisper functionality will really shine. (For those that are unaware, Whisper is an AI model that processes audio and writes subs)
Once I'm satisfied with my Bazarr setup, I'll be moving into metadata, posters, and title cards. Not totally certain what tools to use for that yet. Plex Meta Manager (now Kometa?) seems like it'll do some of that, but I'm not sure if it'll do the title cards so I was also looking at TitleCardMaker as well.
Managing these containers is getting out of hand, I'll need to find something to help me organize them. I'm mostly using LinuxServer.io images, so I was curious about their fleet image. I've also heard of portainer. I haven't delved into this at all, so any input on the management aspect would be welcome.
Honestly, for a single host machine with everything in docker containers, I've found docker-compose is Good Enough unless you actively want to use it as a chance to learn something else.
The main thing I'd wanted was more organized logging and I hadn't even considered that docker was saving and managing the logs already! Thank you for the input and reminding me not to over-solve.
Someone here knows TextBlaze? It's a tool for pasting snippets using keywords, but it's incredibly powerful for its intended purpose: it can be "programmed" to do a bunch of things with commands like IF/ELSE, and EXTRACTREGEX, so even though I know nothing about programming it's been very fun to develop tools for my work using this thing. The last I was creating was fun snippet for studying Japanese verb conjugation, also another for practicing two digit multiplication (I have a lot of time at work).
But the best are the tools I have created to help me with my actual work: copy an entire website to the clipboard and write the snippet: TextBlaze will automatically build the snippet using the info contained in the tables on the website (which is in the clipboard) to make the output. The manual way was that I had to manually copy and paste (or write) each piece of information, so this makes it better: I don't have to look it up (it's not always easy to find).
It's a shame I only have it at work though. Someday I'll buy it to use it at home, but right now there's no need for me to do that.
Autohotkey is the tool that most people use on windows for this sort of stuff, on macos I know there is Keyboard Maestro and on Linux Espanso, but I don't know if they are as powerful, sounds pretty crazy what you're doing with it! Personally I would generally delegate most of the logic to python (because that's what I'm used to) as soon as it gets moderately complex, helps keep it fairly tool-agnostic. For scraping websites sometimes beautifulsoup or selenium can be less hacky. Obviously the tool you have is better than the one you don't though ;)
Years ago I wrote a tool for managing computers on a Windows network in Autohotkey. It would give me logon/log off time of each user/computer on the network, integrated vnc, remote desktop, ping, tracert, wake on lan, remote cmd execution and gathered PC specs like memory, drive, processor video card etc. it was all written to a sqlite database.
Neat. Over 10 years ago I did all the info gathering with a WMI batch script. Company had a lot of PCs at remote sites and sometimes mobile that had infrequent internet connections and needed to be properly inventoried.
Yeah, I used WMI too within the AHK script. Called those parts from a group policy in AD.
I've been working on some extensions/new versions of a web app that I released recently called the British Placename Mapper (https://placenames.rtwilson.com/). That site lets people search for parts of British placenames (like names that start with "Great" or end with "burgh") and see them on a map. It got a fair amount of traction when I released it, and a load of people were interested in similar apps for other countries.
Someone has released one for Lithuania using the same code (with my permission), but I've been playing with a few new apps too. I've done the same app for the United States, but I'm not sure my technical approach will work so well with the larger number of places in such a big country - it worked ok (but a bit slowly) for me but a number of friends who tried it said it failed to load properly.
I've also been thinking about other little geographic web apps I could create, as this one went down so well. My main expertise is in backend stuff (I'm a freelancer in geospatial software engineering), but I'm trying to gain more experience with web development and front-end stuff. I've currently got a very simple MVP of a 'centroid app' that lets you find the centroid of a polygon on a web app. Very simple geographically and coding-wise, but I've seen a lot of people discussing where the geographic centre of various places (cities, counties etc) actually is - so people might find it interesting.
Our little hobby group has finished designing a prototype of a portable gaming console based on RP2040. We have found a bunch of bugs that need fixing in the next batch and are currently writing some retro games for the two boards we have now.
If anybody is interested, I have a kxstudio container working!, so the idea is then that if anybody wanted to, say, make a Cardinal distrobox environment using upstream's package, there's an Ubuntu image with the repo, you just need a Distrobox manifest and you can spin it up in two minutes (gotta download that image first, which is the long part). I'm mostly using it for upstream Carla (virttual patchbay) and Cardinal (falktx's fork of VCV Rack 2, with all of the open source plugins), but it's helped streamline my process significantly and should work on any distro that can easily run distrobox.
I also wanted to use carla's wine VST bridging, which was why I'm switching it over from my previous Arch image.