What programming/technical projects have you been working on?

I spent the better part of last weekend bashing my head against the wall trying to solve a simple problem with the wrong tools. Yesterday I got the right tools and got everything working in under a minute.

For the past month or two I've been preparing to pick morel mushrooms in the remote wilderness of British Columbia. To navigate the terrain I've been creating a map in QGIS using as much publicly available data as possible. After testing it out, it's fairly difficult to navigate the thousands of random forest service roads without GPS telling where you currently are. So I spent some time looking for ways to get GPS data to my laptop.

I tried a few apps for Android that should be able to provide this functionality. It should be pretty straightforward, since GPS devices are typically just serial ports that spew NMEA data, which is a text-based format that gives your lat/long, speed, heading, altitude, current time, etc. It turns out recent versions of Android make this more difficult than it should be. There's a bluetooth serial profile, that provides a COM port over bt, so seeing as Android phones are just tiny linux computers this should be really simple. For whatever reason the app doesn't work, and people have been complaining about similar apps in the reviews, saying new versions of android don't let them run in the background, and there's a deep rabbit hole of Xposed framework root hacks to do things "the proper linux way".

Anyway, I dug through my junk drawer and turned up a Sparkfun Venus GPS receiver. This is a really good GPS receiver with high output rates, and the external antenna should work a lot better than what's inside a phone. It's just a serial device, though it does talk at 3.3v levels, meaning you can't use a RS232 adapter; It needs to be TTL levels.

I have a USB-TTL "FTDI" adapter, used for programming microcontrollers, but at the time it was 200km away in storage, so I decided I can make a USB-TTL serial adapter using an arduino I have laying around. This was a big mistake.

Arduino Nano microcontrollers have a hardware serial port, wired up to its USB port for programming. You can also make a software serial port by bit-banging the protocol over a GPIO pin. In this way you can write a small program to send data from one port to the other, forming a bridge. HOWEVER, the last time I used this GPS device I was trying to measure the acceleration of vehicles at a drag race, so it was configured for the highest output rate, 30hz, which requires a serial connection at 115200 baud. Arudinos using their CPU to emulate a serial port in real time have a hard time at such high baud rates, which led to data corruption.

So it ended up working, kinda. After an hour of figuring out that the baud rate is actually 115200, I'd connect to the serial port and I'd get something that looks suspiciously like NMEA data:

$⸮⸮⸮⸮⸮,⸮,3,05,02,20,18,12,25,,,,,,,3.2,1.6,2.8*36
$⸮⸮⸮⸮⸮,2,1,07,25,55,210,31,20,54,085,38,05,53,135,41,02,43,063,35*79
$⸮⸮⸮⸮⸮,2,2,07,11,39,060,31,12,27,168,27,18,23,242,29*4⸮
$⸮⸮⸮⸮⸮,012438.124,⸮,4909.9418,⸮,11934.4732,⸮,000.0,327.2,280322,,,⸮*76
$⸮⸮⸮⸮⸮,327.2,⸮,,⸮,000.0,⸮,000.0,⸮,⸮*09

This is... Almost usable. For whatever reason it's able to transmit numbers, but fails hard on letters. I'm still baffled why this is. This should be encoded as UTF-8 or ASCII or something, where numbers and letters are the same number of bits, just with the value offset by 20 or 30 or something.

Anyway, the GPS unit accepts binary commands to set its data rate, which can be stored in flash memory, so I spent hours trying to send the correct sequence of bytes to tell it to drop into 1Hz refresh, 9600baud communications, but with the data corruption this was impossible. I tried sending the commands on a loop, letting it run five minutes to see if one of the thousands of commands got through intact, but alas it didn't work.

Apparently arduino nanos with a ATMega 328p might be able to handle such a high serial load, so I tried some other knockoff arduinos I had laying around, but they failed too. Also apparently most knockoff arduino nano's on ebay use counterfeit Atmel chips, so that was another consideration. I could just keep trying the same thing with different parts forever, and maybe one of them might work.

Anyway, that didn't work, and I really didn't want to drive 200km for a $10 part with gas the price it is, and I don't currently have an address to order something off amazon, so I tried working with the garbage data. Like I mentioned, the numbers are mostly intact, it's just the letters that are corrupted, and they don't contain much information. So I spent a while staring at these NMEA statements, figured if the 8th field is "000.0" then it's a $GPRMC string, which should be enough for navigation. I fudged the letters, since I know I'm always going to be in the northwest quadrant of the globe I can assume the letters N and W, etc.

I came up with this really janky code to convert the garbage data into valid NMEA:

void displayGPS()
{
  char field[20];  
  getField(field, 7);
  if ( strcmp(field, "000.0") == 0) {
    String message = "GPRMC,";
    getField(field, 1);
    message = message + field + ",A,";
    getField(field, 3);
    message = message + field + ",N,";
    getField(field, 5);
    message = message + field + ",W,";
    getField(field, 7);
    message = message + field + ",";
    getField(field, 8);
    message = message + field + ",";
    getField(field, 9);
    message = message + field + ",";
    getField(field, 10);
    message = message + field + ",A";
    int checksum = 0;
    //Serial.println(message);
    for(int i=0; i<=message.length();i++){
      checksum = checksum^message.charAt(i);
    }
    Serial.println("$" + message + "*" + String(checksum,HEX));
  }
}

It's mostly janky because I couldn't be bothered to learn about Arduino's String datatype, and how to concatenate them with C++'s null terminated character pointer strings. ...But it worked! Or at least it gave me lat/long on a map, and after pointing gpsd to the serial port I could actually track myself on the map. I still had a few problems though. First of all the update rate was awful. It would only update once every 10+ seconds. Secondly, it'd sometimes be off in some direction by a lot. Presumably one of the numbers in the lat/long was being corrupted, yet my dumb checksum routine didn't care, and passed it off as valid data. So I'd get a GPS track that keeps jumping hundreds of kilometres north, west, east, or south, then jumping back.

I considered using the checksum I got from the GPS and throwing out anything that's been corrupted, but I also have the problem where the checksum is in hex, and I have a problem transmitting letters, so only 10/16 of the checksums I receive have a chance of being valid.

Sooo, long story short I was in the area of my storage, so I grabbed the FTDI serial adapter. Soldered the GPS module to it, plugged it in, and everything works perfectly, the first time, no hassle, running at 115200 baud, full refresh rate. Everything's smooth and nothing went wrong at all. I should have done this from the start.

It's been a great lesson for me that even if I think I can get by with what I have on hand, using the right tools can save a ton of wasted time.

I finally have a use for one of my Raspberry Pis that I have laying around.

I bought all the parts yesterday to turn into an ADSB Receiver so that when I am flying I can have better situational awareness of the aircraft flying around me. Specifically, I am building a Stratux which is an open source implementation of a receiver that can cost upwards of $500. So far, my total costs have been $180 so huge savings. Just waiting for all the parts to come in the mail for me to slap together in my Pi.

I did a bit more work on my grammar for parsing IPA transcriptions. It's still pretty naïve, but it's better than my first iteration, and now tries to parse syllables with required nuclei and optional onsets/codas. (It still can't disambiguate syllable boundaries totally correctly or handle any language-specific phonotactic constraints, so there are likely some edge cases I haven't thought of yet.)

Project lives here.

The test cases I pulled out from some Wikipedia pages make a decent demo (and show how slow the parser is):

(ipa) $ ./tests/run.zsh 
/mǎi mài mâi mái/ PASS
/ˈkatən/ PASS
[ˈkhætn̩] PASS
[ˈdʒæk|pɹəˌpɛəɹɪŋ ðə ˈweɪ|wɛnt ˈɒn‖] PASS
[↑bɪn.ðɛɹ↘|↑dɐn.ðæt↘‖] PASS
[túrán↑tʃí nè] PASS
[xɤn˧˥ xaʊ˨˩˦] PASS
[ˈɹɪðm̩] PASS
[ˈhuːˀsð̩ɣ] PASS
[ˈsr̩t͡sɛ] PASS
[ɹ̝̍] PASS
[ʙ̞̍] PASS
èlʊ́kʊ́nyá PASS
huʔ˩˥ PASS
mā PASS
nu.jam.ɬ̩ PASS
a˩˥˥˩˦˥˩˨˧˦˧ PASS
[u ↑ˈvẽ.tu ˈnɔ.ɾtɯ ku.mɯˈso.ɐ.suˈpɾaɾ.kõˈmũi.tɐ ˩˧fu.ɾiɐ | mɐʃ ↑ˈku̯ɐ̃.tu.maiʃ.su˩˧pɾa.vɐ | maiz ↑u.viɐ↓ˈʒɐ̃.tɯ.si.ɐk.õʃ↓ˈɡa.va.suɐ ˧˩ka.pɐ | ɐˈtɛ ↑kiu ˈvẽ.tu ˈnɔɾ.tɯ ˧˩d̥z̥ʃtiu ǁ] PASS
( while read l; do; echo -n "$l " | tee /dev/stderr | ( ./ipa_grammar.py - > )  5.86s user 0.20s system 99% cpu 6.113 total

I'm about to get started on building a Raspberry Pi-based thermostat module to work with Home Assistant and the HVAC setup in our house. We have hydronic heating with 5 zones (each with their own thermostats), and forced-air cooling with 2 zones (which overlap with the 5 heat zones). My plan is to make one module that can sit in the basement and connect to all of the heating and AC relays, then use Home Assistant to implement the actual thermostat rules based on temperature sensors in the house, occupancy, scheduling, etc.
This project has been on my mind for a while, but I managed to snag a Pi and it seems like a good time to get started now.

I'm learning Vue.js for work via a course recommended to me on Scrimba. I believe the course may be a bit out of date, but it's still going OK.

Despite being a lifelong numbers-cruncher, I have at least a passing familiarity with major concepts like the DOM and experience with JavaScript in the context of Node (as well as getting it to run on the JVM), so it hasn't been too rocky. The thing I'm most worried about is that I just... don't feel like I have an inclination towards design in general, like I don't tend to notice the visual details of sites I interact with. But hey, trying to have a growth mindset about it!