What programming/technical projects have you been working on?

I found some interesting software called Syntorial that teaches you how to use a synthesizer. It has lots of ear-training exercises where you listen to a patch and try to match its sound. I'm no closer to figuring out how to synthesize an accordion sound, but I learned how to make a synthesizer meow, quack, and boing, so I guess that's something :)

I was able to launch my frozen python script to the team I had been working with and it seems like there are few issues, if any. Have had daily check-ins with them and no issues have been raised. I still have to write out a comprehensive documentation file for them and one for the code to make changes, but overall this was a success. I wasn't able to package the portable browser, but I kept a copy of a browser located with the distribution files and added a check to ensure it was located in the folder the program is being run from.

• JSON parser/writer in golang for arbitrarily-structured data (ie not tied to a struct definition) (complete)
• alternative encoding for JSON structured data in golang (complete)
• budgeting software in golang (in progress)

Nothing particularly interesting here, I’m doing this in order to learn golang.

I've been learning a lot about reinforcement learning. In particular, I've become somewhat obsessed with the OpenAI Gym LunarLander-v2 environment. As the name implies, your algorithm controls a small spacecraft that is supposed to land on a landing pad in the center of the screen by firing its directional and main thrusters at appropriate times.

From what I can tell, the environment is considered solved when your average score over the past 100 episodes reaches or exceeds 200. I've seen several reports of people solving the environment within 600 episodes, which is something I still can't do. Sometimes I suspect they don't use the same criterion for calling the environment solved, but that is hard to verify.

There is an interesting tension between sample efficiency (few frames/episodes) and wall-clock time (few minutes). At the wall-clock time end of the spectrum, I implemented the Cross-Entropy Method with a linear policy, and it reliably solves the environment in about 10 minutes (on a single CPU), but it can run through 2000+ episodes while doing so.

Aside: That it works so well is somewhat surprising; LunarLander-v2 is very well-suited for gradient-based algorithms due to its dense reward structure, but the Cross-Entropy Method is gradient-free. It works more like an Evolution Strategy or Genetic Algorithm in that it only cares about the sum of rewards over the entire episode and ignores the temporal distribution of the rewards.

At the other extreme is Bootstrapped Dual Policy Iteration. From eyeballing the charts in the paper, it seems to come close to solving the environment within 600 episodes, but it is incredibly slow. It took 10 hours to simulate 1000 episodes on my machine, and sadly it had only reached a score average of about 130 at that time.

I've thrown a lot of different algorithms at the problem over the past six months (REINFORCE, Proximal Policy Optimization, Augmented Random Search, Advantage Weighted Regression, several DQN variants, A2C, UDRL, and probably a couple more I'm forgetting -- I should get a life...). Some are quite complicated, and others surprisingly simple. At times it is extremely frustrating, because you have to re-read and re-read a paper until you figure out how exactly the pieces fit together, but once everything is in place and the spaceship lands, it feels great.

I've been taking time away from Intergrid due to hasted-development burnout, to reconvene my thoughts.

Instead, I've been working on an idea that came to me during the earlier stages of Intergrid's development, as I was researching how to position the caret at cursor position when entering edit mode on a node via clicking. The thing I came upon was a JS library to allowed insert an HTML element at click location – and since I was working on a note-taking app at the time, I immediately considered similar applications.

Long story short, I'm making something like OneNote, confined to a single browser viewport. (Viewport is the part of the page you can see. When you scroll, you slide the page through the viewport, allowing yourself to see different parts of it.)

It's a fun little side project to take my mind off things and challenge myself in a different capacity. I won't be pursuing it afterwards like I would Intergrid. Once it's done, I'll host it on GitHub (and, once that's up, on my personal website), where I will also post the source code under the MIT license. Once it's bug-free, I'll archive it leave it as is. Perhaps others would find a better use for it while I take on other things.

At launch, you'll be able to click anywhere and start a note, then drag it, resize it (beyond its natural sizing), and delete it. There will also be an undo system – which, because it's not nearly as complex as Intergrid's, will be relatively-easy to implement. I would also consider implementing offline support, so you could work with your notes even without an Internet connection, as long as you'd visited the website once. (It would be an excellent warm-up for Intergrid's upcoming offline support, too.)

Yet to come up with a good name. I'd considered Intermap ('cause it uses Inter for its base font, like Intergrid) and flat (because that's what it is: both two-dimensional graphically and affording no depth to the hierarchy of the notes you keep). None of them sounds right. I'll stick to flat for now.

There's also this project, called Memgrid (GitHug repo). I came upon it during development of flat. It's similar to what I'm working on, except it snaps the notes ("memos") to a grid. Seems like the developer is taking his project more seriously than I do this one.