Programming Q&A Thread

It's really not an easy thing. Honestly, I think the difficulty of it is why it's so common for people to think that rewriting entire systems is necessary. Someone new comes into a project, tries to understand the code and has trouble following it, and jumps to, "this code is garbage, we need to rewrite this from scratch." Of course, even if they do rewrite it, the next person that comes in will almost certainly think exactly the same thing about the new code too.

Overall, the ideal of easily-understandable, "beautiful" code is almost impossible for anything beyond extremely focused systems that deal with small, well-defined problems. Larger systems have to deal with a ton of edge cases and unexpected difficulties that lead to complexity. Understanding those systems will never be easy. That being said, I do have a couple pieces of advice, but I want to go off on a tangent first (which may not actually apply to you, I don't know how experienced you are):

Even though it's common advice, I don't think that trying to contribute to open-source projects is necessarily a good idea for relatively new developers. A lot of open-source projects are hugely complex systems that have built up over many years and are often trying to serve the needs of many different groups of people. This makes them hard to understand, especially if you don't know the history of the project and how they evolved.

Personally, the advice I usually give is that people should try to build something by themselves, from scratch, for a problem or use that they're interested in. It doesn't have to be anything amazing or unique, but it's important that it's something you want to exist and would use personally. That's what will keep you motivated to figure out how to implement new capabilities. The nice part about building it by yourself from scratch is that you will understand everything about the system. When you want to add something new, the only new things you have to learn are relevant to that specific addition. You don't have to work through 10 attached concerns in someone else's project before you can even start figuring out how to add the actual update.

If it's one of the first larger projects you've worked on, it will probably end up being a mess, but that's totally fine. It gives you a base to learn with, and will also help by giving you a frame of reference when you do start looking at other people's code. You'll see how they do some things and be able to think, "oh, that would have been a much better way of doing X in my program." You'll also start to get an idea of what seems to work well or poorly based on how hard some parts of your program end up being to keep working with as you keep updating it.

Anyway, tangent over. Here's my advice for if you do need to figure out a complicated system:

• If at all possible, learn how to use a debugger for whatever language/platform you're working on. It can be much easier to get an understanding by setting breakpoints and/or stepping through the program while it's running to be able to watch what it does, examine variables to see what their values are and how they change, and so on. Stepping through code in a debugger can be more like getting pulled through the code instead of needing to find your own way through it, and even though it's confusing in other ways, it can help a lot with understanding what's going on. A lot of people don't seem to learn how to use debuggers, but they're extremely powerful tools.
• If it is an open-source project and you have trouble figuring out some section of code but eventually work it out, consider writing some comments in that code that explain what's happening. Think about what information would have helped you understand it more easily, and write comments with that. Then make a pull request with those comments. If you're nervous about doing this, ask the maintainers first, but most open-source projects would love to have more comments added to their code, since that makes the same process easier for every new contributor in the future. It's not the same as contributing actual code, but it can be extremely helpful too. As a nice side benefit of it, it will also end up with more-knowledgeable people about the project confirming that you're understanding it correctly, since they'll review your comments and tell you if they're correct or missing anything.

Does anyone have experience dumping large amounts of data to disk (~100s of GB) every 5-10ish minutes or so, and if so how do you avoid I/O bottle-necking your application?

That seems way too frequent to be honest. Even my large scale CFD runs would only dump once or twice a reservation; admittedly the time step was very small due to combustion so this still worked out to pretty good time resolution. The first thing I would do is see if you can reduce the amount of info you're writing to disk, either by decreasing the I/O frequency or pruning the data if possible.

From there, OLCF (PDF Warning) has some good info on common IO strategies for MPI. One obvious one you may have overlooked: have an HDF file per rank and worry about combining them into the full dataset after the simulation is complete.

Depending on how much work you're willing to put into this, you could build a wrapper around Android WebView (or GeckoView) that sends spoofed mouse events based on an onscreen controller. I've seen it done to allow cursor navigation on an AndroidTV D-pad, for example, but sadly I don't know of any off the shelf implementations. Not to say they don't exist, of course.

It's that or carry around a bluetooth trackball in your bag, anyway...

I'm always debating using classes. They can quickly become a big mess, but can also provide very useful functionality. To contain certain functionality together, a module (file) is already good enough. When I use classes, they need to have a state. This state needs to be remembered, or can change. This can be some sort of login or open session with a webserver, or simply a user's karma/votes and what not.

Let's take a game as an example: Our game has heroes. These hero have some attributes. Some may change, some never do.

Let's define our class, with the name attribute. This name doesn't change.

class Hero:
    def __init__(self, name):
        self.name = name

Now, we can create one or multiple of these heroes.

you = Hero(name="HydoTypo")
me = Hero(name="timo")

Let's say we want our heroes to fight each other. Let's give our hero some health (a state that changes). We want to keep track of our health during the game.
We could use a function:

class Hero:
    def __init__(self, name):
        self.name = name
        self.health = 100

Let's say we want our heroes to battle it out in a fist fight. We can make punching a function of the Hero. For that, we use a class method:

class Hero:
    def __init__(self, name):
        self.name = name
        self.health = 100

    def punch(self, target, damage):
        print(f"{self.name} punches {target.name} with {damage} damage")
        target.health = target.health - damage

you.punch(target=me, damage=10)

print(f"{me.name} has {me.health} health")
"timo has 90 health"
print(f"{you.name} has {you.health} health")
"HydoTypo has 100 health"

Now, you can see that the state of your hero stays the same, but my hero's state has changed. This is a simplified but typical example of when a class can be useful.

Hope this helps a little :)

what really concerns me is the fact that I still can not get the hang of when to use classes when programming.

So it depends on what you're trying to accomplish, especially with a language like python that supports multiple paradigms. Object orientation is a hard concept to grasp for most people originally. If you're second year, I'm assuming that you've touched on OOP, at least a bit. Classes allow you to abstract your code especially when you start using inheritance/polymorphism/etc. If you're writing a straight-forward, functional script in python (ex a simple input/output school project thing), I'd say there is no reason to use classes. However, if your doing something more complicated, classes can help modularize your code and allow for re-usability.

I also want to break out of solely just using python but I honestly can't break out of it.

Kind of related to the first part but you should try more object oriented languages like java, C#, or C++. While python certainly supports OOP, it doesn't enforce it like the other languages. You're going to want something compilable rather than interpreted. Python is a great language, so I don't think there's any problem with you learning to code with it, but you learn a lot about programming concepts by learning other languages, especially ones that delve into other paradigms.

I also want to break out of solely just using python but I honestly can't break out of it.

This may sound obvious, but consider doing the same small project in several different programming languages. This allows you to compare and contrast what you like and dislike.

In my experience, something of limited scope, but that is non-trivial, works best to learn a new programming language. I like to write game-playing programs; those can be as simple as Noughts and Crosses or Connect4 or as complicated as a chess engine.

The best task I found was writing bots for online challenges such as Vindinium (sadly defunct now) or Halite (possibly defunct now?). It's really motivating to climb the leaderboards, and it is a task that is challenging enough to illuminate the potential of each programming language.

If that is too time-consuming, you could look at Exercism. They have small exercises in just about any programming language.