How do you plan or outline a program?

You're welcome! Glad you found it helpful.

I think Code Complete is exactly the sort of book you're looking for. It's not an "intro to programming" type book, it's targeted at people who already know the nuts and bolts of programming and are struggling with the things you're struggling with.

With the example classes I gave, there really are two different types of classes. I think of them as data classes and worker classes.

side note: the @dataclass stuff added in Python recently is just one way of writing data classes. You can also use a 3rd-party library like attrs, or write them yourself (though this involves a fair bit of boilerplate code). If you read the PEP that introduced Python dataclasses a big part of their motivation is having a convenient way to do data class stuff within the Python standard library, where relying on a 3rd-party library like attrs isn't an option. So don't worry too much about specifics of Python's @dataclass, and focus on the underlying idea of "data classes". You could do all of this in Java or C++ that doesn't have the @dataclass magic that Python has.

The Foo class in my example above is a data class. It's just data, it doesn't do anything. The FooProcessor class is a worker class. It does things with the data in a Foo object.

For a concrete example, let's suppose I have a Raspberry Pi with a little temperature & humidity sensor wired to it. I want to write a script that measures the temperature & humidity once a minute, and sends it off to a database. Then from that database I can draw graphs of temperature over time, or figure out averages, etc.

I can write the skeleton of my program pretty easily:

def main():
  while True:
    temperature, humidity = read_from_sensor()
    timestamp = time.time()
    write_to_database(timestamp, temperature, humidity)
    time.sleep(60)

Now, I could write, all in the same single .py file, the read_from_sensor and write_to_database methods, plus all their helper methods. That would get unwieldy pretty fast. So instead I'm going to split it up into classes. Here's a data class:

@attrs
class WeatherMeasurement:
  timestamp = attrib(type=int)
  temperature = attrib(type=float)
  humidity = attrib(type=float)

It's just a bundle of data representing a measurement. At some time, we read the sensor and got a temperature and humidity value from it.

I've switched to using an @attrs class instead of an @dataclass to underscore the point that the idea of data classes is what's important, not the specifics of how @dataclass functions in Python.

And I've also got some worker classes:

class SensorReader:
  def __init__(self, device_path):
    self.device_path = device_path

  def read(self):
    ...
    return measurement

This class knows all the low-level electronic details of how to talk to the sensor I have connected to my Raspberry Pi, and decode its output format. It has a read() method, which returns an instance of the WeatherMeasurement object. This is encapsulation - I ask the object to read the sensor, and it gives me back a measurement. I don't have to think about how it reads from the sensor. If I'm working on a project with other people, someone else might write that class, and I just call its read method. The class might even be provided in a third-party library - say the manufacturer of the sensor wrote a Python library to handle the low-level details of talking to the sensor for you.

In the constructor I give it a device path (or some other way of identifying which sensor I'm talking to). What that's useful for is, suppose I decide to connect two sensors to one Raspberry Pi, and have one sensor inside a window and the other outside. All I need to do is create two instances of the SensorReader class, and pass in the correct device path for each one.

My other worker class takes those measurements and knows how to write them to a database, let's say over HTTP:

class MeasurementRecorder:
  def __init__(self, database_url):
    self.database_url = database_url

  def record(self, measurement):
    ...

My main method now looks like this:

def main():
  sensor = SensorReader('/dev/sensor0')
  recorder = MeasurementRecorder('http://localhost:8888')
  while True:
    measurement = sensor.read()
    recorder.record(measurement)
    time.sleep(60)

Two worker classes, passing a data class between them, which is a very common thing to do.

A really useful thing to note here is that my worker objects are initialized once, at the beginning of the script, and then re-used over and over again. That's useful because worker classes very often have some long-lived state they keep around as member variables. For my sensor device, I'm opening up some kind of connection to it, and I might want to keep that connection open rather than closing & re-opening it every minute. If that's every minute, it may not be a big deal, but imagine I want to record every second, or even 10 times a second.

That's even more important for the recorder - if I'm talking to the database over HTTP I'd want to use something like requests.Session for connection keep-alives. Otherwise I'd be opening up a whole new HTTP connection (including the TCP handshake and possibly the TLS negotiation too) every single second, which can add significant overhead.

Where this object-oriented design really shines is when things start to change.

Suppose you're using FooDatabase to store your weather measurements, and you aren't quite happy with it. You read on Hacker News about this hot new BarDatabase you want to try. BarDatabase uses a different input format, might use SQL instead of HTTP, or whatever. That's fine. Your current MeasurementRecorder class is really a FooDatabaseRecorder, so you rename it, and you write a brand new and different BarDatabaseRecorder class. But it has the same interface - a record() method that takes a Measurement object. Then with a couple lines changed in your main() function, you can write to both databases at the same time to see which one you like best. When you settle on which one you like, it's really easy to rip out the legacy code, because you only change a couple lines in main() then delete one entire class. Or, you might decide they're useful for different things and write to both permanently.

Similarly, in the example I gave above of having an indoor and outdoor temperature sensor - let's say the outdoor temperature sensor needs to be replaced with a different model because it's not waterproof enough. And the better, more waterproof model has a different low-level interface. Same deal, you write a 2nd WaterproofSensorReader class (or, if the original model is an ABC1234 sensor and the waterproof one is model DEF9876, you can name your classes after that). Both classes have the same interface - a read() method that returns a Measurement object. Another couple lines changed in your main() method, and you're reading Measurement objects from both.

The really important thing to notice here is that reading from the sensor and writing to the database are decoupled - you can do one without the other. Coupling, along with cohesion that I mentioned in the other comment, are the two crucial things to understand for the why of OOP.

Another big benefit of decoupling is ease of development / testing. When you're writing & debugging your WaterproofSensorReader class, you don't need to set up a database (either a Foo or Bar one). You can just create an instance of that Reader class and ask it for a Measurement. And you can write the BarDatabaseRecorder class without needing access to any sensor hardware at all. All it needs to do is accept a Measurement and do the right thing with it, so you can feed it canned example data. -500 degrees and 200 percent humidity yesterday.

Yes, a dataclass is a regular python class and python doesn't treat them any differently.

I think there are 2 main reasons to use them:

1. For you to declare they are primarily intended for storing state (data) rather than any processing or manipulation of that state. You can still write as many methods as you want but if they aren't trivial I think it's better to make it a regular class.
2. A shorthand to declare such classes otherwise called syntactic sugar. To make it easier to read and write code for those cases.