Fortnightly Programming Q&A Thread

Probably the best thing to do would be pick a framework (vue, angular, htmx, or even plain old jquery) and dig into the tutorials and documentation as you go. Since you already know how to code I'd skip general Javascript tutorials and primers and do a web search whenever you need to know the specific syntax or available features for something you know/suspect should exist (like "javascript switch statement" or "javascript asynchronous programming" or whatever). Even the crappy resources like w3schools will usually get you what you need at a quick glance.

One thing that throws off a lot of programmers getting into web development is the expectation that there's a single recommended default vanilla way to make a good website, in the same way that there's a main recommended way (language, tooling, and framework) by Apple to make iOS apps, there's a main recommended way by Google to make Android apps, etc. There's not. There's no single official website/webapp-making SDK. Most web developers use piles of modular open source tools that fit well together. The stack I use for most stuff personally and for work mainly consists of VSCode, Typescript, React, Next.js, Node & Express on the server, npm, ESLint, Prettier, and Jest. I would point a newcomer at a framework like Next.js because it comes with a working React + (optionally) Typescript + Node server-side-rendering setup, it's the most popular modern (frontend+backend, supports interactive components that are server or client rendered) web framework, has decent intro tutorials, works well with npm libraries, it supports fully server-rendered non-hydrated pages unlike most modern non-React frameworks, and is really easy to host yourself or on free services like Vercel.

There is plenty to learn about web development that's separate from these tools, but sometimes I see new web developers set out to do everything themselves the "vanilla" way without using any external tools in order to focus on that. I think that's a big mistake and self-handicap to put on oneself, at least for any more than a practice or small-scope project. At best it leads to a project full of not-invented-here syndrome and at worst it involves someone setting very low expectations for themselves because of how hard many basic tasks are. It's painful and involves a lot of redundant work to support things like interactive components that can be both server and client rendered if you aren't using a modern view library like React.

MDN and web.dev are my go-to resources for JS and web APIs.

Knowing when and where to break your code up into abstractions is the most subjective and contextual aspect of programming that there is, and the details really depend on what language you're using. Expect that you'll naturally get better with time and practice.

But to lay out some general steps:

1. Write your script in the simplest way possible. Once you've got that starting point:
2. Look for clusters of variables that directly depend on one another, or often change as a unit. Organize those into simple associative data structures, and collections thereof. (In python, you'll use lists/dicts; in JS you'll use arrays/objects; in some other languages, sequences/maps - but it's all the same idea regardless of label.) If this is too confusing, or doesn't apply, skip to step 3.
3. Wherever you see yourself repeating similar lines of code with the same intent, turn that into functions. Where applicable, write your functions to operate on the common data structures you made in step 2.
4. Where you see clusters of functions creating/changing/working on a unique set of simple data structures, break that into classes. (Optional - esp. for scripting.)
5. Wherever you see your script breaking into self-contained "topics" focused on a common theme, break those into self-contained files.

And most importantly, read code that has been written by other people. Don't expect to understand most of it at first, and don't spend more than thirty minutes on this per session. As long as you can get the gist of some basic parts and sniff out the general intent, you'll learn by pure osmosis.

There are existing design patterns that can help organize a project, the most popular probably being MVC (model-view-controller), but there are a ton of them (multitier architecture, microservices, pubsub--here's a good Wikipedia article that gives a broad overview). Even if none of these exactly fit your project, reading up a bit and understanding architectural patterns can still help inform your decisions.

Probably the key takeaway I'd emphasize is to focus on separation of concerns--keep related code grouped together and separated from unrelated code. That being said it's all highly subjective and there's no "best" way to do things that fits every project.

There are many good books on the subject. Some of the better ones I read were the pragmatic programmer and code complete.

a thread from a few years ago you might find helpful: https://tildes.net/~comp/pum/how_do_you_plan_or_outline_a_program

I have a couple comments there giving a high-level overview of object-orientation, which is how many (but not all) languages deal with the problem of modularization in large-scale programs.

Since FluffyKittens already gave you some great tips, I'm going to add some more general stuff and resources here.

Most people start by using the MVC pattern, which is rather easy to use and introduces you to the world of software architecture.

When you have understood and used MVC successfully (and maybe have discovered some other patterns), you can move on to more sophisticated things such as DDD (Domain Driven Design). This is probably far in the future, but it's good to know that it exists (and it rocks).

It's also important to know that software patterns are often based on object-oriented programming (OOP). So maybe have a look into that first.

• Check out MDN's OOP introduction here.
• You can learn more about patterns here.
• You can learn more about MVC here.
• If you want to know more about DDD, check this out.

And here's another tip. I haven't used Angular in ages, but I think that the way it enforces certain practices and project layouts is perfect for learning some architectural basics. It's no wonder so many big companies love it; most Angular projects look really similar to each other.

• Check out Angular here.

I'm running a very similar setup on an intel nuc with Cloudflare tunnels and Dockers for multiple websites. Just set it up a few months ago.

Why did you need to free up port 80? Cloudflare tunnels wouldn't need it. See here for how it works: https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/#:~:text=Cloudflare%20Tunnel%20provides%20you%20with,connections%20to%20Cloudflare's%20global%20network

I don't run PiHole so maybe that's where some of the issues are taking place. I'm curious what your cloudflare tunnel config looks like.

Edit: Just now saw you are running Cloudflared in a container as well. I run mine locally, perhaps it is not able to reach the other containers? Are you able to get into that container and test connectivity? Might be something with your docker networking.