Been lurking for a week on tildes now and I am really glad this place exists. The crow here is exactly what I have been missing on Reddit for a while now.

Having said that, the whole Reddit situation has some-what motivated me to get the balls rolling on an idea that I have had for a while and I am looking for advice on the same.

I have often heard this phrase "Learn programming by building" but whenever I dive in to the resources, I fall flat due to the information overload and the general abstractness that the field has (I appreciate abstractness but here it demotivates me) and I have never found a proper resource that I could follow to actually build something instead of just blindly following tutorials and playing with them.

So, my question is how do I translate "learn by building a project" into a practical framework.

I know of 100 days of swift and I really like that approach however I don't think I want to start with swift or build an iOS app right now.

A common theme in web development, and the crux of the so-called "Web 2.0" is scrolling through dynamic lists of content. Tildes is such an example: you can scroll through about 50 topics on the front page before you reach a "next" button if you want to keep looking.

There's a certain beauty in the simplicity of the next/previous page. When done right it's fast, it's easy, and fits neatly into a server-side rendered model. However, it does cause that small bit of friction where you need to hit the next button to go forward -- taking you out of the "flow", so-to-speak. It's slick, but it could be slicker. Perhaps more importantly, it's an interesting problem to solve.

A step up from the next/previous button is to load the next page of content when you reach the end of the list, inserting it below. If the load is pretty fast, this will hardly interrupt your flow at all! The ever-so-popular reddit enhancement suite does precisely that for reddit: instead of a next button, when you reach the bottom, the next page of items simply plops into place. If the loading isn't fast enough, perhaps instead of loading when they reach the last item, you might choose to load when they hit the fifth from last item, etc.

To try to keep this post more concrete, and more helpful, here's how this type of pagination would work in practice, in typescript and using the Intersection Observer API but otherwise framework agnostic:

/**
 * Allows the user to scroll forever through the given list by calling the given loadMore()
 * function whenever the bottom element (by default) becomes visible. This assumes that
 * loadMore is the only thing that modifies the list, and that the list is done being modified
 * once the promise returned from loadMore resolves
 *
 * @param list The element which contains the individual items
 * @param loadMore A function which can be called to insert more items into the list. Can return
 *   a rejected promise to indicate that there are no more items to load
 * @param triggerLoadAt The index of the child in the list which triggers the load. Negative numbers
 *   are interpreted as offsets from the end of the list. 
 */
function handlePagination(list: Element, loadMore: () => Promise<void>, triggerLoadAt: number = -1) {
    manageIntersection();
    return;

    function handleIntersection(ele: Element, handler: () => void): () => void {
        let active = true;
        const observer = new IntersectionObserver((entries) => {
            if (active && entries[0].isIntersecting) {
                handler()
            }
        }, { root: null, threshold: 0.5 });
        observer.observe(ele);
        return () => {
            if (active) {
                active = false;
                observer.disconnect();
            }
        }
    }

    function manageIntersection() {
        const index = triggerLoadAt < 0 ? list.children.length + triggerLoadAt : triggerLoadAt;
        if (index < 0 || index >= list.children.length) {
            throw new Error(`index=${index} is not valid for a list of ${list.children.length} items`);
        }

        const child = list.children[index];
        const removeIntersectionHandler = handleIntersection(child, () => {
            removeIntersectionHandler();
            loadMore().then(() => {
                manageIntersection();
            }).catch((e) => {});
        });
    }
}

If you're sane, this probably suffices for you. However, there is still one problem: as you scroll,
the number of elements on the DOM get longer and longer. This means they necessarily take up
some amount of memory, and browsers probably have to do some amount of work to keep
track of them. Thus, in theory, if you were to scroll long enough, the page would get slower and
slower! How long "long enough" is would depend mostly on how complicated each item is: if each one
is a unique 20k element svg, it'll get slow pretty quickly.

The trick to avoid this, and to get a constant overhead, is that when adding new items below, remove the same number of items above! Of course, if the user scrolls back up they'll be expecting those items to be there, but no worries, the handlePagination from before works just as well for loading items before the first item.

However, this simple change is where a key problem arises: inserting elements below doesn't cause any layout shift, but inserting an item above ought to--right?

The answer is: it depends on the browser! Back in 2017 chrome realized that it's often convenient to be able to insert items into the dom above the viewport, and implemented scroll anchoring, which basically ensures that if you insert an item 50px tall above the viewport, then scroll 50px down so that there's no visual layout shift. Firefox followed suite in 2019, and edge got support in 2020. But alas, safari both on mac and ios does not support scroll anchoring (though they expressed interest in it since 2017)

Now, there's two responses to this:

• Surely Safari support is coming soon, they've posted on that bug as recently as April! Just use simpler pagination for now
• Pshhhh, just implement scroll anchoring ourself!

Of course, I've gone and done #2, and it almost perfectly works. Here's the idea:

• Right before loadMore, find the first item in the list which is inside the viewport. This is the item whose position we don't want to move. Use getBoundingClientRect to find it's top position.
• Perform the DOM manipulation as desired
• Use getBoundingClientRect again to find the new top of that item.
• Insert (or remove) the appropriate amount of blank space at the top of the list to offset the change in client rect (note that if there's scroll anchoring support in the browser this should always be zero, which means this effectively works as progressive enhancement)

Now, the function to do this is a tad too long for this post. I implemented it in React, however, and combined it with some stronger preloading object (we don't need all the items we've fetched from the API on the DOM, so we can use before, onTheDom, after lists to avoid getting a bunch of api requests just from scrolling down and up within the same small number of items).

What's interesting is that it still works perfectly on chrome even with scroll-anchoring disabled (via overflow-anchor: none), but on Safari there is still, sometimes, 1 frame where it renders the wrong scroll position before immediately adjusting. Because I implemented it in react, however, my current hypothesis is I have a mistake somewhere which causes the javascript to yield to the renderer before all the manipulations are done, and it only shows up on Safari because of the generally higher framerates there

If it's interesting to people, I could extract the infinite list component outside of this project: I certainly like it, and in my case I do expect people to want to quickly scroll through hundreds to thousands of items, so the lighter DOM feels worth it (though perhaps it wouldn't if I had known, when starting, how painful getting it to work on Safari would be!).

What do you think of this type of "true" infinite scrolling for web? Good thing, neutral thing, bad thing? Would you use it, if the component were available? Would you remove it, if you saw someone doing this? Are there other questions about how this was accomplished? Is this an appropriate post for Tildes?

I'm thinking about increasing the level of my open source contributions a bit. Instead of searching blindly until I stumble upon an issue that:
A) Piques my interest
B) I feel somewhat qualified to implement

I figured I'd check with the tildes community. Is there any Open Source software that you use that is missing a feature/capability? Can you give a brief description of it (bonus points for links to an issue tracker with an open ticket :))?

Can't of course promise anything will come of it, but if I do pick up your suggestion at least I'll give you a ping if I make any progress!

They ask for years of experience, skills that no Jr would know since, well, it is a Jr and the process to apply for jobs are surreal. Thousands of tests, interviews that goes nowhere and lots of ghosting. And the pay is not that good. No wonder after 2 or 3 years of experience a lot of develpers starts working for companies outside of Brazil.

Last one to contact me sent me a test to do it in 1 week. I went above and beyond and learned a lot of things. Before this, i had some small projects in Go and Python. Now i needed to learn Docker, tests, github actions, Postgresql and other things. Not everything was mandatory, but i did my best and did it all. I finished in 5 days since i have a day job.

Here is the result: https://github.com/crdpa/conservice

Showing the data in the browser was not necessary, but i think it was a nice touch and well made. If this does not land me a job as a junior developer i don't know what else could.

I'm glad i already have a job in another area, but me and my SO are separated by a 4 hour drive and i'm tired. I want to work from home to be near her and our dog. Paying rent in two places is becoming a burden.

I would be happy if you guys could test the application i made. It only needs docker.

And do you guys have any tips from now on?

This below is a summary of some real-world performance investigation I recently went through. The tools I used are installed on all linux systems, but I know some people don't know them and would straight up jump to heavyweight log analysis services and what not, or writing their own solution.

Let's say you have request log sampling in a bunch of log files that contain lines like these:

127.0.0.1 [2021-05-27 23:28:34.460] "GET /static/images/flags/2/54@3x.webp HTTP/2" 200 1806 TLSv1.3 HIT-CLUSTER SessionID:(null) Cache:max-age=31536000
127.0.0.1 [2021-05-27 23:51:22.019] "GET /pl/player/123456/changelog/ HTTP/1.1" 200 16524 TLSv1.2 MISS-CLUSTER SessionID:(null) Cache:

You might recognize Fastly logs there (IP anonymized). Now, there's a lot you might care about in this log file, but in my case, I wanted to get a breakdown of hits vs misses by URL.

So, first step, let's concatenate all the log files with cat *.log > all.txt, so we can work off a single file.

Then, let's split the file in two: hits and misses. There are a few different values for them, the majority are covered by either HIT-CLUSTER or MISS-CLUSTER. We can do this by just grepping for them like so:

grep HIT-CLUSTER all.txt > hits.txt; grep MISS-CLUSTER all.txt > misses.txt

However, we only care about url and whether it's a hit or a miss. So let's clean up those hits and misses with cut. The way cut works, it takes a delimiter (-d) and cuts the input based on that; you then give it a range of "fields" (-f) that you want.

In our case, if we cut based on spaces, we end up with for example: 127.0.0.1 [2021-05-27 23:28:34.460] "GET /static/images/flags/2/54@3x.webp HTTP/2" 200 1806 TLSv1.3 HIT-CLUSTER SessionID:(null) Cache:max-age=31536000.

We care about the 5th value only. So let's do: cut -d" " -f5 to get that. We will also sort the result, because future operations will require us to work on a sorted list of values.

cut -d" " -f5 hits.txt | sort > hits-sorted.txt; cut -d" " -f5 misses.txt | sort > misses-sorted.txt

Now we can start doing some neat stuff. wc (wordcount) is an awesome utility, it lets you count characters, words or lines very easily. wc -l counts lines in an input, since we're operating with one value per line we can easily count our hits and misses already:

$ wc -l hits-sorted.txt misses-sorted.txt
  132523 hits-sorted.txt
  220779 misses-sorted.txt
  353302 total

220779 / 132523 is a 1:1.66 ratio of hits to misses. That's not great…

Alright, now I'm also interested in how many unique URLs are hit versus missed. uniq tool deduplicates immediate sequences, so the input has to be sorted in order to deduplicate our entire file. We already did that. We can now count our urls with uniq < hits-sorted.txt | wc -l; uniq < misses-sorted.txt | wc -l. We get 49778 and 201178, respectively. It's to be expected that most of our cache misses would be in "rarer" urls; this gives us a 1:4 ratio of cached to uncached URL.

Let's say we want to dig down further into which URLs are most often hitting the cache, specifically. We can add -c to uniq in order to get a duplicate count in front of our URLs. To get the top ones at the top, we can then use sort, in reverse sort mode (-r), and it also needs to be numeric sort, not alphabetic (-n). head lets us get the top 10.

$ uniq -c < hits-sorted.txt | sort -nr | head
    815 /static/app/webfonts/fa-solid-900.woff2?d720146f1999
    793 /static/app/images/1.png
    786 /static/app/fonts/nunito-v9-latin-ext_latin-regular.woff2?d720146f1999
    760 /static/CACHE/js/output.cee5c4089626.js
    758 /static/images/crest/3/light/notfound.png
    757 /static/CACHE/css/output.4f2b59394c83.css
    756 /static/app/webfonts/fa-regular-400.woff2?d720146f1999
    754 /static/app/css/images/loading.gif?d720146f1999
    750 /static/app/css/images/prev.png?d720146f1999
    745 /static/app/css/images/next.png?d720146f1999

And same for misses:

$ uniq -c < misses-sorted.txt | sort -nr | head
     56 /
     14 /player/237678/
     13 /players/
     12 /teams/
     11 /players/top/
<snip>

So far this tells us static files are most often hit, and for misses it also tells us… something, but we can't quite track it down yet (and we won't, not in this post). We're not adjusting for how often the page is hit as a whole, this is still just high-level analysis.

One last thing I want to show you! Let's take everything we learned and analyze those URLs by prefix instead. We can cut our URLs again by slash with cut -d"/". If we want the first prefix, we can do -f1-2, or -f1-3 for the first two prefixes. Let's look!

cut -d'/' -f1-2 < hits-sorted.txt | uniq -c | sort -nr | head
 100189 /static
   5948 /es
   3069 /player
   2480 /fr
   2476 /es-mx
   2295 /pt-br
   2094 /tr
   1939 /it
   1692 /ru
   1626 /de

cut -d'/' -f1-2 < misses-sorted.txt | uniq -c | sort -nr | head
  66132 /static
  18578 /es
  17448 /player
  17064 /tr
  11379 /fr
   9624 /pt-br
   8730 /es-mx
   7993 /ru
   7689 /zh-hant
   7441 /it

This gives us hit-miss ratios by prefix. Neat, huh?

I'm attempting to provision two mirror staging and production environments for a future SaaS application that we're close to launching as a company, and I'd like to get some feedback on the provisioning script I've created that takes a default VPS from our hosting provider, DigitalOcean, and readies it for being a secure hosting environment for our application instance (which runs inside Docker, and persists data to an unrelated managed database).

I'm sticking with a simple infrastructure architecture at the moment: A single VPS which runs both nginx and the application instance inside a containerised docker service as mentioned earlier. There's no load balancers or server duplication at this point. @Emerald_Knight very kindly provided me in the Tildes Discord with some overall guidance about what to aim for when configuring a server (limit damage as best as possible, limit access when an attack occurs)—so I've tried to be thoughtful and integrate that paradigm where possible (disabling root login, etc).

I’m not a DevOps or sysadmin-oriented person by trade—I stick to programming most of the time—but this role falls to me as the technical person in this business; so the last few days has been a lot of reading and readying. I’ll run through the provisioning flow step by step. Oh, and for reference, Ubuntu 20.04 LTS.

First step is self-explanatory.

#!/bin/sh

# Name of the user to create and grant privileges to.
USERNAME_OF_ACCOUNT=

sudo apt-get -qq update
sudo apt install -qq --yes nginx
sudo systemctl restart nginx

Next, create my sudo user, add them to the groups needed, require a password change on first login, then copy across any provided authorised keys from the root user which you can configure to be seeded to the VPS in the DigitalOcean management console.

useradd --create-home --shell "/bin/bash" --groups sudo,www-data "${USERNAME_OF_ACCOUNT}"
passwd --delete $USERNAME_OF_ACCOUNT
chage --lastday 0 $USERNAME_OF_ACCOUNT

HOME_DIR="$(eval echo ~${USERNAME_OF_ACCOUNT})"
mkdir --parents "${HOME_DIR}/.ssh"
cp /root/.ssh/authorized_keys "${HOME_DIR}/.ssh"

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown --recursive "${USERNAME_OF_ACCOUNT}":"${USERNAME_OF_ACCOUNT}" "${HOME_DIR}/.ssh"

sudo chmod 775 -R /var/www
sudo chown -R $USERNAME_OF_ACCOUNT /var/www
rm -rf /var/www/html

Installation of docker, and run it as a service, ensure the created user is added to the docker group.

sudo apt-get install -qq --yes \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88

sudo add-apt-repository --yes \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

sudo apt-get -qq update
sudo apt install -qq --yes docker-ce docker-ce-cli containerd.io

# Only add a group if it does not exist
sudo getent group docker || sudo groupadd docker
sudo usermod -aG docker $USERNAME_OF_ACCOUNT

# Enable docker
sudo systemctl enable docker

sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
docker-compose --version

Disable root logins and any form of password-based authentication by altering sshd_config.

sed -i '/^PermitRootLogin/s/yes/no/' /etc/ssh/sshd_config
sed -i '/^PasswordAuthentication/s/yes/no/' /etc/ssh/sshd_config
sed -i '/^ChallengeResponseAuthentication/s/yes/no/' /etc/ssh/sshd_config

Configure the firewall and fail2ban.

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https
sudo ufw reload
sudo ufw --force enable && sudo ufw status verbose

sudo apt-get -qq install --yes fail2ban
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Swapfiles.

sudo fallocate -l 1G /swapfile && ls -lh /swapfile
sudo chmod 0600 /swapfile && ls -lh /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile && sudo swapon --show
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Unattended updates, and restart the ssh daemon.

sudo apt install -qq unattended-upgrades
sudo systemctl restart ssh

Some questions

You can assume these questions are cost-benefit focused, i.e. is it worth my time to investigate this, versus something else that may have better gains given my limited time.

1. Obviously, any critiques of the above provisioning process are appreciated—both on the micro level of criticising particular lines, or zooming out and saying “well why don’t you do this instead…”. I can’t know what I don’t know.

2. Is it worth investigating tools such as ss or lynis (https://github.com/CISOfy/lynis) to perform server auditing? I don’t have to meet any compliance requirements at this point.

3. Do I get any meaningful increase in security by implementing 2FA on login here using google authenticator? As far as I can see, as long as I'm using best practices to actually ssh into our boxes, then the likeliest risk profile for unwanted access probably isn’t via the authentication mechanism I use personally to access my servers.

4. Am I missing anything here? Beyond the provisioning script itself, I adhere to best practices around storing and generating passwords and ssh keys.

Some notes and comments

1. Eventually I'll use the hosting provider's API to spin up and spin down VPS's on the fly via a custom management application, which gives me an opportunity to programmatically execute the provisioning script above and run some over pre- and post-provisioning things, like deployment of the application and so forth.

2. Usage alerts and monitoring is configured within DigitalOcean's console, and alerts are sent to our business' Slack for me to action as needed. Currently, I’m settling on the following alerts:
   1. Server CPU utilisation greater than 80% for 5 minutes.
   2. Server memory usage greater than 80% for 5 minutes.
   3. I’m also looking at setting up daily fail2ban status alerts if needed.

Hello! After spending many development hours in my past years running on Virtualbox/Vagrant-style setups, I've decided to take the plunge into learning Docker, and after getting a few containers working, I'm now looking to figure out how to deploy this to production. I'm not a DevOps or infrastructure guy, my bread and butter is software, and although I've become significantly better at deploying & provisioning Linux VPS's, I'm still not entirely confident in my ability to deploy & manage such systems at scale and in production. But, I am now close to running my own business, so these requirements are suddenly going from "nice to have" to "critical".

As I mentioned, in the past when I've previously developed applications that have been pushed onto the web, I've tended to develop on my local machine, often with no specific configuration environment. If I did use an environment, it'd often be a Vagrant VM instance. From here, I'd push to GitHub, then from my VPS, pull down the changes, run any deployment scripts (recompile, restart nginx, etc), and I'm done.

I guess what I'm after with Docker is something that's more consistent between dev, testing, & prod, and is also more hands off in the deployment process. Yet, what I'm currently developing still does have differing configuration needs between dev and prod. For example, I'd like to use a hosted DB solution such as DigitalOcean Managed Databases in production, yet I'm totally fine using a Docker container for MySQL for local development. Is something like this possible? Does anyone have any recommendations around how to accomplish this, any do's and dont's, or any catches that are worth mentioning?

How about automating deployment from GitHub to production? I've never touched any CI/CD tools in my life, yet I know it's a hugely important part of the process when dealing with software in production, especially software that has clients dependent on it to function. Does anything specifically work well with Docker? Or GitHub? Ideally I want to be avoiding manual processes where I have to ssh in, and pull down the latest changes, half-remembering the commands I need to write to recompile and run the application again.

Hi,

I've been mulling ideas about a game for a while now, I'd like to hack out a prototype, and my default would be Love2D. (As an aside: one of the things I like about Love2D was that you could make a basic 'game' in a couple of LoC, and it was 'efficient enough' for what you got. Perhaps the only gripe I had with it was that it didn't output compiled binaries (I mean, you could make it do that, but it seemed like a hack). I think Polycode seemed to be a semi-serious contender, but last I checked (a year or two ago) it's pretty much as dead as a doornail. Some of the other alternatives I remember seeing (Godot? Unity?) felt too much like Blender.

So I've been wondering, it's been a while since I've been keeping tabs on the 'gamedev community', so I don't know if there have been any more recent development in that space.

So I guess my question is: What are people using for game jams nowadays? Preach to me (and everyone else) about your favorite framework and language :)

Pretty straight forward question, but basically I was watching a discussion panel the other day talking about the ethics of Self-Driving cars. A topic came up about people writing crappy code, and more than that, people not testing their code. And if they do, they do point testing. I am in my last semester of uni and I am working with some companies where we are doing pretty extensive testing, happy flows and a lot of alternate flows, as well as UI/UX testing. I wanted to extend this question to you, do you guys do testing, what type? How much do you focus on it? And if u love it/hate it?

Background

There's been some talk on ~ before, and it seems like there are quite a few people who are either interested in, learning, or working in game development, so I thought this could be a fun programming challenge.

This one is fairly open-ended: make a game in 1 hour. Any game, any engine, don't worry about art or sound or anything.

Doing is the best way to learn. Most people's first project is something overly ambitious, and when they find that it's more difficult than they thought, they can get discouraged, or even give up entirely. This is why the 1 hour limit is important: it forces you to finish something, even if it's small. When you're done, you can come out of it saying you made a game, and you learned from it.

Chances are the game might not be fun, look bad, be buggy, etc. But don't worry about that, everyone's game will have problems, and if you do create something really fun or innovative, congratulations, you have a prototype that you can expand on later!

"Rules"

Like I said before, these "rules" are pretty simple: make a game in (approximately) 1 hour. You can use any tools you want. If you use external assets (art, sound), it's probably best you use something you have the rights to (see resources). If you're completely new to game development/programming, your goal could even be to finish a tutorial.

If you're the kind of person who tends to get carried away with these things, you might want to post a comment saying you're starting, then another one once you've finished your game.

Please share your finished game, I'm sure everyone would love to try them! If your game is web-based, it can be hosted for free on Github Pages or Itch.io. If downloadable, it can be hosted for free on Google Drive, Mega, Dropbox, Itch.io, etc.

Resources

Engines

If you're a beginner, a good engine to start with is LÖVE. It's very simple, and uses Lua, which is very easy to learn.

If you're familiar with another language, you could use a library to make it in that language. Some examples:

C++: SFML, SDL, Allegro

Javascript: kontra, Phaser, pixi.js

Python: pygame

Rust: Piston, ggez, Amethyst

If you want something more complex, consider Godot, Unity, or Unreal.

You can also try something visual like Construct, Clickteam Fusion, or GDevelop

Art

For such a short time constraint, I'd suggest you use your own "programmer art": just use some basic shapes. Your primary focus should be gameplay.

If you think you have time to find something, try looking on OpenGameArt.

Sound

You can make simple sound effects very quickly with sfxr (or in this case, a web port of sfxr called jsfxr).

Hi everyone, I'm a frontend developer and do a bit of backend work as well. I'd really like to contribute some of my downtime to helping build this site. I've checked out the issue tracker on gitlab and some of the docs, tech goals and announcement, but I'd like to start getting my hands dirty and contribute some code.

• Is there a roadmap for development?
• A feature list to implement?
• Bugs to fix?

How can I help out? What's the best way to get started?

Also for all non-devs, what is the best way that they can start helping out?