This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I'm mostly just curious what types of in-depth projects you all get up to. The weekly "what are you working on threads" are always full of cool things, so I am curious what people's self-rated most difficult or complex projects have been.

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I don't need answers so much as an idea of where to start.

Essentially, I have a Google Sheet that uses importjson.gs to pull from the following APIs

• OMDB (IMDB)
• TheMovieDB
• TVMaze

I also use another script to scrape Letterboxd for ratings.

This works well, but sometimes it'll time out or I'll hit urlFetch limits that Google has in place.

Basically, I'd like to have a text file (input.txt) where I pop in a bunch of titles and year or IMDB IDs, then the script runs and pulls set endpoints from all of these, outputting everything on one line (a pipe as a delimiter.)

My thinking is that I can then pull that info a sheet and run all of the formatting, basic math, and whatever else so it suits my Sheet.

I have a feeling I'll be using requests for the JSON and beautifulsoup for letterboxd -- or maybe a module.

Can anyone point me in the right direction? I don't think it'll be too difficult and should work well for a first python project.

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I'm vaguely in the "market" for a new computer algebra system (Mathematica, Maple, Sage, SymPy, etc etc) and I am curious what experience you all might have with these types of programs. In the past I've used Mathematica, and once you get over the odd aspects of the Language (and hefty license fee), it is a pretty great piece of software. Nonetheless, now that my old student license has expired and I would have to buy a new one (though through my school I can get it at a steep discount) I'm looking at some alternatives. Sage seems particularly appealing since it is open source and apparently well documented, and of course SymPy is powerful although that as stand-alone as these other choices.

There are tons of other options as well, with a wide range of functionality (it really is amazing how much these things can do). My main use cases with these types of things are either: a) algebraic manipulation, b) linear algebra, c) visualization. For these three it does really seem that Mathematica is at the forefront, but I'm all ears for other opinions.

I'm curious, if you have used any or all of these in the past, what were you using them for and what is your opinion?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I still mainly use Windows, although I've dual-booted Linux a few times and I have Linux Mint on an old laptop right now. One thing I've never understood about Linux is all the different distributions - their different reputations and why they have them. What is the mechanical difference between using one distribution of Linux and another? Or are the differences usually not mechanical?

For example, Ubuntu and Debian seem to be large families, meaning that a lot of other distributions are based on them (using packages built for them in their package managers at least) as well as being popular distros on their own. But what's different between the two of them, and between each and the other distros based on them? (and what's similar? I gather they all use the Linux kernel at least!)

I also know that people are quite opinionated on their choice of distro, I wondered what reasons people had for their choice. What things are easier or harder for you in your distro of choice? Is it mainly day-to-day tasks that are important or more how the OS works underneath? How much difference does your preferred distro make?

For myself, I've only used Kubuntu (though not much) and Linux Mint, which was mainly for UI reasons, and particularly for the latter, ease of use for someone used to Windows (at least that was what I found years ago when I first looked into it).

Though I doubt I'll ever fully move away from Windows I would like / need to have access to a Linux OS, so maybe this will help me to know what is important to look for. But I also hope it'll be a useful and interesting discussion topic. Also, there are some previous discussions on the latter question so I'd be more interested in learning about the main topic.

also, please do add more tags

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I see some people using NixOs on their servers. I would like to try it out to self host some services and learn about NixOs.

I use hetzner and they have an NixOs iso available so I can just use that to install NixOs. But how do people manage remote instances of NixOs? They would just use ansible or something like it, to run nix on the host, or is there a better way?

Thanks

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I thought it'd be fun to see how some of you keep your computer organized. Do you follow some general scheme, keep it arbitrary, throw everything on the Desktop or in Downloads or just abuse the search bar?

Feel free to go as general or as in-depth as you want.

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I've just read a journal from someone on another site saying that they wished there was a blacklisting system which stopped them seeing submissions from, to or about certain other users in their feed, as it is potentially trauma-inducing for them to log in and keep seeing them come up. I assume that the person looking for this blacklist isn't realistically able to just leave the site, because it's the most populous of its kind and serves as an art portfolio or source of income.

Is this, or something like it, actually possible to do with custom ad-blocker rules, or do they need to just wait for the site's admins to get around to it?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

The question mostly says it all, but I will give one of my ideas. I would replace the concept of a holistic app and instead replace it with a loose collection of features. The point would be to allow the end user to build a GUI from elements of multiple apps. You could have buttons that perform multiple functions in multiple apps. It would allow you to streamline and remove remove annoying elements; you could lock interface elements so you don't have to relearn the interface when the devs randomly decide to change it.

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I'm attempting to provision two mirror staging and production environments for a future SaaS application that we're close to launching as a company, and I'd like to get some feedback on the provisioning script I've created that takes a default VPS from our hosting provider, DigitalOcean, and readies it for being a secure hosting environment for our application instance (which runs inside Docker, and persists data to an unrelated managed database).

I'm sticking with a simple infrastructure architecture at the moment: A single VPS which runs both nginx and the application instance inside a containerised docker service as mentioned earlier. There's no load balancers or server duplication at this point. @Emerald_Knight very kindly provided me in the Tildes Discord with some overall guidance about what to aim for when configuring a server (limit damage as best as possible, limit access when an attack occurs)—so I've tried to be thoughtful and integrate that paradigm where possible (disabling root login, etc).

I’m not a DevOps or sysadmin-oriented person by trade—I stick to programming most of the time—but this role falls to me as the technical person in this business; so the last few days has been a lot of reading and readying. I’ll run through the provisioning flow step by step. Oh, and for reference, Ubuntu 20.04 LTS.

First step is self-explanatory.

#!/bin/sh

# Name of the user to create and grant privileges to.
USERNAME_OF_ACCOUNT=

sudo apt-get -qq update
sudo apt install -qq --yes nginx
sudo systemctl restart nginx

Next, create my sudo user, add them to the groups needed, require a password change on first login, then copy across any provided authorised keys from the root user which you can configure to be seeded to the VPS in the DigitalOcean management console.

useradd --create-home --shell "/bin/bash" --groups sudo,www-data "${USERNAME_OF_ACCOUNT}"
passwd --delete $USERNAME_OF_ACCOUNT
chage --lastday 0 $USERNAME_OF_ACCOUNT

HOME_DIR="$(eval echo ~${USERNAME_OF_ACCOUNT})"
mkdir --parents "${HOME_DIR}/.ssh"
cp /root/.ssh/authorized_keys "${HOME_DIR}/.ssh"

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown --recursive "${USERNAME_OF_ACCOUNT}":"${USERNAME_OF_ACCOUNT}" "${HOME_DIR}/.ssh"

sudo chmod 775 -R /var/www
sudo chown -R $USERNAME_OF_ACCOUNT /var/www
rm -rf /var/www/html

Installation of docker, and run it as a service, ensure the created user is added to the docker group.

sudo apt-get install -qq --yes \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88

sudo add-apt-repository --yes \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

sudo apt-get -qq update
sudo apt install -qq --yes docker-ce docker-ce-cli containerd.io

# Only add a group if it does not exist
sudo getent group docker || sudo groupadd docker
sudo usermod -aG docker $USERNAME_OF_ACCOUNT

# Enable docker
sudo systemctl enable docker

sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
docker-compose --version

Disable root logins and any form of password-based authentication by altering sshd_config.

sed -i '/^PermitRootLogin/s/yes/no/' /etc/ssh/sshd_config
sed -i '/^PasswordAuthentication/s/yes/no/' /etc/ssh/sshd_config
sed -i '/^ChallengeResponseAuthentication/s/yes/no/' /etc/ssh/sshd_config

Configure the firewall and fail2ban.

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https
sudo ufw reload
sudo ufw --force enable && sudo ufw status verbose

sudo apt-get -qq install --yes fail2ban
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Swapfiles.

sudo fallocate -l 1G /swapfile && ls -lh /swapfile
sudo chmod 0600 /swapfile && ls -lh /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile && sudo swapon --show
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Unattended updates, and restart the ssh daemon.

sudo apt install -qq unattended-upgrades
sudo systemctl restart ssh

Some questions

You can assume these questions are cost-benefit focused, i.e. is it worth my time to investigate this, versus something else that may have better gains given my limited time.

1. Obviously, any critiques of the above provisioning process are appreciated—both on the micro level of criticising particular lines, or zooming out and saying “well why don’t you do this instead…”. I can’t know what I don’t know.

2. Is it worth investigating tools such as ss or lynis (https://github.com/CISOfy/lynis) to perform server auditing? I don’t have to meet any compliance requirements at this point.

3. Do I get any meaningful increase in security by implementing 2FA on login here using google authenticator? As far as I can see, as long as I'm using best practices to actually ssh into our boxes, then the likeliest risk profile for unwanted access probably isn’t via the authentication mechanism I use personally to access my servers.

4. Am I missing anything here? Beyond the provisioning script itself, I adhere to best practices around storing and generating passwords and ssh keys.

Some notes and comments

1. Eventually I'll use the hosting provider's API to spin up and spin down VPS's on the fly via a custom management application, which gives me an opportunity to programmatically execute the provisioning script above and run some over pre- and post-provisioning things, like deployment of the application and so forth.

2. Usage alerts and monitoring is configured within DigitalOcean's console, and alerts are sent to our business' Slack for me to action as needed. Currently, I’m settling on the following alerts:
   1. Server CPU utilisation greater than 80% for 5 minutes.
   2. Server memory usage greater than 80% for 5 minutes.
   3. I’m also looking at setting up daily fail2ban status alerts if needed.

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

Hi! I've heard tilderinos talking about the gemini-verse on some other posts; I tried it out this evening and it honestly felt strange browsing in terminal and even stranger navigating the web without search engines. I was wondering if anyone had a gentler introduction than the official site? I feel like I've got a ship, but no map to this new verse.

Suppose I am trying to iteratively produce a completed image from some subset using a combination of convolutional/DNN methods. What Image norm is best?

The natural (for me) norm to ascribe to an image is to take the bitmap as a vector with L₂. If the input image is anime or something else, the uniform coloring makes this very likely to be a good fit in a low dimension - that is: no overfitting.

However: pictures of fur. Given a small square, the AI, set to extrapolate more fur from that single image, should be expected to get that stuff right next to the given subimage right, but further away, i want it to get the texture right, not the exact representation. So, if the AI shifts the fur far away from the image left by just the right amount, it could get an incredibly poor score.

If I were to use the naive L₂ norm directly, I would be guaranteed to overfit, and you can see this with some of the demo algorithms for image generation around the web. Now, the answer to this is probably to use a fourier or a wavelet transform and then take the L_N norm over the transformed space instead (correct me if I'm wrong.)

However, we get to the most complex class: images with different textures in them. In this case, I have a problem. Wavelet-type transforms don't behave well with discrete boundaries, while pixel-by-pixel methods don't do well with the textured parts of images. Is there a good method of determining image similarity for these cases?

More philosophically, what is the mathematical notion of similarity that our eye picks out? Any pointers or suggestions are appreciated. This is the last of two issues I have with a design I built for a Sparse NN.

Edit: For those interested, here is an example, notice how the predictions tend to blur details

Last Thursday, at my workplace, we rolled out a software upgrade across the company. The server side was upgraded overnight to ensure there was minimal downtime, and we had instructions for users posted on our Intranet (pinned to the top for the next 4 days), on exactly what they needed to do to run the upgrade on their PCs and ensure everything was working correctly.

The instructions were written with the help of my 4-year-old to ensure it was clear enough for anyone to read and follow along.

I still received at least 40 messages and emails from people complaining the upgrade didn't work or that certain Outlook plugins are now missing (which was covered in the instructions).

My question is, has anyone found a good way to ensure people follow instructions, or the best way to ensure that your instructions are easy to understand and follow along with?

It is very frustrating to take the time to ensure things go smoothly and write what even my 4-year-old thought was clear instruction, and still have a third of the company not be able to figure it out?

This is not meant to be mean hearted in any way, I genuinely would like some advice or tips on how I can improve on this the next time around.

Thanks.

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

Specifically, I've got a big honkin' pile of photos stored on an online storage space, and I want some kind of software that lets me share links to a pic, or a group of pics, or a folder, etc ...

... and here's the crux ... without making extra copies of the pics ... just has renamed, custom-permissioned links to the original pic(s).

In database terms, I want something that gives me Views of my photo collection.

I run my own Nextcloud instance, which is close. It has very nice, very granular photo management and sharing capabilities ... but as far as I can tell, whenever I share a pic with someone, it actually makes another copy of it for the shared instance. If I share the photo 5 times w/5 different people/groups, then suddenly, I have 6 copies of the pic.

Any recommendations?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I'm thinking of registering a new domain to host a personal blog and a few other pages that is somewhat separate from my current personal portfolio page, as the blog will likely not be directly relevant to my work. But since this is a more personal project, I want to branch out in terms of TLDs - I've previously used exclusively .com endings as they lead to the least confusion and are the most accepted.

I'm a little overwhelmed by the number of options these days, so I wanted to take a little straw poll to see which ones people think are the best. The name itself will be my initials "gpxl", and the endings I am considering are

• .info
• .xyz
• .fyi
• .computer
• .me

But I'm curious to see what others think. I would go with .xyz but I feel like then my domain name becomes an unintelligble string of letters - gpxl.xyz, so I'm wondering what others think and what arguments exist for choosing one over another.

P.S. If anyone has recommendations for registars I'm open to those too. Currently using google domains but open to switching.

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I've come across this website recently, and I'm genuinely wondering if it's really that useful for passing interviews.

For example, I think it can not replace a solid background in algorithms/data structures and maybe CS courses in general.

I could have post this on Reddit but in my experience, nothing really happens over there since things are too impersonal.

I realize that's not an enticing proposition for most people since programmers are usually busy people, but I figure I'd give it a shot. I believe many people that already interacted with me have some idea about my personality. Besides being a stickler for logic, I'm very flexible and eager to learn and make it a habit to force myself to admit when I am wrong. I had to abandon software engineering college because of financial concerns and this hurt me quite a bit, not just because of the content, but because of the social stimulus from my peers.

I already have some knowledge mostly in Python and basic C algorithms, from the basics until rudiments OOP. Math is not my strong suit, but I don't hate it either and I am very fond of logic (including philosophy). I'm also very good at Googling and reading documentation, so I wouldn't give you too much trouble.

I'm proficient in Linux/Unix (as an advanced user, not an administrator). I'm comfortable in the command line and an enthusiastic user of Emacs (but I can use whatever you want of course).

It doesn't even have to be Python, just anything you're comfortable teaching at a beginner level that works on a Mac (and later n a Linux Machine). And is also wouldn't have to be super intensive or frequent, being unemployed (hahaha) I can work around any schedule. I just need some human contact with a nice dude or dudette that cares about imparting knowledge.

My English is pretty good in writing and understanding but I do have a thick accent -- I believe it's understandable though.

I'm super shy and video is not a requirement, but it might be nice.

I'm also a slow learner and have ADHD, so you might need a little patience.

Anyway, here I am asking for help!

EDIT: dear Tilderinos, you're awesome and I love you all. I'm also aware of many if not most learning resources for self-didact beginners programmers on the internet -- especially if they use Python. The reason I'm posting this is that I am failing at learning by myself. I welcome ALL suggestions with great gratitude, but this is not the purpose of this thread.

EDIT2 VERY IMPORTANT!!! I’d be willing to teach Portuguese or anything else I happen to know in exchanging for the tutoring!!!!!!!

EDIT3:

Thank you very much everyone. I'm very sorry tor taking so long to answer. I had to make a decision about my study plans for 2020.

Thing is, back in 2019 I was doing pretty well on a software engineering course at a local private university. But money got tight and I had to drop out. Since then I've been trying to make it on my own, but after more than a year I came to the conclusion that this is simply not going to work. At the university, I was at the top of the class. By myself, I'm a lazy bastard with half the IQ. Go figure.

So I decided that instead of trying force my way into learning by myself, I'll just study for the Brazilian national exam and get into some great public educational facility that will most certainly provide me a better education for free.

I took the exam before without studying, and my grade made the cut. So I'm confident that if apply myself this time I'll be able to enter at least one of three major public universities in my city that offer dozens of IT-related courses.

So I'm laying off from programming until at least January 17 and devoting myself entirely to the exam.

Cheers!

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

I'm working on an application that allows a user to view playlists belonging to a particular radio show and stream/download/favourite the tracks in them. It has 4 core entities: User, Show, Playlist and Track.

• Each show has multiple playlists (one-to-many)
• Each playlist has multiple tracks (one-to-many)

To be able to reference a playlist belonging to a particular show. I gave those playlists the same uuid as the show they belong to. A few questions though.

1. Is this the right/best way to associate data?
2. As a track could potentially belong to multiple playlists, I can't take the same approach as I do for (show/playlist) How would be best to handle this? Ideally I would like to have a single "Track" table containing all tracks for all playlists.

For any experienced database designers out there, how would you structure this data? What would you consider in designing the schema and why? If I did go with 4 tables only, presumably there would be performance implications given the potential amount of data in any one of those tables, particularly tracks. If that is the case, how best to structure this kind of thing with performance in mind? Thanks in advance for any help :)

For reference, in case it's of importance, I'm using sqlite3.

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

Hey everyone,

I've been programming for some time now but notice without any formalized education in CS I often get lost in the weeds when it comes to developing larger applications. I'm familiar with the principles of TDD and SOLID - which have helped with maintainability - however still feel that I'm lacking in the ability to architect a properly structured system. As an example, I'm currently developing a flask REST API for a website (just for learning purposes). This involves parsing a html response and serializing the result as JSON. I'm still quite unclear as to structuring this sort of thing. If any more experienced developers could point me in the right direction/offer up their opinion I'd be very appreciative. Currently I have something like this (based - I hope correctly? - on uncle bob's clean architecture).

Firstly, I'm defining the domain model. i.e the structure of the API response. Then, from outside in.

1. Infrastructure (Flask): User makes request via interface (in my case a request to some endpoint)
2. Adapters: request object checks if the request is valid (on the way back it checks if the response is valid) - Is this layer only for error handling?
3. Repository: I'm struggling a bit here, AFAIUI this layer is traditionally a database. In my case however, where the request is valid, is this where I should handle the networking layer? i.e all the requests to return the website source? I'm also confused given at this stage I should be returning the relevant domain model, like an ORM, but as my data is unstructured, in order to do this I need to transform the response first. Where would it be best to handle this?
4. Use Cases: Here I transform the domain model depending on the request. For example, filter all objects by id. Have I understood this correctly?
5. Serializers: Encode the domain model as JSON to return from flask route.

If you got this far, thanks so much for reading. I really hope to hear the opinions of more experienced devs who can steer me in the right direction/correct me should I have misunderstood anything.

I've recently built a brainfuck just-in-time compiler and I'm looking for my next project. I think compiling a more complicated (and more easily written) language would build on what I've learned so far. Rather than design my own language from the ground up, I'd prefer to work with a toy language that already has existing programs and a spec. This would both save me some work and give me more solid ground to build upon.

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?