Preface

Software security is one of those subjects that often gets overlooked, both in academia and in professional projects, unless you're specifically working with some existing security-related element (e.g. you're taking a course on security basics, or updating your password hashing algorithm). As a result, we frequently see stories of rather catastrophic data leaks from otherwise reputable businesses, leaks which should have been entirely preventable with even the most basic of safeguards in place.

With that in mind, I thought I would switch things up and discuss something security-related this time.

Background

It's commonplace for complex software systems to avoid unnecessarily large expenses, especially in terms of technical debt and the capital involved in the initial development costs of building entire systems for e.g. geolocation or financial transactions. Instead of reinventing the wheel and effectively building a parallel business, we instead integrate with existing third-party systems, typically by using an API.

The problem, however, is that sometimes these third-party systems process requests over a long period of time, potentially on the order of minutes, hours, days, or even longer. If, for example, you have users who want to purchase something using your online platform, then it's not a particularly good idea to having potentially thousands of open connections to that third-party system all sitting there waiting multiple business days for funds to clear. That would just be stupid. So, how do we handle this in a way that isn't incredibly stupid?

There are two commonly accepted methods to avoid having to wait around:

1. We can periodically contact the third-party system and ask for the current status of a request, or
2. We can give the third-party system a way to contact us and let us know when they're finished with a request.

Both of these methods work, but obviously there will be a potentially significant delay in #1 between when a request finishes and when we know that it has finished (with a maximum delay of the wait time between status updates), whereas in #2 that delay is practically non-existent. Using #1 is also incredibly inefficient due to the number of wasted status update requests, whereas #2 allows us to avoid that kind of waste. Clearly #2 seems like the ideal option.

Method #2 is what we call a webhook.

May I see your ID?

The problem with webhooks is that when you're implementing one, it's far too easy to forget that you need to restrict access to it. After all, that third-party system isn't a user, right? They're not a human. They can't just give us a username and password like we want them to. They don't understand the specific requirements for our individual, custom-designed system.

But what happens if some malicious actor figures out what the webhook endpoint is? Let's say that all we do is log webhook requests somewhere in a non-capped file or database table/collection. Barring all other possible attack vectors, we suddenly find ourselves susceptible to that malicious actor sending us thousands, possibly millions of fraudulent data payloads in a small amount of time thanks to a botnet, and now our server's I/O utilization is spiking and the entire system is grinding to a halt--we're experiencing a DDoS!

We don't want just anyone to be able to talk to our webhook. We want to make sure that anyone who does is verified and trusted. But since we can't require a username and password, since we can't guarantee that the third-party system will even know how to make use of them, what can we do?

The answer is to use some form of token-based authentication--we generate a unique token, kind of like an ID card, and we attach it to our webhook endpoint (e.g. https://example.com/my_webhook/{unique_token}). We can then check that token for validity every time someone touches our webhook, ensuring that only someone we trust can get in.

Class is in Session

Just as there are two commonly accepted models for how to handle receiving updates from third-party systems, there are also two common models for how to assign a webhook to those systems:

1. Hard-coding the webhook in your account settings, or
2. Passing a webhook as part of request payload.

Model #1 is, in my experience, the most common of the two. In this model, our authentication token is typically directly linked to some user or user-like object in our system. This token is intended to be persisted and reused indefinitely, only scrapped in the event of a breach or a termination of integration with the service that uses it. Unfortunately, if the token is present within the URL, it's possible for your token to be viewed in plaintext in your logs.

In model #2, it's perfectly feasible to mirror the behavior of model #1 by simply passing the same webhook endpoint with the same token in every new request; however, there is a far better solution. We can, instead, generate a brand new token for each new request to the third-party system, and each new token can be associated with the request itself on our own system. Rather than only validating the token itself, we then validate that the token and the request it's supposed to be associated with are both valid. This ensures that even in the event of a breach, a leaked authentication token's extent of damage is limited only to the domain of the request it's associated with! In addition, we can automatically expire these tokens after receiving a certain number of requests, ensuring that a DDoS using a single valid token and request payload isn't possible. As with model #1, however, we still run into problems of token exposure if the token is present in the URL.

Model #2 treats each individual authentication token not as a session for an entire third-party system, but as a session for a single request on that system. These per-request session tokens require greater effort to implement, but are inherently safer due to the increased granularity of our authentication and our flexibility in allowing ourselves to expire the tokens at will.

Final Thoughts

Security is hard. Even with per-request session tokens, webhooks still aren't as secure as we might like them to be. Some systems allow us to define tokens that will be inserted into the request payload, but more often than not you'll find that only a webhook URL is possible to specify. Ideally we would stuff those tokens right into the POST request payload for all of our third-party systems so they would never be so easily exposed in plaintext in log files, but legacy systems tend to be slow to catch up and newer systems often don't have developers with the security background to consider it.

Still, as far as securing webhooks goes, having some sort of cryptographically secure authentication token is far better than leaving the door wide open for any script kiddie having a bad day to waltz right in and set the whole place on fire. If you're integrating with any third-party system, your job isn't to make it impossible for them to get their hands on a key, but to make it really difficult and to make sure you don't leave any gasoline lying around in case they do.

I was thinking about the intersection of internet privacy and politics. You could even say I was having a bit of a mini-crisis. I like to think of myself as being pretty liberal, but I wondering how that fits into privacy. I was a little upset when I learned that Obama called Edward Snowden unpatriotic. I was kind of thinking that what he did was patriotic. Wasn't the NSA monitoring US citizens without warrants. That's morally wrong right? I think I would be pretty fine with the government monitoring someone if they had a warrant given to them by a non-secret court. I'm wondering if anyone here can give me some insight on this or if anyone else feels/has felt this way.

The study published in the Lancet: Food in the Anthropocene: the EAT–Lancet Commission on healthy diets from sustainable food systems

The editorial in the Lancet: The 21st-century great food transformation

An article in Cosmos for people (like me!) who don't have access to the Lancet: Feeding the planet: a call for radical action

Personally, I strongly dislike it. Every aspect of every film is way overblown there.

If there's a funny scene in a movie, they LITERALLY die laughing and wake their whole neighbourhood up.

If there's a scene that is in the slightest bit sad, they're going to cry their eyes out for months.

If there's a movie that's decently good, then it's an absolute masterpiece and the best movie of the decade.

And so on... Everything is always really exaggerated.

On top of that, there's always the circlejerk hivemind aspect. Threads are closed after 6 months, so the whole discussion about the film is divided between many threads, but because every thread is small and new, you often get the same fluff comments.

For more popular flims, it is the absolute worst. With half the thread being just funny quotes from the movie with no additional commentary or anything valuable, yet having thousands upon thousands of upvotes. It's kind of sad.

I used to go to IMDb boards, –which, admittedly, had their own issues– but they were still pretty useful for discussion. And shutting people up wasn't as easy as it is on Reddit, so the opinions there were much more varied. However, since they shut them down, Reddit is the closest thing I've found. Moviechat.org is supposed to be a replacement to the IMDb boards, but it's pretty inactive.

So, even though I kind of despise r/movies, I'm sort of forced to use them. But reading it makes me somewhat bitter.

What about you?

I was using Antergos for like 15 days and I really loved it! Not Antergos but Arch, I like how simple everything is in arch. Before installing antergos I tried to install arch on vm but failed. so i installed antergos with i3wm. somewhere i wanted to install vanilla arch.

Initially I was referring to the wiki with elinks and doing it carefully but failed. my setup was going to be arch + grub + luks, for some reason grub didn't show up while booting. i also encountered other error which made grub-mkconfig to hang. later i decided to drop luks so arch + grub, but again same error.

i've used debian family distros for a long time and grub was the most used bootloader so i wanted to install that. next i looked for a guide online and followed another guide which had same commands as arch wiki. again that failed.

after some more searching i found archfi, so basically it is a script that will ask me questions and install everything. again that grub thing failed so i went with systemd for the second time with this script. & voila!

later i used archdi to setup lightdm-gtk-greeter and installed i3wm.

i didn't install it myself but i am happy with my arch and probably someday would do it myself.

btw, i use arch

What's your favorite movie that features average-looking people? We all know the industry selects for youth and attractiveness mostly, but have any good movies risked casting only normal-looking people?

Edit: TV show suggestions are also welcome!

Every year we try to put together a 'best of' list that is devoid of the same 200 albums that make up all of the other music lists on the internet. We do this by intentionally excluding popular/mainstream artists from our submission pool. Our cutoff is generally no more than three tracks with a million plays on spotify, though we do fudge it a bit especially for artists that only have regional success or put out something really great. The popularity limits are more of a guideline than a hard and fast rule.

I used last year's results as a test post on tildes, if you want to see what the final results will look like. Once we finish voting on the albums we generate the playlists on spotify and use automated tools to replicate them to other streaming services. We have a google spreadsheet that helps us manage the process. There's a submission form that will drop recommendations right into the spreadsheet where we can work on them.

We're not trying to find albums that are 'better' than the mainstream. We're just trying to include more of the great music that gets released every year, particularly the good stuff from new and overlooked artists that gets lost by the wayside while all the major music publications argue about the right order of the year's press darlings. Consider AlbumOfTheYear's List as the 'official' record of what's popular in the music press.

We do this by asking people for their favorites - in listentothis, in letstalkmusic, and today here on Tildes. I'd like to invite everyone on Tildes to submit their picks for the best overlooked music they've heard this year.

How do you know if an album you like is good enough? Everyone has their own way of listening, but generally, if you've had the record on repeat at all, that's the sign that it's worthy of attention. If you've got one you can't stop spinning and can't get out of your head, that's 'must listen' territory which goes at the top of the list. Make sure to leave a note in your comment when you submit, if you think it's that good. ;)

You can submit albums right here using this form. Please put a ~ in front of your username when you submit, so we know it came from a tildes user rather than a reddit user.

This spotify playlist contains the 175 albums from 2018 we've gathered so far. That's what's in our spreadsheet right now, at the start of the process. We haven't vetted/voted on these yet, just made sure they come in near the popularity cutoff. If any of the albums in this list really knock your socks off, let us know in the comments below. The first 1/3 of the list is mostly from random redditors, quality may be a bit dicey. The last 2/3 is the fruit of the l2t crew's cratedigging all year, submissions from our 30+ moderators. Every genre you can imagine (and some you can't) is in this list, they aren't sorted by style yet - this is one big bucket that goes all over the map.

I will update this playlist as new albums come in, though not exactly in real time - expect a day or two delay. If you want to keep up with it, just follow that list on spotify.

We're taking until the end of Feb. to finish the 2018 set, so roughly 6-8 weeks to listen to all of this stuff before we push the finished set out the door. If you want to help us listen, bookmark this thread here on Tildes, and leave a comment here when one of the albums grabs you. Feel free to submit new albums to that spreadsheet right up until the deadline at the end of Feb.

Happy listening. :)

What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as well, we'd love to see your hauls :)

Feel free to give recs or discuss anything about each others' listening habits.

You can make a chart if you use last.fm:

http://www.tapmusic.net/lastfm/

Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

Anyone else play visual novels? If you don't know what they are, it's like an interactive story with pictures and sometimes player choice. People called Bandersnatch a kind of visual novel!

I got into them by playing Katawa Shoujo, which is astonishingly well-written and empathetic and funny. Recently I played Don't Forget Our Esports Dream, about two Starcraft pro gamers trying to chase their dreams while facing the realities of their lives. I love how it talks about the soul of esports, and if you ever played Starcraft, this game will bring back all the memories. What visual novels do other people recommend?

It's usually a question with a two word answer, but I'm sure there's more that could be said - that you wouldn't normally say because you don't want to waffle on.

Either way, I'm elated and excited. I revised a few hours for a test instead of watching a TV show, and my result went from shit to actually pretty bloody great. I'm really happy about it, but I'm not going to revise until GCSE's because it's made me absolutely knackered.

I'm pretty excited because I've got work experience next week. I haven't the foggiest about what it'll be like (fearing a very, very boring week), but I'm looking forward to it. It'll probably be a bit awkward without my friends around, but I'll get used to it.

So, onto you. How are you?