Preface

Software security is one of those subjects that often gets overlooked, both in academia and in professional projects, unless you're specifically working with some existing security-related element (e.g. you're taking a course on security basics, or updating your password hashing algorithm). As a result, we frequently see stories of rather catastrophic data leaks from otherwise reputable businesses, leaks which should have been entirely preventable with even the most basic of safeguards in place.

With that in mind, I thought I would switch things up and discuss something security-related this time.

Background

It's commonplace for complex software systems to avoid unnecessarily large expenses, especially in terms of technical debt and the capital involved in the initial development costs of building entire systems for e.g. geolocation or financial transactions. Instead of reinventing the wheel and effectively building a parallel business, we instead integrate with existing third-party systems, typically by using an API.

The problem, however, is that sometimes these third-party systems process requests over a long period of time, potentially on the order of minutes, hours, days, or even longer. If, for example, you have users who want to purchase something using your online platform, then it's not a particularly good idea to having potentially thousands of open connections to that third-party system all sitting there waiting multiple business days for funds to clear. That would just be stupid. So, how do we handle this in a way that isn't incredibly stupid?

There are two commonly accepted methods to avoid having to wait around:

1. We can periodically contact the third-party system and ask for the current status of a request, or
2. We can give the third-party system a way to contact us and let us know when they're finished with a request.

Both of these methods work, but obviously there will be a potentially significant delay in #1 between when a request finishes and when we know that it has finished (with a maximum delay of the wait time between status updates), whereas in #2 that delay is practically non-existent. Using #1 is also incredibly inefficient due to the number of wasted status update requests, whereas #2 allows us to avoid that kind of waste. Clearly #2 seems like the ideal option.

Method #2 is what we call a webhook.

May I see your ID?

The problem with webhooks is that when you're implementing one, it's far too easy to forget that you need to restrict access to it. After all, that third-party system isn't a user, right? They're not a human. They can't just give us a username and password like we want them to. They don't understand the specific requirements for our individual, custom-designed system.

But what happens if some malicious actor figures out what the webhook endpoint is? Let's say that all we do is log webhook requests somewhere in a non-capped file or database table/collection. Barring all other possible attack vectors, we suddenly find ourselves susceptible to that malicious actor sending us thousands, possibly millions of fraudulent data payloads in a small amount of time thanks to a botnet, and now our server's I/O utilization is spiking and the entire system is grinding to a halt--we're experiencing a DDoS!

We don't want just anyone to be able to talk to our webhook. We want to make sure that anyone who does is verified and trusted. But since we can't require a username and password, since we can't guarantee that the third-party system will even know how to make use of them, what can we do?

The answer is to use some form of token-based authentication--we generate a unique token, kind of like an ID card, and we attach it to our webhook endpoint (e.g. https://example.com/my_webhook/{unique_token}). We can then check that token for validity every time someone touches our webhook, ensuring that only someone we trust can get in.

Class is in Session

Just as there are two commonly accepted models for how to handle receiving updates from third-party systems, there are also two common models for how to assign a webhook to those systems:

1. Hard-coding the webhook in your account settings, or
2. Passing a webhook as part of request payload.

Model #1 is, in my experience, the most common of the two. In this model, our authentication token is typically directly linked to some user or user-like object in our system. This token is intended to be persisted and reused indefinitely, only scrapped in the event of a breach or a termination of integration with the service that uses it. Unfortunately, if the token is present within the URL, it's possible for your token to be viewed in plaintext in your logs.

In model #2, it's perfectly feasible to mirror the behavior of model #1 by simply passing the same webhook endpoint with the same token in every new request; however, there is a far better solution. We can, instead, generate a brand new token for each new request to the third-party system, and each new token can be associated with the request itself on our own system. Rather than only validating the token itself, we then validate that the token and the request it's supposed to be associated with are both valid. This ensures that even in the event of a breach, a leaked authentication token's extent of damage is limited only to the domain of the request it's associated with! In addition, we can automatically expire these tokens after receiving a certain number of requests, ensuring that a DDoS using a single valid token and request payload isn't possible. As with model #1, however, we still run into problems of token exposure if the token is present in the URL.

Model #2 treats each individual authentication token not as a session for an entire third-party system, but as a session for a single request on that system. These per-request session tokens require greater effort to implement, but are inherently safer due to the increased granularity of our authentication and our flexibility in allowing ourselves to expire the tokens at will.

Final Thoughts

Security is hard. Even with per-request session tokens, webhooks still aren't as secure as we might like them to be. Some systems allow us to define tokens that will be inserted into the request payload, but more often than not you'll find that only a webhook URL is possible to specify. Ideally we would stuff those tokens right into the POST request payload for all of our third-party systems so they would never be so easily exposed in plaintext in log files, but legacy systems tend to be slow to catch up and newer systems often don't have developers with the security background to consider it.

Still, as far as securing webhooks goes, having some sort of cryptographically secure authentication token is far better than leaving the door wide open for any script kiddie having a bad day to waltz right in and set the whole place on fire. If you're integrating with any third-party system, your job isn't to make it impossible for them to get their hands on a key, but to make it really difficult and to make sure you don't leave any gasoline lying around in case they do.

What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it.

Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 · Week #6 · Week #7 · Week #8 · Week #9 · Week #10

The study published in the Lancet: Food in the Anthropocene: the EAT–Lancet Commission on healthy diets from sustainable food systems

The editorial in the Lancet: The 21st-century great food transformation

An article in Cosmos for people (like me!) who don't have access to the Lancet: Feeding the planet: a call for radical action

Personally, I strongly dislike it. Every aspect of every film is way overblown there.

If there's a funny scene in a movie, they LITERALLY die laughing and wake their whole neighbourhood up.

If there's a scene that is in the slightest bit sad, they're going to cry their eyes out for months.

If there's a movie that's decently good, then it's an absolute masterpiece and the best movie of the decade.

And so on... Everything is always really exaggerated.

On top of that, there's always the circlejerk hivemind aspect. Threads are closed after 6 months, so the whole discussion about the film is divided between many threads, but because every thread is small and new, you often get the same fluff comments.

For more popular flims, it is the absolute worst. With half the thread being just funny quotes from the movie with no additional commentary or anything valuable, yet having thousands upon thousands of upvotes. It's kind of sad.

I used to go to IMDb boards, –which, admittedly, had their own issues– but they were still pretty useful for discussion. And shutting people up wasn't as easy as it is on Reddit, so the opinions there were much more varied. However, since they shut them down, Reddit is the closest thing I've found. Moviechat.org is supposed to be a replacement to the IMDb boards, but it's pretty inactive.

So, even though I kind of despise r/movies, I'm sort of forced to use them. But reading it makes me somewhat bitter.

What about you?

In lieu of the recent Gillette ad, and seeing as the conversation around it has stirred the pot quite a bit, I wanted to propose a conversation where we start from the very beginning:

Without yet talking about subsets, variants, or interpretations of masculinity/femininity (toxic or otherwise). How do you define it for yourself: what makes you masculine or feminine, or what parts of you would you describe as such, do you feel that those things go as universal descriptors or are they specific to your case?

There may also be some deeper questions in here about where you think you gained this conception (your family? your immediate circle of contacts? Role models?) or who you think best embodies your ideal definition of your gender.

So I have all my passwords, TOTP backup codes, and account recovery codes in my password manager (Bitwarden.) In turn, Bitwarden is secured with a master password and TOTP 2FA. I have a recovery code for the 2FA in the event that I can't get to andOTP anymore (2FA app.) The thing is, where do I put that code? I can't put it in a note app or anything, because if I'm locked out of Bitwarden, I don't have my passwords. Do you see my problem? I was thinking about physically writing it down, but that makes me nervous because I might lose it. Are there any good solutions to this problem?

Every year we try to put together a 'best of' list that is devoid of the same 200 albums that make up all of the other music lists on the internet. We do this by intentionally excluding popular/mainstream artists from our submission pool. Our cutoff is generally no more than three tracks with a million plays on spotify, though we do fudge it a bit especially for artists that only have regional success or put out something really great. The popularity limits are more of a guideline than a hard and fast rule.

I used last year's results as a test post on tildes, if you want to see what the final results will look like. Once we finish voting on the albums we generate the playlists on spotify and use automated tools to replicate them to other streaming services. We have a google spreadsheet that helps us manage the process. There's a submission form that will drop recommendations right into the spreadsheet where we can work on them.

We're not trying to find albums that are 'better' than the mainstream. We're just trying to include more of the great music that gets released every year, particularly the good stuff from new and overlooked artists that gets lost by the wayside while all the major music publications argue about the right order of the year's press darlings. Consider AlbumOfTheYear's List as the 'official' record of what's popular in the music press.

We do this by asking people for their favorites - in listentothis, in letstalkmusic, and today here on Tildes. I'd like to invite everyone on Tildes to submit their picks for the best overlooked music they've heard this year.

How do you know if an album you like is good enough? Everyone has their own way of listening, but generally, if you've had the record on repeat at all, that's the sign that it's worthy of attention. If you've got one you can't stop spinning and can't get out of your head, that's 'must listen' territory which goes at the top of the list. Make sure to leave a note in your comment when you submit, if you think it's that good. ;)

You can submit albums right here using this form. Please put a ~ in front of your username when you submit, so we know it came from a tildes user rather than a reddit user.

This spotify playlist contains the 175 albums from 2018 we've gathered so far. That's what's in our spreadsheet right now, at the start of the process. We haven't vetted/voted on these yet, just made sure they come in near the popularity cutoff. If any of the albums in this list really knock your socks off, let us know in the comments below. The first 1/3 of the list is mostly from random redditors, quality may be a bit dicey. The last 2/3 is the fruit of the l2t crew's cratedigging all year, submissions from our 30+ moderators. Every genre you can imagine (and some you can't) is in this list, they aren't sorted by style yet - this is one big bucket that goes all over the map.

I will update this playlist as new albums come in, though not exactly in real time - expect a day or two delay. If you want to keep up with it, just follow that list on spotify.

We're taking until the end of Feb. to finish the 2018 set, so roughly 6-8 weeks to listen to all of this stuff before we push the finished set out the door. If you want to help us listen, bookmark this thread here on Tildes, and leave a comment here when one of the albums grabs you. Feel free to submit new albums to that spreadsheet right up until the deadline at the end of Feb.

Happy listening. :)

I buy a bunch of books each year, but it seems less than half of them get read.

Sometimes I'm just satisfied by the notion that I will eventually read the book and never get around to it.
However, I do actually go back to some of these books. For instance I bought Gene Wolfe's Shadow of the Torturer/Sword of the Lictor back in 2015 but recently finished it back in summer of 2018.

Some have sat on the shelf for much longer.

Orson Scott Card's Xenocide comes to mind. I read Ender's Game back in 2013, read the sequel some time in 2014, and told myself I'd get around to the third book (Xenocide) but I never have.

Another one I picked up last year that I've been meaning to read is The Confessions of Saint Augustine.

So, what are some books you've been neglecting? Write them down here to put them to rest, or even better, to motivate yourself to actually read them!!

What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as well, we'd love to see your hauls :)

Feel free to give recs or discuss anything about each others' listening habits.

You can make a chart if you use last.fm:

http://www.tapmusic.net/lastfm/

Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

@Khromier, @Soulmaster and I are trying to learn Russian. It's the first time we've (I've?) properly tried learning a language.

I was wondering if any of you had advice to offer for learning a new language, even if it isn't necessarily specific to Russian.
We've been using https://russianlessons.net so far, but other resources would be interesting.

Спасибо!

Of all video games out there where a good guy beats a bad guy, which bad guy do you personally feel got the shittiest end of the stick? I recently beat Mother 3 and SPOILER Porky's fate seemed kinda fucked, what with him being permanently sealed away, unable to die, for all eternity in the Absolutely Safe Capsule.