• Activity
  • Votes
  • Comments
  • New
  • All activity
  • Showing only topics with the tag "recurring". Back to normal view
    1. Code Quality Tip: Cyclomatic complexity in depth.

      Preface Recently I briefly touched on the subject of cyclomatic complexity. This is an important concept for any programmer to understand and think about as they write their code. In order to...

      Preface

      Recently I briefly touched on the subject of cyclomatic complexity. This is an important concept for any programmer to understand and think about as they write their code. In order to provide a more solid understanding of the subject, however, I feel that I need to address the topic more thoroughly with a more practical example.


      What is cyclomatic complexity?

      The concept of "cyclomatic complexity" is simple: the more conditional branching and looping in your code, the more complex--and therefore the more difficult to maintain--that code is. We can visualize this complexity by drawing a diagram that illustrates the flow of logic in our program. For example, let's take the following toy example of a user login attempt:

      <?php
      
      $login_data = getLoginCredentialsFromInput();
      
      $login_succeeded = false;
      $error = '';
      if(usernameExists($login_data['username'])) {
          $user = getUser($login_data['username']);
          
          if(!isDeleted($user)) {
              if(!isBanned($user)) {
                  if(!loginRateLimitReached($user)) {
                      if(passwordMatches($user, $login_data['password'])) {
                          loginUser($user);
                          $login_succeeded = true;
                      } else {
                          $error = getBadPasswordError();
                          logBadLoginAttempt();
                      }
                  } else {
                      $error = getLoginRateLimitError($user);
                  }
              } else {
                  $error = getUserBannedError($user);
              }
          } else {
              $error = getUserDeletedError($user);
          }
      } else {
          $error = getBadUsernameError($login_data['username']);
      }
      
      if($login_succeeded) {
          sendSuccessResponse();
      } else {
          sendErrorResponse($error);
      }
      
      ?>
      

      A diagram for this logic might look something like this:

      +-----------------+
      |                 |
      |  Program Start  |
      |                 |
      +--------+--------+
               |
               |
               v
      +--------+--------+    +-----------------+
      |                 |    |                 |
      |    Username     +--->+    Set Error    +--+
      |    Exists?      | No |                 |  |
      |                 |    +-----------------+  |
      +--------+--------+                         |
               |                                  |
           Yes |                                  |
               v                                  |
      +--------+--------+    +-----------------+  |
      |                 |    |                 |  |
      |  User Deleted?  +--->+    Set Error    +->+
      |                 | Yes|                 |  |
      +--------+--------+    +-----------------+  |
               |                                  |
            No |                                  |
               v                                  |
      +--------+--------+    +-----------------+  |
      |                 |    |                 |  |
      |  User Banned?   +--->+    Set Error    +->+
      |                 | Yes|                 |  |
      +--------+--------+    +-----------------+  |
               |                                  |
            No |                                  |
               v                                  |
      +--------+--------+    +-----------------+  |
      |                 |    |                 |  |
      |   Login Rate    +--->+    Set Error    +->+
      | Limit Reached?  | Yes|                 |  |
      |                 |    +-----------------+  |
      +--------+--------+                         |
               |                                  |
            No |                                  |
               v                                  |
      +--------+--------+    +-----------------+  |
      |                 |    |                 |  |
      |Password Matches?+--->+    Set Error    +->+
      |                 | No |                 |  |
      +--------+--------+    +-----------------+  |
               |                                  |
           Yes |                                  |
               v                                  |
      +--------+--------+    +----------+         |
      |                 |    |          |         |
      |   Login User    +--->+ Converge +<--------+
      |                 |    |          |
      +-----------------+    +---+------+
                                 |
                                 |
               +-----------------+
               |
               v
      +--------+--------+
      |                 |
      |   Succeeded?    +-------------+
      |                 | No          |
      +--------+--------+             |
               |                      |
           Yes |                      |
               v                      v
      +--------+--------+    +--------+--------+
      |                 |    |                 |
      |  Send Success   |    |   Send Error    |
      |    Message      |    |    Message      |
      |                 |    |                 |
      +-----------------+    +-----------------+
      

      It's important to note that between nodes in this directed graph, you can find certain enclosed regions being formed. Specifically, each conditional branch that converges back into the main line of execution generates an additional region. The number of these distinct enclosed regions is directly proportional to the level of cyclomatic complexity of the system--that is, more regions means more complicated code.


      Clocking out early.

      There's an important piece of information I noted when describing the above example:

      . . . each conditional branch that converges back into the main line of execution generates an additional region.

      The above example is made complex largely due to an attempt to create a single exit point at the end of the program logic, causing these conditional branches to converge and thus generate the additional enclosed regions within our diagram.

      But what if we stopped trying to converge back into the main line of execution? What if, instead, we decided to interrupt the program execution as soon as we encountered an error? Our code might look something like this:

      <?php
      
      $login_data = getLoginCredentialsFromInput();
      
      if(!usernameExists($login_data['username'])) {
          sendErrorResponse(getBadUsernameError($login_data['username']));
          return;
      }
      
      $user = getUser($login_data['username']);
      if(isDeleted($user)) {
          sendErrorResponse(getUserDeletedError($user));
          return;
      }
      
      if(isBanned($user)) {
          sendErrorResponse(getUserBannedError($user));
          return;
      }
      
      if(loginRateLimitReached($user)) {
          logBadLoginAttempt($user);
          sendErrorResponse(getLoginRateLimitError($user));
          return;
      }
      
      if(!passwordMatches($user, $login_data['password'])) {
          logBadLoginAttempt($user);
          sendErrorResponse(getBadPasswordError());
          return;
      }
      
      loginUser($user);
      sendSuccessResponse();
      
      ?>
      

      Before we've even constructed a diagram for this logic, we can already see just how much simpler this logic is. We don't need to traverse a tree of if statements to determine which error message has priority to be sent out, we don't need to attempt to follow indentation levels, and our behavior on success is right at the very end and at the lowest level of indentation, where it's easily and obviously located at a glance.

      Now, however, let's verify this reduction in complexity by examining the associated diagram:

      +-----------------+
      |                 |
      |  Program Start  |
      |                 |
      +--------+--------+
               |
               |
               v
      +--------+--------+    +-----------------+
      |                 |    |                 |
      |    Username     +--->+   Send Error    |
      |    Exists?      | No |    Message      |
      |                 |    |                 |
      +--------+--------+    +-----------------+
               |
           Yes |
               v
      +--------+--------+    +-----------------+
      |                 |    |                 |
      |  User Deleted?  +--->+   Send Error    |
      |                 | Yes|    Message      |
      +--------+--------+    |                 |
               |             +-----------------+
            No |
               v
      +--------+--------+    +-----------------+
      |                 |    |                 |
      |  User Banned?   +--->+   Send Error    |
      |                 | Yes|    Message      |
      +--------+--------+    |                 |
               |             +-----------------+
            No |
               v
      +--------+--------+    +-----------------+
      |                 |    |                 |
      |   Login Rate    +--->+   Send Error    |
      | Limit Reached?  | Yes|    Message      |
      |                 |    |                 |
      +--------+--------+    +-----------------+
               |
            No |
               v
      +--------+--------+    +-----------------+
      |                 |    |                 |
      |Password Matches?+--->+   Send Error    |
      |                 | No |    Message      |
      +--------+--------+    |                 |
               |             +-----------------+
           Yes |
               v
      +--------+--------+
      |                 |
      |   Login User    |
      |                 |
      +--------+--------+
               |
               |
               v
      +--------+--------+
      |                 |
      |  Send Success   |
      |    Message      |
      |                 |
      +-----------------+
      

      Something should immediately stand out here: there are no enclosed regions in this diagram! Furthermore, even our new diagram is much simpler to follow than the old one was.


      Reality is rarely simple.

      The above is a really forgiving example. It has no loops, and loops are going to create enclosed regions that can't be broken apart so easily; it has no conditional branches that are so tightly coupled with the main path of execution that they can't be broken up; and the scope of functionality and side effects are minimal. Sometimes you can't break those regions up. So what do we do when we inevitably encounter these cases?

      High cyclomatic complexity in your program as a whole is inevitable for sufficiently large projects, especially in a production environment, and your efforts to reduce it can only go so far. In fact, I don't recommend trying to remove all or even most instances of cyclomatic complexity at all--instead, you should just be keeping the concept in mind to determine whether or not a function, method, class, module, or other component of your system is accumulating technical debt and therefore in need of refactoring.

      At this point, astute readers might ask, "How does refactoring help if the cyclomatic complexity doesn't actually go away?", and this is a valid concern. The answer to that is simple, however: we're hiding complexity behind abstractions.

      To test this, let's forget about cyclomatic complexity for a moment and instead focus on simplifying the refactored version of our toy example using abstraction:

      <?php
      
      function handleLoginAttempt($login_data) {
          if(!usernameExists($login_data['username'])) {
              sendErrorResponse(getBadUsernameError($login_data['username']));
              return;
          }
      
          $user = getUser($login_data['username']);
          if(isDeleted($user)) {
              sendErrorResponse(getUserDeletedError($user));
              return;
          }
      
          if(isBanned($user)) {
              sendErrorResponse(getUserBannedError($user));
              return;
          }
      
          if(loginRateLimitReached($user)) {
              logBadLoginAttempt($user);
              sendErrorResponse(getLoginRateLimitError($user));
              return;
          }
      
          if(!passwordMatches($user, $login_data['password'])) {
              logBadLoginAttempt($user);
              sendErrorResponse(getBadPasswordError());
              return;
          }
      
          loginUser($user);
          sendSuccessResponse();
      }
      
      $login_data = getLoginCredentialsFromInput();
      
      handleLoginAttempt($login_data);
      
      ?>
      

      The code above is functionally identical to our refactored example from earlier, but has an additional abstraction via a function. Now we can diagram this higher-level abstraction as follows:

      +-----------------+
      |                 |
      |  Program Start  |
      |                 |
      +--------+--------+
               |
               |
               v
      +--------+--------+
      |                 |
      |  Attempt Login  |
      |                 |
      +-----------------+
      

      This is, of course, a pretty extreme example, but this is how we handle thinking about complex program logic. We abstract it down to the barest basics so that we can visualize, in its simplest form, what the program is supposed to do. We don't actually care about the implementation unless we're digging into that specific part of the system, because otherwise we would be so bogged down by the details that we wouldn't be able to reason about what our program is supposed to do.

      Likewise, we can use these abstractions to hide away the cyclomatic complexity underlying different components of our software. This keeps everything clean and clutter-free in our head. And the more we do to keep our smaller components simple and easy to think about, the easier the larger components are to deal with, no matter how much cyclomatic complexity all of those components share as a collective.


      Final Thoughts

      Cyclomatic complexity isn't a bad thing to have in your code. The concept itself is only intended to be used as one of many tools to assess when your code is accumulating too much technical debt. It's a warning sign that you may need to change something, nothing more. But it's an incredibly useful tool to have available to you and you should get comfortable using it.

      As a general rule of thumb, you can usually just take a glance at your code and assess whether or not there's too much cyclomatic complexity in a component by looking for either of the following:

      • Too many loops and/or conditional statements nested within each other, i.e. you have a lot of indentation.
      • Many loops in the same function/method.

      It's not a perfect rule of thumb, but it's useful for at least 90% of your development needs, and there will inevitably be cases where you will prefer to accept some greater cyclomatic complexity because there is some benefit that makes it a better trade-off. Making that judgment is up to you as a developer.

      As always, I'm more than willing to listen to feedback and answer any questions!

      25 votes
    2. This Week's Releases 01/03 - Solange, Pond, 2-Chainz and more.

      Releases of the week 23/02/2019 - 01/03/2019 Featured Release Solange - When I Get Home (Neo-Soul, Alternative R&B) Solange has unveiled her new album, When I Get Home. Spanning 19 tracks, it...

      Releases of the week 23/02/2019 - 01/03/2019


      Featured Release

      Solange - When I Get Home (Neo-Soul, Alternative R&B)

      Solange has unveiled her new album, When I Get Home. Spanning 19 tracks, it marks Solange’s fourth album to date and serves as the follow-up to her 2016 opus, A Seat at the Table.
      When I Get Home was written, performed, and executive produced by Solange herself, but she was hardly in the studio. Among the album’s many contributors were Earl Sweatshirt, Panda Bear, Tyler the Creator, Blood Orange’s Dev Hynes, Sampha, Pharrell Williams, Gucci Mane, Playboi Carti, Raphael Saadiq, Metro Boomin, The-Dream, Cassie, Abra, and The Internet’s Steve Lacy.
      “Y’all! I’m filled w so much joy right now!!! Wow! I can’t thank y’all enough for this moment and for all the feelings i feel in my body!” Solange wrote in a tweet. “I’m bringing home w me everywhere I go yalll and I ain’t running from shit no more. Your love lifts me up so high. Thank you!”

      Source: Consequence of Sound

      Listen to single

      Stream

      Other Notable Relases

      Pond - Tasmania (Neo-Psychedelia, Psychedelic Pop)

      Listen to single
      Stream

      2 Chainz - Rap or Go to the League (Trap Rap, Southern Hip Hop)

      Listen to single
      Stream

      Sun Kil Moon - I Also Want To Die In New Orleans (Contemporary Folk, Spoken Word)

      Listen to single
      Stream

      T-Pain - 1UP (Alternative R&B, Trap Rap)

      Listen to single
      Stream

      Feel free to discuss or feature any and all other releases in the comments below

      Discussion Points

      Have you listened to any of these releases?
      What are your thoughts?
      What are you looking forward to listen to?
      What have you enjoyed from these artists in the past?

      // All feedback on this format welcome.

      7 votes
    3. what creative projects are you working on?

      this seems like a good time to bring back this question and maybe make it more consistent and recurring since there's just been an influx of new people. i last asked this about three months ago...

      this seems like a good time to bring back this question and maybe make it more consistent and recurring since there's just been an influx of new people. i last asked this about three months ago and i'm sure there are both new people to answer this question and new ideas that people who already answered or would answer have come up with since.


      for my part, i did this post just now as a short little thing. on the larger scale, i've been intending to get back into editing my personal worldbuilding wiki because there's a bunch of shit i want to do with that, but college isn't exactly leaving a lot of time for it and every time i try to start on stuff gets tedious so i've been holding off on it for a little bit. i've also been chipping away at the fun that will be one of several religious books, but i don't really know how i want to structure it yet so the verses pictured and others are liable to get shuffled around at this point.

      33 votes
    4. What have you been watching/reading this week? (Anime/Manga)

      What have you been watching and reading this week? You don't need to give us a whole essay if you don't want to, but please write something! Feel free to talk about something you saw that was...

      What have you been watching and reading this week? You don't need to give us a whole essay if you don't want to, but please write something! Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its pages on Anilist, MAL, or any other database you use!

      15 votes
    5. What are you reading these days? #13

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it. Notes: I could not start the thread yesterday on Friday like...

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it.

      Notes: I could not start the thread yesterday on Friday like I used to, I'm sorry for the delay.

      Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 · Week #6 · Week #7 · Week #8 · Week #9 · Week #10 · Week #11 · Week #12

      23 votes
    6. What have you been listening to this week?

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as...

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as well, we'd love to see your hauls :)

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      31 votes
    7. What have you been watching/reading this week?

      I'm gonna move this to Fridays from now on, it's easier for me to remember to post. Anyway, what have you been watching/reading this week? Feel free to talk about something you saw that was cool,...

      I'm gonna move this to Fridays from now on, it's easier for me to remember to post.

      Anyway, what have you been watching/reading this week?

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      10 votes
    8. What have you been watching/reading this week? (Anime/Manga)

      Picking these threads up from @Cleb after talking to her about it :) What have you been watching and reading this week? You don't need to give us a whole essay if you don't want to, but please...

      Picking these threads up from @Cleb after talking to her about it :)

      What have you been watching and reading this week? You don't need to give us a whole essay if you don't want to, but please write something! Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its pages on Anilist, MAL, or any other database you use!

      20 votes
    9. What have you been listening to this week?

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as...

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as well, we'd love to see your hauls :)

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      17 votes
    10. What are you reading these days? #12

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it. Edit 2019-01-16: Add the link for Week #11 below. Past weeks:...

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it.

      Edit 2019-01-16: Add the link for Week #11 below.

      Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 · Week #6 · Week #7 · Week #8 · Week #9 · Week #10 · Week #11

      11 votes
    11. What have you been watching/reading this week? (Anime/Manga)

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of. If you want to, feel free to find the thing you're...

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      11 votes
    12. What have you been watching/reading this week? (Anime/Manga)

      Forgot to post the thread yesterday. Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of. If you want to,...

      Forgot to post the thread yesterday.

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      6 votes
    13. What have you been listening to this week?

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as...

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as well, we'd love to see your hauls :)

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      10 votes
    14. What have you been listening to this week?

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as...

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as well, we'd love to see your hauls :)

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      15 votes
    15. What are you reading these days? #11

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it. Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 ·...

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it.

      Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 · Week #6 · Week #7 · Week #8 · Week #9 · Week #10

      14 votes
    16. What have you been watching/reading this week? (Anime/Manga)

      I'd say something here about thinking a monthly thread for this is better or something, but that would be a lie and the truth is that I was busy during Christmas and subsequently forgot to post...

      I'd say something here about thinking a monthly thread for this is better or something, but that would be a lie and the truth is that I was busy during Christmas and subsequently forgot to post this thread for about three weeks running. Sorry about that.

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      9 votes
    17. This Week's Releases 18/01 - James Blake, Sharon Von Etten, Future and more.

      Releases of the week 12/01/2019 - 18/01/2019 Featured Release James Blake - Assume Form (Alternative R&B, Art Pop) "After dropping two new songs yesterday — the subtle trapper “Mile High” and the...

      Releases of the week 12/01/2019 - 18/01/2019


      Featured Release

      James Blake - Assume Form (Alternative R&B, Art Pop)

      "After dropping two new songs yesterday — the subtle trapper “Mile High” and the haunting “Lullaby For My Insomniac” — James Blake reveals today the entirety of Assume Form, his fourth full-length to date and follow-up to 2016’s The Colour In Anything. It’s available to stream in full below via Apple Music and Spotify.
      The album spans 12 tracks, including last year’s excellent “Don’t Miss It”, and features guest spots from Travis Scott, Metro Boomin, Rosalía, André 3000, and Moses Sumney. Blake previously contributed to the most recent full-length efforts from Oneothrix Point Never and Travis Scott, as well as on singles from Kendrick Lamar, André 3000, and Moses Sumney.”

      Source: Consequence of Sound

      Listen to single

      Stream

      Other Notable Relases

      Sharon Van Etten - Remind Me Tomorrow (Indie Pop, Synth)

      Listen to single
      Stream

      Future - The WIZRD (Hip Hop, Trap)

      Listen to single
      Stream

      Deerhunter - Why Hasn’t Everything Already Disappeared (Neo-Psychedelia, Indie Rock)

      Listen to single
      Stream

      Toro Y Moi - Outer Peace (Alternative R&B, Synth Funk)

      Listen to single
      Stream

      Feel free to discuss or feature any and all other releases in the comments below

      Discussion Points

      Have you listened to any of these releases?
      What are your thoughts?
      What are you looking forward to listen to?
      What have you enjoyed from these artists in the past?

      // All feedback on this format welcome below.

      4 votes
    18. A Brief Look at Webhook Security

      Preface Software security is one of those subjects that often gets overlooked, both in academia and in professional projects, unless you're specifically working with some existing security-related...

      Preface

      Software security is one of those subjects that often gets overlooked, both in academia and in professional projects, unless you're specifically working with some existing security-related element (e.g. you're taking a course on security basics, or updating your password hashing algorithm). As a result, we frequently see stories of rather catastrophic data leaks from otherwise reputable businesses, leaks which should have been entirely preventable with even the most basic of safeguards in place.

      With that in mind, I thought I would switch things up and discuss something security-related this time.


      Background

      It's commonplace for complex software systems to avoid unnecessarily large expenses, especially in terms of technical debt and the capital involved in the initial development costs of building entire systems for e.g. geolocation or financial transactions. Instead of reinventing the wheel and effectively building a parallel business, we instead integrate with existing third-party systems, typically by using an API.

      The problem, however, is that sometimes these third-party systems process requests over a long period of time, potentially on the order of minutes, hours, days, or even longer. If, for example, you have users who want to purchase something using your online platform, then it's not a particularly good idea to having potentially thousands of open connections to that third-party system all sitting there waiting multiple business days for funds to clear. That would just be stupid. So, how do we handle this in a way that isn't incredibly stupid?

      There are two commonly accepted methods to avoid having to wait around:

      1. We can periodically contact the third-party system and ask for the current status of a request, or
      2. We can give the third-party system a way to contact us and let us know when they're finished with a request.

      Both of these methods work, but obviously there will be a potentially significant delay in #1 between when a request finishes and when we know that it has finished (with a maximum delay of the wait time between status updates), whereas in #2 that delay is practically non-existent. Using #1 is also incredibly inefficient due to the number of wasted status update requests, whereas #2 allows us to avoid that kind of waste. Clearly #2 seems like the ideal option.

      Method #2 is what we call a webhook.


      May I see your ID?

      The problem with webhooks is that when you're implementing one, it's far too easy to forget that you need to restrict access to it. After all, that third-party system isn't a user, right? They're not a human. They can't just give us a username and password like we want them to. They don't understand the specific requirements for our individual, custom-designed system.

      But what happens if some malicious actor figures out what the webhook endpoint is? Let's say that all we do is log webhook requests somewhere in a non-capped file or database table/collection. Barring all other possible attack vectors, we suddenly find ourselves susceptible to that malicious actor sending us thousands, possibly millions of fraudulent data payloads in a small amount of time thanks to a botnet, and now our server's I/O utilization is spiking and the entire system is grinding to a halt--we're experiencing a DDoS!

      We don't want just anyone to be able to talk to our webhook. We want to make sure that anyone who does is verified and trusted. But since we can't require a username and password, since we can't guarantee that the third-party system will even know how to make use of them, what can we do?

      The answer is to use some form of token-based authentication--we generate a unique token, kind of like an ID card, and we attach it to our webhook endpoint (e.g. https://example.com/my_webhook/{unique_token}). We can then check that token for validity every time someone touches our webhook, ensuring that only someone we trust can get in.


      Class is in Session

      Just as there are two commonly accepted models for how to handle receiving updates from third-party systems, there are also two common models for how to assign a webhook to those systems:

      1. Hard-coding the webhook in your account settings, or
      2. Passing a webhook as part of request payload.

      Model #1 is, in my experience, the most common of the two. In this model, our authentication token is typically directly linked to some user or user-like object in our system. This token is intended to be persisted and reused indefinitely, only scrapped in the event of a breach or a termination of integration with the service that uses it. Unfortunately, if the token is present within the URL, it's possible for your token to be viewed in plaintext in your logs.

      In model #2, it's perfectly feasible to mirror the behavior of model #1 by simply passing the same webhook endpoint with the same token in every new request; however, there is a far better solution. We can, instead, generate a brand new token for each new request to the third-party system, and each new token can be associated with the request itself on our own system. Rather than only validating the token itself, we then validate that the token and the request it's supposed to be associated with are both valid. This ensures that even in the event of a breach, a leaked authentication token's extent of damage is limited only to the domain of the request it's associated with! In addition, we can automatically expire these tokens after receiving a certain number of requests, ensuring that a DDoS using a single valid token and request payload isn't possible. As with model #1, however, we still run into problems of token exposure if the token is present in the URL.

      Model #2 treats each individual authentication token not as a session for an entire third-party system, but as a session for a single request on that system. These per-request session tokens require greater effort to implement, but are inherently safer due to the increased granularity of our authentication and our flexibility in allowing ourselves to expire the tokens at will.


      Final Thoughts

      Security is hard. Even with per-request session tokens, webhooks still aren't as secure as we might like them to be. Some systems allow us to define tokens that will be inserted into the request payload, but more often than not you'll find that only a webhook URL is possible to specify. Ideally we would stuff those tokens right into the POST request payload for all of our third-party systems so they would never be so easily exposed in plaintext in log files, but legacy systems tend to be slow to catch up and newer systems often don't have developers with the security background to consider it.

      Still, as far as securing webhooks goes, having some sort of cryptographically secure authentication token is far better than leaving the door wide open for any script kiddie having a bad day to waltz right in and set the whole place on fire. If you're integrating with any third-party system, your job isn't to make it impossible for them to get their hands on a key, but to make it really difficult and to make sure you don't leave any gasoline lying around in case they do.

      8 votes
    19. What have you been listening to this week?

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as...

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! If you've just picked up some music, please update on that as well, we'd love to see your hauls :)

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      7 votes
    20. What have you been listening to this week?

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! Feel free to give recs or discuss anything about each others'...

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      16 votes
    21. What are you reading these days? #10

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it. Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 ·...

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it.

      Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 · Week #6 · Week #7 · Week #8 · Week #9

      14 votes
    22. What have you been listening to this week?

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! Feel free to give recs or discuss anything about each others'...

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      11 votes
    23. What have you been listening to this week?

      Hi, I'm covering this week at Whom's request again. What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! Feel free...

      Hi, I'm covering this week at Whom's request again.

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      13 votes
    24. What are you reading these days? #9

      Edit: #9, not #8; sorry for messing the title up. Would be glad if someone can fix it for me, I can't edit it apparently. Thanks a lot, Deimos, for fixing it up! What are you reading currently?...

      Edit: #9, not #8; sorry for messing the title up. Would be glad if someone can fix it for me, I can't edit it apparently. Thanks a lot, Deimos, for fixing it up!

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it.

      Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 · Week #6 · Week #7 · Week #8

      12 votes
    25. What have you been watching/reading this week? (Anime/Manga)

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of. If you want to, feel free to find the thing you're...

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      10 votes
    26. ~music Listening Club 24 - Intro to Shoegaze

      So, it's time to switch things up a little as discussed in the last thread. We're going to have multiple records a week from now on and follow themes rather than the alternating weeks we've had...

      So, it's time to switch things up a little as discussed in the last thread. We're going to have multiple records a week from now on and follow themes rather than the alternating weeks we've had until now.

      The first week of our new format is the beginning of a genre introduction series that I'm running: Intro to Shoegaze! We will likely have several different themes switching off each week, and I'll kick off a few over the next couple weeks. These aren't intended to be tailored for genreheads (at least not right now), but rather a way to have a conversation about the all-time greats of specific genres, scenes, labels, etc. while also having something fresh that those already into that kind of music might not already be familiar with.

      Shoegaze

      Shoegazing (or shoegaze, initially known as "dream pop") is a subgenre of indie and alternative rock that emerged in the United Kingdom in the late 1980s. It is characterised by its ethereal-sounding mixture of obscured vocals, guitar distortion and effects, feedback, and overwhelming volume. The term "shoegazing" was coined by the British music press to ridicule the stage presence of a wave of neo-psychedelic groups who stood still during live performances in a detached, introspective, non-confrontational state with their heads down. This was because the heavy use of effects pedals meant the performers were often looking down at the readouts on their effects pedals during concerts.

      Most shoegazing bands drew from the glide guitar template set by My Bloody Valentine on their early EPs and 1988 debut Isn't Anything. A loose label given to the shoegazing scene and other affiliated bands in London in the early 1990s was The Scene That Celebrates Itself. In the early 1990s, shoegazing groups were pushed aside by the American grunge movement and early Britpop acts such as Suede, forcing the relatively unknown bands to break up or reinvent their style altogether. In the 2000s, there was renewed interest in the genre among "nu gaze" bands.

      Essential Album: My Bloody Valentine - Loveless - Listen to it!

      The shoegaze genre has a pretty much univerally agreed-upon "Big 3" records, the most prominent and influential of which being My Bloody Valentine's Loveless, which is undoubtedly the #1 starting place if you're looking to check out shoegaze for yourself. It's the darling of the genre and of internet music nerdom in general for a reason.

      Loveless is the second studio album by English-Irish rock band My Bloody Valentine. It was released on 4 November 1991 in the United Kingdom by Creation Records and in the United States by Sire Records. The album was recorded over a two-year period between 1989 and 1991, with vocalist and guitarist Kevin Shields leading the recording sessions and experimenting with guitar tremolo techniques and tuning systems, samplers, and meticulous production methods. The band cycled through nineteen different studios and many engineers during the album's prolonged recording, with its production cost rumoured to have reached £250,000.

      Preceded by the EPs Glider (1990) and Tremolo (1991), Loveless peaked at number 24 on the UK Albums Chart and was widely praised by critics for its sonic innovations and Shields's "virtual reinvention of the guitar". However, after its release, Creation Records owner Alan McGee removed the band from the label, as he found Shields too difficult to work with, a factor alleged to have contributed to the label's eventual bankruptcy. My Bloody Valentine struggled to record a follow-up to the album and broke up in 1997, and Loveless was their last full-length release until MBV in 2013.

      Since its release, Loveless has been widely cited by critics as one the greatest albums of the 1990s, a landmark work of the shoegazing subgenre, and as a significant influence on various subsequent artists. In 2012, it was reissued as a two-CD set, including remastered tracks and a previously unreleased half-inch analogue tape version, and peaked on several international charts. In 2013, Loveless was certified silver by the British Phonographic Industry.

      Minor Album: Vyva Melinkolya - Vyva Melinkolya - Listen to it!

      This one is a shoegaze record which lies a little more on the dream pop / ethereal side of the genre than Loveless, and is my personal album of the year for 2018 so far by a longshot. I would suggest this for anyone who wants to to live in that wonderful dreamy yet noisy middle ground. The artist is also nonbinary, and while I can't speak for everyone, I find that this speaks to gender dysphoria in a way that only the best shoegaze can.

      Here's the place to discuss your thoughts on the records, your history with them or the artists, and basically talk about whatever you want to that goes along with Loveless and Vyva Melinkolya! Remember that this is intended to be a slow moving thing, feel free to take your time and comment at any point in the week!

      Again, if you'd like to stream or buy the albums, they can be found on most platforms here (Loveless) and here (Vyva Melinkolya).

      12 votes
    27. This Week's Releases 14/12 - Charlotte Gainsburg, Kodak Black, Vic Mensa and more.

      Releases of the week 8/12/2018 - 14/12/2018 Featured Release Charlotte Gainsburg - Take 2 - EP (Art Pop, Synth Pop) "Charlotte Gainsbourg has announced a new EP called Take 2. The five-track...

      Releases of the week 8/12/2018 - 14/12/2018


      Featured Release

      Charlotte Gainsburg - Take 2 - EP (Art Pop, Synth Pop)

      "Charlotte Gainsbourg has announced a new EP called Take 2. The five-track project, which features her cover of Kanye West’s “Runaway,” is out December 14 via Because Music. Today, she’s shared the EP’s first song “Such a Remarkable Day.” Check it out below with a video of Gainsbourg’s live performances.
      Take 2 is produced by Frank Ocean collaborator SebastiAn and mixed by Tom Elmhirst. See the artwork below. Last year, Gainsbourg released Rest, which featured on Pitchfork’s “50 Best Albums of 2017.”

      Source: Pitchfork

      Listen to single

      Stream

      Other Notable Relases

      Kodak Black - Dying To Live (Hip Hop, Trap)

      Listen to single
      Stream

      Vic Mensa - Hooligans EP (Conscious Hip Hop)

      Listen to single
      Stream

      $ilkMoney - *I Hate My Life and I Really Wish People Would Stop Telling Me Not To * (Hip Hop, Trap)

      Listen
      Stream

      Feel free to discuss or feature any and all other releases in the comments below

      Discussion Points

      Have you listened to any of these releases?
      What are your thoughts?
      What are you looking forward to listen to?
      What have you enjoyed from these artists in the past?

      // All feedback on this format welcome below.

      4 votes
    28. What have you been listening to this week?

      Whom couldn't make it today so here I am with the weekly post. What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!...

      Whom couldn't make it today so here I am with the weekly post.

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      10 votes
    29. ~music Listening Club 25 - 1992

      On the second week of the new format, we're going to try out themes based on periods of time, this time being the year of 1992. Themes, both years like this one or whatever else you think would...

      On the second week of the new format, we're going to try out themes based on periods of time, this time being the year of 1992. Themes, both years like this one or whatever else you think would work well for the listening club, can be suggested in this thread or if you join the unofficial Tildes Discord server in the #listening-club channel.

      Essential Album: Aphex Twin - Selected Ambient Works 85-92 - Listen!

      While a lot of excellent and long-lasting work was released in 1992, Selected Ambient Works 85-92 stands out as one of the most influential and has stood the test of time as well as any of them. As much as everyone hates the label of "Intelligent Dance Music" (IDM), there's no doubt that SAW is one of the cornerstone records of that genre that has an impact in the world of music at large that can't be ignored.

      Selected Ambient Works 85–92 is the debut studio album by the English electronic musician Richard D. James under the pseudonym of Aphex Twin, released as a very limited import in late November 1992 by Apollo Records, an imprint of the more prominent label R&S Records, and later widely in February 1993. The 1992 LP was James' third release overall, and collected tracks dating back as early as 1985. An analogue remaster was released in 2006, and a digital remaster in 2008.

      Selected Ambient Works 85–92 received widespread acclaim and has been characterised as a landmark of electronica, ambient music, and IDM. It was followed by Selected Ambient Works Volume II (1994). On the week ending 27 September 2014, the album entered at #30 in the UK Dance Albums Chart after the release of his 2014 album Syro.

      Widely regarded by critics as one of the pioneering works in early IDM and modern electronic music, retrospective reviews mention its influence on electronic artists. Warp Records refers to it as "the birthplace and the benchmark of modern electronic music" and has stated that "every home should have a copy." In 2003, the album was placed #92 in "NME's 100 Best Albums" poll. Nine years later, it was named the greatest album of the 1990s by FACT Magazine. The album was also featured in the book 1001 Albums You Must Hear Before You Die.

      Minor Album: Divine Styler - Spiral Walls Containing Autumns of Light - Listen!

      This is a strange one, possibly the strangest hip hop album released as of 1992. It's so disjointed and all over the place that instead of describing it myself, I'll steal the descriptors from RateYourMusic: "Islamic, psychedelic, anxious, cryptic, conscious, abstract, introspective, surreal, eclectic, male vocals, avant-garde, noisy, spiritual."

      Spiral Walls Containing Autumns of Light is the second album by hip hop artist Divine Styler, released in 1992 on Giant Records. This album marked a significant change in Divine Styler's musical direction, incorporating a much broader range of styles and influences than 1989's Word Power. Despite the album's unusual style, it was actually released by a major label but, perhaps inevitably, failed to sell as well as record executives expected. Although the album was largely regarded as a commercial failure, it has since developed a large cult following. Styler was signed to Giant because of Ice-T’s relationship with Warner Bros. Records.

      The album is notoriously experimental—Allmusic referred to the album as being like '"The Residents meets Funkadelic"—and takes influence from a wide variety of music genres such as hip hop, rock, electronic, funk and even elements of spoken word and noise.

      The majority of the album was produced and arranged by Divine Styler himself, which included him performing and processing all of the vocals, playing the guitar, drums (and drum programming), and keyboards, among other instruments. Due to the artist's dominance over the album's direction, it is generally regarded as a very introspective and personal album.

      Here's the place to discuss your thoughts on the records, your history with them, the artists, or music in 1992 in general and basically talk about whatever you want to that goes along with Selected Ambient Works 85-92 and Spiral Walls Containing Autumns of Light! Remember that this is intended to be a slow moving thing, feel free to take your time and comment at any point in the week!

      Again, if you'd like to stream or buy the albums, they can be found on most platforms here (Selected Ambient Works 85-92) and here (Spiral Walls Containing Autumns of Light).

      4 votes
    30. What have you been watching/reading this week? (Anime/Manga)

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of. If you want to, feel free to find the thing you're...

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      6 votes
    31. What are you reading these days? #8

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it. Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 ·...

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it.

      Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 · Week #6 · Week #7

      17 votes
    32. What have you been listening to this week?

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! Feel free to give recs or discuss anything about each others'...

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      12 votes
    33. What have you been watching/reading this week? (Anime/Manga)

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of. If you want to, feel free to find the thing you're...

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      8 votes
    34. This Week's Releases 30/11 - Earl Sweatshirt, The 1975, J.I.D., Jeff Tweedy

      Releases of the week 24/11/2018 - 30/11/2018 Featured Release Earl Sweatshirt - Some Rap Songs (Hip Hop, Abstract) It’s always been Earl versus the world. Fame found him at the age of 16, making...

      Releases of the week 24/11/2018 - 30/11/2018


      Featured Release

      Earl Sweatshirt - Some Rap Songs (Hip Hop, Abstract)

      It’s always been Earl versus the world. Fame found him at the age of 16, making him an internet sensation, then a meme, then an enigma, and finally, an icon. For an introverted kid who knew he could rap but was reluctant to accept the exposure and invasions of privacy that came with being a bona fide pop culture phenomenon, it’s been an uncomfortable evolution. Voracious fans threatened to consume not just his music but his personal life too. That same entitlement caused the “FREE EARL” campaign to mutate from eager appreciation to scary obsession and stoked fans’ demand for music during the three years since his last album—even as he was mourning his father’s death earlier this year. Rather than bask in the attention, he recoiled from it, setting himself apart from peers who maintain relevance through carefully strategized ubiquity. As he receded from the spotlight, his mystique grew—as did fans’ desire to hear him to do what he does best.

      Source: Pitchfork

      Listen to single

      Stream Links

      Other Notable Relases

      The 1975 - A Brief Inquiry Into Online Relationships (Pop, Synthpop)

      Listen to single

      J.I.D. - DiCaprio 2 (Hip Hop, Trap)

      Listen to single

      Jeff Tweedy - Warm (Indie, Alt-Country)

      Listen to single

      Meek Mill - CHAMPIONSHIPS (Hip Hop, Trap)

      Listen to single

      Feel free to discuss or feature any and all other releases in the comments below

      Discussion Points

      Have you listened to any of these releases?
      What are your thoughts?
      What are you looking forward to listen to?
      What have you enjoyed from these artists in the past?

      // All feedback on this format welcome below.

      6 votes
    35. What have you been watching/reading this week? (Anime/Manga)

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of. If you want to, feel free to find the thing you're...

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      8 votes
    36. What have you been listening to this week?

      Oops, just noticed I didn't post this week! My mistake. What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! Feel...

      Oops, just noticed I didn't post this week! My mistake.

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      11 votes
    37. ~music Weekly Music Tracks Thread 1 - Uplifting Earworms

      Some people have mentioned they'd like to have some sort of weekly track-sharing thread, so let's have a little fun and find some good music in the process. Everybody's got that playlist somewhere...

      Some people have mentioned they'd like to have some sort of weekly track-sharing thread, so let's have a little fun and find some good music in the process.

      Everybody's got that playlist somewhere with all of your favorite earworms - the songs you put on repeat to the point where you annoy the hell out of everyone else in the room because you love them so much. Let's collect some of those earworms here and see what we can come up with.

      In particular, let's go for the uplifting kind - feel good music. When the thread settles down I'll pull these all together in a nice playlist and share that here as a separate link submission.

      Any time period, any genre, any style, popular, obscure, or even your own music, it's all good - just as long as it's positive energy and you can't stop spinning it. If you're on mobile, don't worry about making it into links, others can linkify it for you (and eventually, Tildes can do that automatically to make this all easier in the future). Share as many as you've got. If you've already got a playlist like this for yourself, you can share that too. ;)

      Oh, and don't worry about nebulous 'standards' or if people will like it. If you like it, that's all that matters. Don't overthink it!

      Edit: Almost forgot, feel free to make suggestions for the topics of upcoming share threads in the next few weeks!

      13 votes
    38. An Alternative Approach to Configuration Management

      Preface Different projects have different use cases that can ultimately result in common solutions not suiting your particular needs. Today I'm going to diverging a bit from my more abstract,...

      Preface

      Different projects have different use cases that can ultimately result in common solutions not suiting your particular needs. Today I'm going to diverging a bit from my more abstract, generalized topics on code quality and instead focus on a specific project structure example that I encountered.


      Background

      For a while now, I've found myself being continually frustrated with the state of my project configuration management. I had a single configuration file that would contain all of the configuration options for the various tools I've been using--database, API credentials, etc.--and I kept running into the problem of wanting to test these tools locally while not inadvertently committing and pushing sensitive credentials upstream. For me, part of my security process is ensuring that sensitive access credentials never make it into the repository and to limit access to these credentials to only people who need to be able to access them.


      Monolithic Files Cause Monolithic Pain

      The first thing I realized was that having a single monolithic configuration file was just terrible practice. There are going to be common configuration options that I want to have in there with default values, such as local database configuration pointing to a database instance running on the same VM as the application. These should always be in the repo, otherwise any dev who spins up an instance of the VM will need to manually tread documentation and copy-paste the missing options into the configuration. This would be incredibly time-consuming, inefficient, and stupid.

      I also use different tools which have different configuration options associated with them. Having to dig through a single file containing configuration options for all of these tools to find the ones I need to modify is cumbersome at best. On top of that, having those common configuration options living in the same place that sensitive access credentials do is just asking for a rogue git commit -A to violate the aforementioned security protocol.


      Same Problem, Different Structure

      My first approach to resolving this problem was breaking the configuration out into separate files, one for each distinct tool. In each file, a "skeleton" config was generated, i.e. each option was given a default empty value. The main config would then only contain config options that are common and shared across the application. To avoid having the sensitive credentials leaked, I then created rules in the .gitignore to exclude these files.

      This is where I ran into problem #2. I learned that this just doesn't work. You can either have a file in your repo and have all changes to that file tracked, have the file in your repo and make a local-only change to prevent changes from being tracked, or leave the file out of the repo completely. In my use case, I wanted to be able to leave the file in the repo, treat it as ignored by everyone, and only commit changes to that file when there was a new configuration option I wanted added to it. Git doesn't support this use case whatsoever.

      This problem turned out to be really common, but the solution suggested is to have two separate versions of your configuration--one for dev, and one for production--and to have a flag to switch between the two. Given the breaking up of my configuration, I would then need twice as many files to do this, and given my security practices, this would violate the no-upstream rule for sensitive credentials. Worse still, if I had several different kinds of environments with different configuration--local dev, staging, beta, production--then for m such environments and n configuration files, I would need to maintain n*m separate files for configuration alone. Finally, I would need to remember to include a prefix or postfix to each file name any time I needed to retrieve values from a new config file, which is itself an error-prone requirement. Overall, there would be a substantial increase in technical debt. In other words, this approach would not only not help, it would make matters worse!


      Borrowing From Linux

      After a lot of thought, an idea occurred to me: within Linux systems, there's an /etc/skel/ directory that contains common files that are copied into a new user's home directory when that user is created, e.g. .bashrc and .profile. You can make changes to these files and have them propagate to new users, or you can modify your own personal copy and leave all other new users unaffected. This sounds exactly like the kind of behavior I want to emulate!

      Following their example, I took my $APPHOME/config/ directory and placed a skel/ subdirectory inside, which then contained all of the config files with the empty default values within. My .gitignore then looked something like this:

      $APPHOME/config/*
      !$APPHOME/config/main.php
      !$APPHOME/config/skel/
      !$APPHOME/config/skel/*
      # This last one might not be necessary, but I don't care enough to test it without.
      

      Finally, on deploying my local environment, I simply include a snippet in my script that enters the new skel/ directory and copies any files inside into config/, as long as it doesn't already exist:

      cd $APPHOME/config/skel/
      for filename in *; do
          if [ ! -f "$APPHOME/config/$filename" ]; then
              cp "$filename" "$APPHOME/config/$filename"
          fi
      done
      

      (Note: production environments have a slightly different deployment procedure, as local copies of these config files are saved within a shared directory for all releases to point to via symlink.)

      All of these changes ensure that only config/main.php and the files contained within config/skel/ are whitelisted, while all others are ignored, i.e. our local copies that get stored within config/ won't be inadvertently committed and pushed upstream!


      Final Thoughts

      Common solutions to problems are typically common for a good reason. They're tested, proven, and predictable. But sometimes you find yourself running into cases where the common, well-accepted solution to the problem doesn't work for you. Standards exist to solve a certain class of problems, and sometimes your problem is just different enough for it to matter and for those standards to not apply. Standards are created to address most cases, but edge cases will always exist. In other words, standards are guidelines, not concrete rules.

      Sometimes you need to stop thinking about the problem in terms of the standard approach to solving it, and instead break it down into its most abstract, basic form and look for parallels in other solved problems for inspiration. Odds are the problem you're trying to solve isn't as novel as you think it is, and that someone has probably already solved a similar problem before. Parallels, in my experience, are usually a pretty good indicator that you're on the right track.

      More importantly, there's a delicate line to tread between needing to use a different approach to solving an edge case problem you have, and needing to restructure your project to eliminate the edge case and allow the standard solution to work. Being able to decide which is more appropriate can have long-lasting repercussions on your ability to manage technical debt.

      16 votes
    39. What have you been listening to this week?

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! Feel free to give recs or discuss anything about each others'...

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      8 votes
    40. This Week's Releases 23/11 - Oneohtrix Point Never, Dipset, Rita Ora

      Releases of the week 17/11/2018 - 23/11/2018 Featured Release Oneohtrix Point Never - Love In The Time Of Lexapro (EP) (Prog Electronic, Ambient) Oneohtrix Point Never has dropped off his new EP,...

      Releases of the week 17/11/2018 - 23/11/2018


      Featured Release

      Oneohtrix Point Never - Love In The Time Of Lexapro (EP) (Prog Electronic, Ambient)

      Oneohtrix Point Never has dropped off his new EP, Love in the Time of Lexapro....
      The new effort follows producer Daniel Lopatin’s impressive 2018 album, Age Of, and The Station EP from July. It contains a pair of new songs and alternate, collaborative versions of two Age Of tracks.
      The two previously unreleased originals are titled, “Thank God I’m a Country Girl” and “Love in the Time of Lexapro”. The latter title track has been an audience favorite on Lopatin’s Age Of tour, but has never been properly recorded and released until now. Also included is a rework of Age Of’s “Last Known Image Of A Song” from veteran Japanese experimental musician and composer Ryuichi Sakamoto (The Revenant), as well as an updated take on Age Of highlight “Babylon” featuring additional contributions from (Sandy) Alex G.”

      Source: Consequence of Sound

      Stream Links

      Other Notable Relases

      Dipset - Diplomatic Ties (Trap, Hip Hop)

      Listen to single

      Rita Ora - Let You Love Me (Pop, Electropop)

      Listen to single

      Boosie Badass - Boosie Blues Cafe (Hip Hop, Blues)

      Listen

      Art Brut - Wham! Bang! Pow! Let’s Rock Out! (Indie, Rock)

      Listen to single

      Feel free to discuss or feature any and all other releases in the comments below

      Discussion Points

      Have you listened to any of these releases yet?
      What are your thoughts?
      What are you looking forward to listen to?
      What have you enjoyed from these artists in the past?

      // All feedback on this format welcome below.

      4 votes
    41. What are you reading these days? #7

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it. Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 ·...

      What are you reading currently? Fiction or non-fiction, any genre, any language! Tell us what you're reading, and talk a bit about it.

      Past weeks: Week #1 · Week #2 · Week #3 · Week #4 · Week #5 · Week #6

      11 votes
    42. What have you been watching/reading this week? (Anime/Manga)

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of. If you want to, feel free to find the thing you're...

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      7 votes
    43. ~music Listening Club 23 - Dark Side of the Moon

      Welcome to the 23rd week, it had to come eventually...another classic record discussion: The Dark Side of the Moon by Pink Floyd! Posting a little early since I'll be busy celebrating...

      Welcome to the 23rd week, it had to come eventually...another classic record discussion: The Dark Side of the Moon by Pink Floyd! Posting a little early since I'll be busy celebrating Thanksgiving, have a great day everyone.

      The Dark Side of the Moon is the eighth studio album by English rock band Pink Floyd, released on 1 March 1973 by Harvest Records. It built on ideas explored in Pink Floyd's earlier recordings and performances, but without the extended instrumentals that characterised their earlier work. Its themes explore conflict, greed, time, and mental illness, the latter partly inspired by the deteriorating health of founding member Syd Barrett, who left in 1968.

      Developed during live performances, Pink Floyd premiered an early version of The Dark Side of the Moon several months before recording began. New material was recorded in two sessions in 1972 and 1973 at Abbey Road Studios in London. The group used advanced recording techniques at the time, including multitrack recording and tape loops; analogue synthesizers are prominent, and snippets from interviews with Pink Floyd's road crew and others provide philosophical quotations. Engineer Alan Parsons was responsible for many sonic aspects and the recruitment of singer Clare Torry, who appears on one track. The iconic sleeve was designed by Storm Thorgerson; following keyboardist Richard Wright's request for a "simple and bold" design, it depicts a prism spectrum, representing the band's lighting and the record's themes.

      The Dark Side of the Moon produced two singles: "Money" and "Us and Them". The album topped the Billboard chart for a week, and remained on the chart for 741 weeks from 1973 to 1988. Following a change in how Billboard counts sales in 2009, it re-entered the chart and has since appeared for over 900 weeks. With estimated sales of over 45 million, it is Pink Floyd's bestselling album and one of the bestselling worldwide. It has been remastered and rereleased several times, and covered in its entirety by several acts. It is regarded as one of the greatest albums of all time.

      Here's the place to discuss your thoughts on the record, your history with it or the artist, and basically talk about whatever you want to that goes along with The Dark Side of the Moon! Remember that this is intended to be a slow moving thing, feel free to take your time and comment at any point in the week!

      If you'd like to stream or buy the album, it can be found on most platforms here.

      Rather than get an obscure record this week, I'd love to hear from you in responses to this comment!

      9 votes
    44. What have you been listening to this week?

      Filling in again this week at Whom's request. What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! Feel free to give...

      Filling in again this week at Whom's request.

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      12 votes
    45. ~music Listening Club 22 - Survival

      Welcome to week 22! Here we've got this week's user-voted record: Survival by Bob Marley & The Wailers! Taken from @koan's pitch: Does Bob Marley have gold and platinum records? Definitely....

      Welcome to week 22! Here we've got this week's user-voted record: Survival by Bob Marley & The Wailers!

      Taken from @koan's pitch:

      Does Bob Marley have gold and platinum records? Definitely. Survival is not one of them, but in my opinion it is by far his greatest album -- hands down, no competition. Everybody has an opinion about Bob Marley, whether you've actually given him a shot or not. Reggae can be polarizing. Some love it, some think it's corny. But Survival is not corny. It's Marley's greatest roots reggae record.

      While some Bob Marley songs might make you want to relax on a beach and sip cold cocktails, the songs on Survival make you want to get up and do something about shit. When I was absolutely stewing in dissatisfaction with my corporate job, listening to Survival on my commute in the morning inspired me to change my life. Be careful, because listening to it too much might turn you into a revolutionary.

      If you're unfamiliar with reggae in general, or you think it's silly, give this record a chance. It might change your perspective about a very deep and varied genre of music.

      Here's the place to discuss your thoughts on the record, your history with it or the artist, and basically talk about whatever you want to that goes along with Survival. Remember that this is intended to be a slow moving thing, feel free to take your time and comment at any point in the week!

      If you'd like to stream or buy the album, it can be found on most platforms here.

      7 votes
    46. What have you been listening to this week?

      Filling in again this week. What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something! Feel free to give recs or discuss...

      Filling in again this week.

      What have you been listening to this week? You don't need to do a 6000 word review if you don't want to, but please write something!

      Feel free to give recs or discuss anything about each others' listening habits.

      You can make a chart if you use last.fm:

      http://www.tapmusic.net/lastfm/

      Remember that linking directly to your image will update with your future listening, make sure to reupload to somewhere like imgur if you'd like it to remain what you have at the time of posting.

      14 votes
    47. This Week's Releases 16/11 - Anderson .Paak, The Smashing Pumpkins, Imagine Dragons

      Releases of the week 10/11/2018 - 16/11/2018 Featured Release Anderson .Paak - Oxnard (R&B, Hip Hop) Oxnard marks .Paak’s first release on Dre’s Aftermath Entertainment and last of “his beach...

      Releases of the week 10/11/2018 - 16/11/2018


      Featured Release

      Anderson .Paak - Oxnard (R&B, Hip Hop)

      Oxnard marks .Paak’s first release on Dre’s Aftermath Entertainment and last of “his beach series.” “You know, we went to Venice, we went to Malibu,” .Paak noted, “so it’s only right that we take it to the next place, up the coast, up to the next beach.” As hinted previously, Dre was “heavily” involved in the making of the LP, serving as executive producer. “His music was everything to me,” Paak said of his mentor. “It molded me.”
      Another famous name that pops up on Oxnard is Madlib, a veteran rapper and producer who is also known for his collaborations with DOOM, J Dilla, and Freddie Gibbs.
      The new album features “sprawling psychedelic grooves and confident verses,” according to Rolling Stone, and per .Paak, a special ingredient missing from the current musical landscape. “I feel like ambition is missing from today’s music,” he explained. “This is the album I dreamed of making in high school, when I was listening to [Jay-Z]’s The Blueprint, The Game’s The Documentary, and [Kanye West’s] The College Dropout.”

      Source: Consequence of Sound

      Listen

      Notable Relases

      The Smashing Pumpkins - Shiny and Oh So Bright, Vol. 1 / LP: No Past. No Future. No Sun. (Rock, Grunge)

      Imagine Dragons - Origins (Pop Rock)

      Mariah Carey - Caution (R&B, Pop)

      The Black Eyed Peas - Masters of the Sun, Vol. 1 (Hip Hop, Pop Rap)

      Feel free to discuss or feature any and all other releases in the comments below

      Discussion Points

      Have you listened to any of these releases?
      What are your thoughts?
      What are you looking forward to listen to?
      What have you enjoyed from these artists in the past?

      This is a new format I'm trying out to help immerse people into new album discussion. I welcome and look forward to any feedback!

      8 votes
    48. What have you been watching/reading this week? (Anime/Manga)

      Sorry for the late post. Anyway, what have you been watching/reading this week? Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or...

      Sorry for the late post.

      Anyway, what have you been watching/reading this week?

      Feel free to talk about something you saw that was cool, something that was bad, ask for recommendations, or anything else you can think of.

      If you want to, feel free to find the thing you're talking about and link to its Anilist, MAL, or any other anime/manga database you use!

      5 votes
    49. Code Quality Tip: Wrapping external libraries.

      Preface Occasionally I feel the need to touch on the subject of code quality, particularly because of the importance of its impact on technical debt, especially as I continue to encounter the...

      Preface

      Occasionally I feel the need to touch on the subject of code quality, particularly because of the importance of its impact on technical debt, especially as I continue to encounter the effects of technical debt in my own work and do my best to manage it. It's a subject that is unfortunately not emphasized nearly enough in academia.


      Background

      As a refresher, technical debt is the long-term cost of the design decisions in your code. These costs can manifest in different ways, such as greater difficulty in understanding what your code is doing or making non-breaking changes to it. More generally, these costs manifest as additional time and resources being spent to make some kind of change.

      Sometimes these costs aren't things you think to consider. One such consideration is how difficult it might be to upgrade a specific technology in your stack. For example, what if you've built a back-end system that integrates with AWS and you suddenly need to upgrade your SDK? In a small project this might be easy, but what if you've built a system that you've been maintaining for years and it relies heavily on AWS integrations? If the method names, namespaces, argument orders, or anything else has changed between versions, then suddenly you'll need to update every single reference to an AWS-related tool in your code to reflect those changes. In larger software projects, this could be a daunting and incredibly expensive task, spanning potentially weeks or even months of work and testing.

      That is, unless you keep those references to a minimum.


      A Toy Example

      This is where "wrapping" your external libraries comes into play. The concept of "wrapping" basically means to create some other function or object that takes care of operating the functions or object methods that you really want to target. One example might look like this:

      <?php
      
      class ImportedClass {
          public function methodThatMightBecomeModified($arg1, $arg2) {
              // Do something.
          }
      }
      
      class ImportedClassWrapper {
          private $class_instance = null;
      
          private function getInstance() {
              if(is_null($this->class_instance)) {
                  $this->class_instance = new ImportedClass();
              }
      
              return $this->class_instance;
          }
      
          public function wrappedMethod($arg1, $arg2) {
              return $this->getInstance()->methodThatMightBecomeModified($arg1, $arg2);
          }
      }
      
      ?>
      

      Updating Tools Doesn't Have to Suck

      Imagine that our ImportedClass has some important new features that we need to make use of that are only available in the most recent version, and we're several versions behind. The problem, of course, is that there were a lot of changes that ended up being made between our current version and the new version. For example, ImportedClass is now called NewImportedClass. On top of that, methodThatMightBecomeModified is now called methodThatWasModified, and the argument order ended up getting switched around!

      Now imagine that we were directly calling new ImportedClass() in many different places in our code, as well as directly invoking methodThatMightBecomeModified:

      <?php
      
      $imported_class_instance = new ImportedClass();
      $imported_class_instance->methodThatMightBeModified($val1, $val2);
      
      ?>
      

      For every single instance in our code, we need to perform a replacement. There is a linear or--in terms of Big-O notation--a complexity of O(n) to make these replacements. If we assume that we only ever used this one method, and we used it 100 times, then there are 100 instances of new ImportClass() to update and another 100 instances of the method invocation, equaling 200 lines of code to change. Furthermore, we need to remember each of the replacements that need to be made and carefully avoid making any errors in the process. This is clearly non-ideal.

      Now imagine that we chose instead to use the wrapper object:

      <?php
      
      $imported_class_wrapper = new ImportedClassWrapper();
      $imported_class_wrapper->wrappedMethod($val1, $val2);
      
      ?>
      

      Our updates are now limited only to the wrapper class:

      <?php
      
      class ImportedClassWrapper {
          private $class_instance = null;
      
          private function getInstance() {
              if(is_null($this->class_instance)) {
                  $this->class_instance = new NewImportedClass();
              }
      
              return $this->class_instance;
          }
      
          public function wrappedMethod($arg1, $arg2) {
              return $this->getInstance()->methodThatWasModified($arg2, $arg1);
          }
      }
      
      ?>
      

      Rather than making changes to 200 lines of code, we've now made changes to only 2. What was once an O(n) complexity change has now turned into an O(1) complexity change to make this upgrade. Not bad for a few extra lines of code!


      A Practical Example

      Toy problems are all well and good, but how does this translate to reality?

      Well, I ran into such a problem myself once. Running MongoDB with PHP requires the use of an external driver, and this driver provides an object representing a MongoDB ObjectId. I needed to perform a migration from one hosting provider over to a new cloud hosting provider, with the application and database services, which were originally hosted on the same physical machine, hosted on separate servers. For security reasons, this required an upgrade to a newer version of MongoDB, which in turn required an upgrade to a newer version of the driver.

      This upgrade resulted in many of the calls to new MongoId() failing, because the old version of the driver would accept empty strings and other invalid ID strings and default to generating a new ObjectId, whereas the new version of the driver treated invalid ID strings as failing errors. And there were many, many cases where invalid strings were being passed into the constructor.

      Even after spending hours replacing the (literally) several dozen instances of the constructor calls, there were still some places in the code where invalid strings managed to get passed in. This made for a very costly upgrade.

      The bugs were easy to fix after the initial replacements, though. After wrapping new MongoId() inside of a wrapper function, a few additional conditional statements inside of the new function resolved the bugs without having to dig around the rest of the code base.


      Final Thoughts

      This is one of those lessons that you don't fully appreciate until you've experienced the technical debt of an unwrapped external library first-hand. Code quality is an active effort, but a worthwhile one. It requires you to be willing to throw away potentially hours or even days of work when you realize that something needs to change, because you're thinking about how to keep yourself from banging your head against a wall later down the line instead of thinking only about how to finish up your current task.

      "Work smarter, not harder" means putting in some hard work upfront to keep your technical debt under control.

      That's all for now, and remember: don't be fools, wrap your external tools.

      23 votes
    50. ~music Listening Club 21 - At Folsom Prison

      21 weeks and yet another classic record discussion: At Folsom Prison by Johnny Cash! At Folsom Prison is a live album and 27th overall album by Johnny Cash, released on Columbia Records in May...

      21 weeks and yet another classic record discussion: At Folsom Prison by Johnny Cash!

      At Folsom Prison is a live album and 27th overall album by Johnny Cash, released on Columbia Records in May 1968. After his 1955 song "Folsom Prison Blues", Cash had been interested in recording a performance at a prison. His idea was put on hold until 1967, when personnel changes at Columbia Records put Bob Johnston in charge of producing Cash's material. Cash had recently controlled his drug abuse problems, and was looking to turn his career around after several years of limited commercial success. Backed with June Carter, Carl Perkins and the Tennessee Three, Cash performed two shows at Folsom State Prison in California on January 13, 1968. The resulting album consisted of fifteen tracks from the first show and two tracks from the second.

      Despite little initial investment by Columbia, the album was a hit in the United States, reaching number one on the country charts and the top 15 of the national album chart. The lead single from the album, a live version of "Folsom Prison Blues", was a top 40 hit, Cash's first since 1964's "Understand Your Man". At Folsom Prison received positive reviews and revitalized Cash's career, becoming the first in a series of live albums recorded at prisons that includes "At San Quentin" (1969), "Pa Osteraker" (1973), and "A Concert Behind Prison Walls" (1976). The album was rereleased with additional tracks in 1999, a three-disc set in 2008, and a five LP box set with bonus rehearsals in 2018 for Record Store Day. It was certified three times Platinum on March 27, 2003 by the Recording Industry Association of America for US sales exceeding three million.

      Here's the place to discuss your thoughts on the record, your history with it or the artist, and basically talk about whatever you want to that goes along with At Folsom Prison! Remember that this is intended to be a slow moving thing, feel free to take your time and comment at any point in the week!

      If you'd like to stream or buy the album, it can be found on most platforms here.

      Don't forget to nominate and vote for next week's obscure record in response to this comment!

      8 votes