I have found this to be a semi controversial topic. Its almost becoming a required point for getting a new job to have open source work that you can show. Some people just enjoy working on programming side projects and others don't want to do any more after they leave the office.

Whats your opinion on this? Do you work on any side projects? Do you think its reasonable for interviewers to look for open source work when hiring?

Previous challenges

It's time for another coding challenge!

This challenge isn't mine, it's this challenge (year 5, season 3, challenge 3) by ČVUT FIKS.

The task is to design a network communication protocol. You're sending large amount of bits over the network. The problem is that network is not perfect and the message sometimes arrives corrupted. Design a network protocol, that will guarantee that the decoded message will be exactly same as the message that was encoded.

MESSAGE => (encoding) => message corrupted => (decoding) => MESSAGE

Corruption

Transmitting the message might corrupt it and introduce errors. Each error in a message (there might be more than one error in a single message) will flip all following bits of the message.

Example:

011101 => 011|010

(| is place where an error occured).

There might be more than one error in a message, but there are some rules:

• Minimum distance between two errors in a single message is k

• Number of bits between two errors is always odd number

According to these rules, describe a communication protocol, that will encode a message, and later decode message with errors.

Bonus

• Guarantee your protocol will work always - even when errors are as common as possible

• Try to make the protocol as short as possible.

Hello everyone! It has been a while since last programming challenge, it's time for another one!

This week's goal would be to build your own interpreter.

Interpreter is program that receives input and executes it. For example Python is interpreted language, meaning you are actually writing instructions for the interpreter, which does the magic.

Probably the easiest interpereter to write is Brainfuck interpreter. If someone here doesn't know, Brainfuck is programming language, which contains following instructions: ,.<>[]-+. Other characters are ignored. It has memory in form of array of integers. At the start, pointer that points to one specific memory cell points to cell 0. We can use < to move pointer to left (decrement) and > to move pointer to right (increment). . can be used to print value of cell the pointer is currently pointing to (ascii). , can be used to read one character from stdin and write it to memory. [ is beggining of loop and ] is end of loop. Loops can be nested. Loop is terminated when we reach ] character and current value in memory is equal to 0. - can be used to decrement value in memory by 1 and + can be used to increment value in memory by 1. Here's Hello World:

++++++++++[>+++++++>++++++++++>+++>+<<<<
-]>++.>+.+++++++..+++.>++.<<++++++++++++
+++.>.+++.------.--------.>+.>.

People with nothing to do today can attemp to make an interpreter for the Taxi programming language.

You can even make your own language! There are no limits for this challenge.

Preface

Conceptual models in programming are essential for being able to reason about problems. We see this through code all the time, with implementation details hidden away behind abstractions like functions and objects so that we can ignore the cumbersome details and focus only on the details that matter. Without these abstractions and conceptual models, we might find ourselves overwhelmed by the size and complexity of the problem we’re facing. Of these conceptual models, one of the most easily neglected is that of data and object structure.

Data Types Galore

Possibly one of the most overwhelming aspects of conceptualizing data and object structure is the sheer breadth of data types available. Depending on the programming language you’re working with, you may find that you have more than several dozens of object classes already defined as part of the language’s core; primitives like booleans, ints, unsigned ints, floats, doubles, longs, strings, chars, and possibly others; arrays that can contain any of the objects or primitives, and even other arrays; and several other data structures like queues, vectors, and mixed-type collections, among others.

With so many types of data, it’s incredibly easy to lose track in a sea of type declarations and find yourself confused and unsure of where to go.

Tree’s Company

Let’s start by trying to make these data types a little less overwhelming. Rather than thinking strictly of types, let’s classify them. We can group all data types into one of three basic classifications:

1. Objects, which contain key/value pairs. For example, an object property that stores a string.
2. Arrays, which contain some arbitrary number of values.
3. Primitives, which contain nothing. They’re simply a “flat” data value.

We can also make a couple of additional notes. First, arrays and objects are very similar; both contain references to internal data, but the way that data is referenced differs. In particular, objects have named keys while arrays have numeric, zero-indexed keys. In a sense, arrays are a special case of objects where the keys are more strictly typed. From this, we can condense the classifications of objects and arrays into the more general “container” classification.

With that in mind, we now have the following classifications:

1. Containers.
2. Primitives.

We can now generally state that containers may contain other containers and primitives, and primitives may not contain anything. In other words, all data structures are a composition of containers and/or primitives, where containers may accept containers and/or primitives and primitives may not accept anything. More experienced programmers should notice something very familiar about this description--we’re basically describing a tree structure! Primitive types and empty containers act as the leaves in a tree, whereas objects and arrays act as the nodes.

Trees Help You Breathe

Okay, great. So what’s the big deal, anyway? We’ve now traded a bunch of concrete data types that we can actually think about and abstracted them away into this nebulous mess of containers and primitives. What do we get out of this?

A common mistake many programmers make is planning their data types out from the very beginning. Rather than planning out an abstraction for their data and object architecture, it’s easy to immediately find yourself focusing too much on the concrete implementation details.

Imagine, for example, modeling a user account for an online payment system. A common feature to include is the ability to store payment information for auto-pay, and payment methods typically take the form of some combination of credit/debit cards and bank accounts. If we focus on implementation details from the beginning, then we may find ourselves with something like this in a first iteration:

UserAccount: {
    username: String,
    password: String,
    payment_methods: PaymentMethod[]
}

PaymentMethod: {
    account_name: String,
    account_type: Enum,
    account_holder: String,
    number: String,
    routing_number: String?,
    cvv: String?,
    expiration_date: DateString?
}

We then find ourselves realizing that PaymentMethod is an unnecessary mess of optional values and needing to refactor it. Odds are we would break it off immediately into separate account types and make a note that they both implement some interface. We may also find that, as a result, remodeling the PaymentMethod could result in the need to remodel the UserAccount. For more deeply nested data structures, a single change deeper within the structure could result in those changes cascading all the way to the top-level object. If we have multiple objects, then these changes could propagate to them as well. And what if we decide a type needs to be changed, like deciding that our expiration date needs to be some sort of date object? Or what if we decide that we want to modify our property names? We’re then stuck having to update these definitions as we go along. What if we decide that we don't want an interface for different payment method types after all and instead want separate collections for each type? Then including the interface consideration will have proven to be a waste of time. The end result is that before we’ve even touched a single line of code, we’ve already found ourselves stuck with a bunch of technical debt, and we’re only in our initial planning stages!

To alleviate these kinds of problems, it’s far better to just ignore the implementation details. By doing so, we may find ourselves with something like this:

UserAccount: {
    Username,
    Password,
    PaymentMethods
}

PaymentMethods: // TODO: Decide on this container’s structure.

CardAccount: {
    AccountName,
    CardHolder,
    CardNumber,
    CVV,
    ExpirationDate,
    CardType
}

BankAccount: {
    AccountName,
    AccountNumber,
    RoutingNumber,
    AccountType
}

A few important notes about what we’ve just done here:

1. We don’t specify any concrete data types.
2. All fields within our models have the capacity to be either containers or primitives.
3. We’re able to defer a model’s structural definition without affecting the pace of our planning.
4. Any changes to a particular field type will automatically propagate in our structural definitions, making it trivial to create a definition like ExpirationDate: String and later change it to ExpirationDate: DateObject.
5. The amount of information we need to think about is reduced down to the very bare minimum.
6. By deferring the definition of the PaymentMethods structure, we find ourselves more inclined to focus on the more concrete payment method definitions from the very beginning, rather than trying to force them to be compatible through an interface.
7. We focused only on data representation, ensuring that representation and implementation are both separate and can be handled differently if needed.

SOLIDifying Our Conceptual Model

In object-oriented programming (OOP), there’s a generally recommended set of principles to follow, represented by the acronym “SOLID”:

• Single responsibility.
• Open/closed.
• Liskov substitution.
• Interface segregation.
• Dependency inversion.

These “SOLID” principles were defined to help resolve common, recurring design problems and anti-patterns in OOP.

Of particular note for us is the last one, the “dependency inversion” principle. The idea behind this principle is that implementation details should depend on abstractions, not the other way around. Our new conceptual model obeys the dependency inversion principle by prioritizing a focus on abstractions while leaving implementation details to the future class definitions that are based on our abstractions. By doing so, we limit the elements involved in our planning and problem-solving stages to only what is necessary.

Final Thoughts

The consequences of such a conceptual model extend well beyond simply planning out data and object structures. For example, if implemented as an actual programming or language construct, you could make the parsing of your data fairly simple. By implementing an object parser that performs reflection on some passed object, you can extract all of the publicly accessible object properties of the target object and the data contained therein. Thus, if your language doesn’t have a built-in JSON encoding function and no library yet exists, you could recursively traverse your data structure to generate the appropriate JSON with very little effort.

Many of the most fundamental programming concepts, like data structures ultimately being nothing more than trees at their most abstract representation, are things we tend to take for granted and think very little about. By making ourselves conscious of these fundamental concepts, however, we can more effectively take advantage of them.

Additionally, successful programmers typically solve a programming problem before they’ve ever written a single line of code. Whether or not they’re conscious of it, the tools they use to solve these problems effectively consist largely of the myriad conceptual models they’ve collected and developed over time, and the experience they’ve accumulated to determine which conceptual models need to be utilized to solve a particular problem.

Even when you have a solid grasp of your programming fundamentals, you should always revisit them every now and then. Sometimes there are details that you may have missed or just couldn’t fully appreciate when you learned about them. This is something that I’m continually reminded of as I continue on in my own career growth, and I hope that I can continue passing these lessons on to others.

As always, I'm absolutely open to feedback and questions!

Previous challenges

Hi, it's been very long time from last Programming Challenge, and I'd like to revive the tradition.

The point of programming challenge is to create your own solution, and if you're bored, even program it in your favourite programming language. Today's challenge isn't mine. It was created by ČVUT FIKS (year 5, season 2, challenge #4).

You need to transport plans for your quantum computer through Totalitatia. The problem is, that Totalitatia's government would love to have the plans. And they know you're going to transport the computer through the country. You'll receive number N, which denotes number of cities on the map. Then, you'll get M paths, each going from one city to another. Each path has k traffic controls. They're not that much effective, but the less of them you have to pass, the better. Find path from city A to city B, so the maximum number of traffic controls between any two cities is minimal. City A is always the first one (0) and city B is always the last one (N-1).

Input format:

N
M
A1 B1 K1
A2 B2 K2
...

On the first two lines, you'll get numbers N (number of cities) and M (number of paths). Than, on next M lines, you'll get definition of a path. The definition looks like 1 2 6, where 1 is id of first city and 2 is id of second city (delimited by a space). You can go from city 1 to city 2, or from city 2 to city 1. The third number (6) is number of traffic controls.

Output format:

Single number, which denotes maximum number of traffic controls encountered on one path.

Hint: This means, that path that goes via roads with numbers of traffic controls 4 4 4 is better than path via roads with numbers of traffic controls 1 5 1. First example would have output 4, the second one would have output 5.

Example:

IN:

4
5
0 1 3
0 2 2
1 2 1
1 3 4
2 3 5

OUT:

4

Solution: The optimal path is either 0 2 1 3 or 0 1 3.

Bonus

• Describe time complexity of your algorithm.
• If multiple optimal paths exist, find the shortest one.
• Does your algorithm work without changing the core logic, if the source city and the target city is not known beforehand (it changes on each input)?
• Do you use special collection to speed up minimum value search?

Hints

Special collection to speed up algorithm

It's been over a week since the last programming challenge and the previous one was a bit more difficult, so let's do something easier and more accessible to newer programmers in particular. Write a function that takes two strings as input and returns true if they're anagrams of each other, or false if they're not.

Extra credit tasks:

• Don't consider the strings anagrams if they're the same barring punctuation.
• Write an efficient implementation (in terms of time and/or space complexity).
• Minimize your use of built-in functions and methods to bare essentials.
• Write the worst--but still working--implementation that you can conceive of.

Preface

Recently I briefly touched on the subject of cyclomatic complexity. This is an important concept for any programmer to understand and think about as they write their code. In order to provide a more solid understanding of the subject, however, I feel that I need to address the topic more thoroughly with a more practical example.

What is cyclomatic complexity?

The concept of "cyclomatic complexity" is simple: the more conditional branching and looping in your code, the more complex--and therefore the more difficult to maintain--that code is. We can visualize this complexity by drawing a diagram that illustrates the flow of logic in our program. For example, let's take the following toy example of a user login attempt:

<?php

$login_data = getLoginCredentialsFromInput();

$login_succeeded = false;
$error = '';
if(usernameExists($login_data['username'])) {
    $user = getUser($login_data['username']);
    
    if(!isDeleted($user)) {
        if(!isBanned($user)) {
            if(!loginRateLimitReached($user)) {
                if(passwordMatches($user, $login_data['password'])) {
                    loginUser($user);
                    $login_succeeded = true;
                } else {
                    $error = getBadPasswordError();
                    logBadLoginAttempt();
                }
            } else {
                $error = getLoginRateLimitError($user);
            }
        } else {
            $error = getUserBannedError($user);
        }
    } else {
        $error = getUserDeletedError($user);
    }
} else {
    $error = getBadUsernameError($login_data['username']);
}

if($login_succeeded) {
    sendSuccessResponse();
} else {
    sendErrorResponse($error);
}

?>

A diagram for this logic might look something like this:

+-----------------+
|                 |
|  Program Start  |
|                 |
+--------+--------+
         |
         |
         v
+--------+--------+    +-----------------+
|                 |    |                 |
|    Username     +--->+    Set Error    +--+
|    Exists?      | No |                 |  |
|                 |    +-----------------+  |
+--------+--------+                         |
         |                                  |
     Yes |                                  |
         v                                  |
+--------+--------+    +-----------------+  |
|                 |    |                 |  |
|  User Deleted?  +--->+    Set Error    +->+
|                 | Yes|                 |  |
+--------+--------+    +-----------------+  |
         |                                  |
      No |                                  |
         v                                  |
+--------+--------+    +-----------------+  |
|                 |    |                 |  |
|  User Banned?   +--->+    Set Error    +->+
|                 | Yes|                 |  |
+--------+--------+    +-----------------+  |
         |                                  |
      No |                                  |
         v                                  |
+--------+--------+    +-----------------+  |
|                 |    |                 |  |
|   Login Rate    +--->+    Set Error    +->+
| Limit Reached?  | Yes|                 |  |
|                 |    +-----------------+  |
+--------+--------+                         |
         |                                  |
      No |                                  |
         v                                  |
+--------+--------+    +-----------------+  |
|                 |    |                 |  |
|Password Matches?+--->+    Set Error    +->+
|                 | No |                 |  |
+--------+--------+    +-----------------+  |
         |                                  |
     Yes |                                  |
         v                                  |
+--------+--------+    +----------+         |
|                 |    |          |         |
|   Login User    +--->+ Converge +<--------+
|                 |    |          |
+-----------------+    +---+------+
                           |
                           |
         +-----------------+
         |
         v
+--------+--------+
|                 |
|   Succeeded?    +-------------+
|                 | No          |
+--------+--------+             |
         |                      |
     Yes |                      |
         v                      v
+--------+--------+    +--------+--------+
|                 |    |                 |
|  Send Success   |    |   Send Error    |
|    Message      |    |    Message      |
|                 |    |                 |
+-----------------+    +-----------------+

It's important to note that between nodes in this directed graph, you can find certain enclosed regions being formed. Specifically, each conditional branch that converges back into the main line of execution generates an additional region. The number of these distinct enclosed regions is directly proportional to the level of cyclomatic complexity of the system--that is, more regions means more complicated code.

Clocking out early.

There's an important piece of information I noted when describing the above example:

. . . each conditional branch that converges back into the main line of execution generates an additional region.

The above example is made complex largely due to an attempt to create a single exit point at the end of the program logic, causing these conditional branches to converge and thus generate the additional enclosed regions within our diagram.

But what if we stopped trying to converge back into the main line of execution? What if, instead, we decided to interrupt the program execution as soon as we encountered an error? Our code might look something like this:

<?php

$login_data = getLoginCredentialsFromInput();

if(!usernameExists($login_data['username'])) {
    sendErrorResponse(getBadUsernameError($login_data['username']));
    return;
}

$user = getUser($login_data['username']);
if(isDeleted($user)) {
    sendErrorResponse(getUserDeletedError($user));
    return;
}

if(isBanned($user)) {
    sendErrorResponse(getUserBannedError($user));
    return;
}

if(loginRateLimitReached($user)) {
    logBadLoginAttempt($user);
    sendErrorResponse(getLoginRateLimitError($user));
    return;
}

if(!passwordMatches($user, $login_data['password'])) {
    logBadLoginAttempt($user);
    sendErrorResponse(getBadPasswordError());
    return;
}

loginUser($user);
sendSuccessResponse();

?>

Before we've even constructed a diagram for this logic, we can already see just how much simpler this logic is. We don't need to traverse a tree of if statements to determine which error message has priority to be sent out, we don't need to attempt to follow indentation levels, and our behavior on success is right at the very end and at the lowest level of indentation, where it's easily and obviously located at a glance.

Now, however, let's verify this reduction in complexity by examining the associated diagram:

+-----------------+
|                 |
|  Program Start  |
|                 |
+--------+--------+
         |
         |
         v
+--------+--------+    +-----------------+
|                 |    |                 |
|    Username     +--->+   Send Error    |
|    Exists?      | No |    Message      |
|                 |    |                 |
+--------+--------+    +-----------------+
         |
     Yes |
         v
+--------+--------+    +-----------------+
|                 |    |                 |
|  User Deleted?  +--->+   Send Error    |
|                 | Yes|    Message      |
+--------+--------+    |                 |
         |             +-----------------+
      No |
         v
+--------+--------+    +-----------------+
|                 |    |                 |
|  User Banned?   +--->+   Send Error    |
|                 | Yes|    Message      |
+--------+--------+    |                 |
         |             +-----------------+
      No |
         v
+--------+--------+    +-----------------+
|                 |    |                 |
|   Login Rate    +--->+   Send Error    |
| Limit Reached?  | Yes|    Message      |
|                 |    |                 |
+--------+--------+    +-----------------+
         |
      No |
         v
+--------+--------+    +-----------------+
|                 |    |                 |
|Password Matches?+--->+   Send Error    |
|                 | No |    Message      |
+--------+--------+    |                 |
         |             +-----------------+
     Yes |
         v
+--------+--------+
|                 |
|   Login User    |
|                 |
+--------+--------+
         |
         |
         v
+--------+--------+
|                 |
|  Send Success   |
|    Message      |
|                 |
+-----------------+

Something should immediately stand out here: there are no enclosed regions in this diagram! Furthermore, even our new diagram is much simpler to follow than the old one was.

Reality is rarely simple.

The above is a really forgiving example. It has no loops, and loops are going to create enclosed regions that can't be broken apart so easily; it has no conditional branches that are so tightly coupled with the main path of execution that they can't be broken up; and the scope of functionality and side effects are minimal. Sometimes you can't break those regions up. So what do we do when we inevitably encounter these cases?

High cyclomatic complexity in your program as a whole is inevitable for sufficiently large projects, especially in a production environment, and your efforts to reduce it can only go so far. In fact, I don't recommend trying to remove all or even most instances of cyclomatic complexity at all--instead, you should just be keeping the concept in mind to determine whether or not a function, method, class, module, or other component of your system is accumulating technical debt and therefore in need of refactoring.

At this point, astute readers might ask, "How does refactoring help if the cyclomatic complexity doesn't actually go away?", and this is a valid concern. The answer to that is simple, however: we're hiding complexity behind abstractions.

To test this, let's forget about cyclomatic complexity for a moment and instead focus on simplifying the refactored version of our toy example using abstraction:

<?php

function handleLoginAttempt($login_data) {
    if(!usernameExists($login_data['username'])) {
        sendErrorResponse(getBadUsernameError($login_data['username']));
        return;
    }

    $user = getUser($login_data['username']);
    if(isDeleted($user)) {
        sendErrorResponse(getUserDeletedError($user));
        return;
    }

    if(isBanned($user)) {
        sendErrorResponse(getUserBannedError($user));
        return;
    }

    if(loginRateLimitReached($user)) {
        logBadLoginAttempt($user);
        sendErrorResponse(getLoginRateLimitError($user));
        return;
    }

    if(!passwordMatches($user, $login_data['password'])) {
        logBadLoginAttempt($user);
        sendErrorResponse(getBadPasswordError());
        return;
    }

    loginUser($user);
    sendSuccessResponse();
}

$login_data = getLoginCredentialsFromInput();

handleLoginAttempt($login_data);

?>

The code above is functionally identical to our refactored example from earlier, but has an additional abstraction via a function. Now we can diagram this higher-level abstraction as follows:

+-----------------+
|                 |
|  Program Start  |
|                 |
+--------+--------+
         |
         |
         v
+--------+--------+
|                 |
|  Attempt Login  |
|                 |
+-----------------+

This is, of course, a pretty extreme example, but this is how we handle thinking about complex program logic. We abstract it down to the barest basics so that we can visualize, in its simplest form, what the program is supposed to do. We don't actually care about the implementation unless we're digging into that specific part of the system, because otherwise we would be so bogged down by the details that we wouldn't be able to reason about what our program is supposed to do.

Likewise, we can use these abstractions to hide away the cyclomatic complexity underlying different components of our software. This keeps everything clean and clutter-free in our head. And the more we do to keep our smaller components simple and easy to think about, the easier the larger components are to deal with, no matter how much cyclomatic complexity all of those components share as a collective.

Final Thoughts

Cyclomatic complexity isn't a bad thing to have in your code. The concept itself is only intended to be used as one of many tools to assess when your code is accumulating too much technical debt. It's a warning sign that you may need to change something, nothing more. But it's an incredibly useful tool to have available to you and you should get comfortable using it.

As a general rule of thumb, you can usually just take a glance at your code and assess whether or not there's too much cyclomatic complexity in a component by looking for either of the following:

• Too many loops and/or conditional statements nested within each other, i.e. you have a lot of indentation.
• Many loops in the same function/method.

It's not a perfect rule of thumb, but it's useful for at least 90% of your development needs, and there will inevitably be cases where you will prefer to accept some greater cyclomatic complexity because there is some benefit that makes it a better trade-off. Making that judgment is up to you as a developer.

As always, I'm more than willing to listen to feedback and answer any questions!

My pick is Perl5. Even though a lot people (mostly those who’ve never touched Perl) say it’s a “write only” language, I think it does a lot right. It’s easy to prototype with, and it gives you a lot of freedom in how you want to solve a problem; which I think is one of the most important features of a programming language.

I’d like to know what your picks are!

As someone who is not mainly a web developer, I can barely grasp the immensity of options when it comes to writing a web application.

So far everything I've written has been using PHP and the Slim microframework. PHP because I don't use languages like Python/Ruby/JS that much so I didn't have any prior knowledge of those, and I've found myself to be fairly productive with it. Slim because I didn't want a full-blown framework with 200 files to configure.

I've tried Go because I've used it in the past but I don't see it to be very fit when it comes to websites, I think it's fine for small microservices but doing MVC was a chore, maybe there's a framework out there that solves this.

As for the frontend I've been trying to use as little JavaScript as possible, always vanilla. As of HTML and CSS I'm no designer so I kind of get by copying code and tweaking things here and there.

However I've started a slightly bigger project and I don't fancy myself writing everything from scratch (specially security) besides, ORMs can be useful. Symfony4 is what I've been using for a couple of days, but I've had trouble setting up debugging, and the community/docs don't seem that great since this version is fairly new; so I'm considering trying out something more popular like Django.

So this is why I created the post, I know this will differ greatly depending on the use-case. But I would like to do a quick survey and hear some of your recommendations, both on the backend and frontend. Besides I think it's a good topic for discussion.

Cheers!

Recently, Ian Lance Taylor, one of the most productive contributors to Go and, IIRC, the original author of gccgo, has written a very interesting comment on his view of the language:

(…) Go intentionally has a weak type system, and there are many restrictions that can be expressed in other languages but cannot be expressed in Go. Go in general encourages programming by writing code rather than programming by writing types. (…)

I found this distinction, writing code vs. writing types, very insightful. In my experience, in a language like Rust or (modern fancy) C++ the programmer is constantly forced to think about types, while when I program in Go or C, I almost never think about them. Types are, in fact, almost always obvious. It is also interesting that languages like Haskell and Idris explicitly expect the programmer to program with types.

What do you think?

Preface

Software security is one of those subjects that often gets overlooked, both in academia and in professional projects, unless you're specifically working with some existing security-related element (e.g. you're taking a course on security basics, or updating your password hashing algorithm). As a result, we frequently see stories of rather catastrophic data leaks from otherwise reputable businesses, leaks which should have been entirely preventable with even the most basic of safeguards in place.

With that in mind, I thought I would switch things up and discuss something security-related this time.

Background

It's commonplace for complex software systems to avoid unnecessarily large expenses, especially in terms of technical debt and the capital involved in the initial development costs of building entire systems for e.g. geolocation or financial transactions. Instead of reinventing the wheel and effectively building a parallel business, we instead integrate with existing third-party systems, typically by using an API.

The problem, however, is that sometimes these third-party systems process requests over a long period of time, potentially on the order of minutes, hours, days, or even longer. If, for example, you have users who want to purchase something using your online platform, then it's not a particularly good idea to having potentially thousands of open connections to that third-party system all sitting there waiting multiple business days for funds to clear. That would just be stupid. So, how do we handle this in a way that isn't incredibly stupid?

There are two commonly accepted methods to avoid having to wait around:

1. We can periodically contact the third-party system and ask for the current status of a request, or
2. We can give the third-party system a way to contact us and let us know when they're finished with a request.

Both of these methods work, but obviously there will be a potentially significant delay in #1 between when a request finishes and when we know that it has finished (with a maximum delay of the wait time between status updates), whereas in #2 that delay is practically non-existent. Using #1 is also incredibly inefficient due to the number of wasted status update requests, whereas #2 allows us to avoid that kind of waste. Clearly #2 seems like the ideal option.

Method #2 is what we call a webhook.

May I see your ID?

The problem with webhooks is that when you're implementing one, it's far too easy to forget that you need to restrict access to it. After all, that third-party system isn't a user, right? They're not a human. They can't just give us a username and password like we want them to. They don't understand the specific requirements for our individual, custom-designed system.

But what happens if some malicious actor figures out what the webhook endpoint is? Let's say that all we do is log webhook requests somewhere in a non-capped file or database table/collection. Barring all other possible attack vectors, we suddenly find ourselves susceptible to that malicious actor sending us thousands, possibly millions of fraudulent data payloads in a small amount of time thanks to a botnet, and now our server's I/O utilization is spiking and the entire system is grinding to a halt--we're experiencing a DDoS!

We don't want just anyone to be able to talk to our webhook. We want to make sure that anyone who does is verified and trusted. But since we can't require a username and password, since we can't guarantee that the third-party system will even know how to make use of them, what can we do?

The answer is to use some form of token-based authentication--we generate a unique token, kind of like an ID card, and we attach it to our webhook endpoint (e.g. https://example.com/my_webhook/{unique_token}). We can then check that token for validity every time someone touches our webhook, ensuring that only someone we trust can get in.

Class is in Session

Just as there are two commonly accepted models for how to handle receiving updates from third-party systems, there are also two common models for how to assign a webhook to those systems:

1. Hard-coding the webhook in your account settings, or
2. Passing a webhook as part of request payload.

Model #1 is, in my experience, the most common of the two. In this model, our authentication token is typically directly linked to some user or user-like object in our system. This token is intended to be persisted and reused indefinitely, only scrapped in the event of a breach or a termination of integration with the service that uses it. Unfortunately, if the token is present within the URL, it's possible for your token to be viewed in plaintext in your logs.

In model #2, it's perfectly feasible to mirror the behavior of model #1 by simply passing the same webhook endpoint with the same token in every new request; however, there is a far better solution. We can, instead, generate a brand new token for each new request to the third-party system, and each new token can be associated with the request itself on our own system. Rather than only validating the token itself, we then validate that the token and the request it's supposed to be associated with are both valid. This ensures that even in the event of a breach, a leaked authentication token's extent of damage is limited only to the domain of the request it's associated with! In addition, we can automatically expire these tokens after receiving a certain number of requests, ensuring that a DDoS using a single valid token and request payload isn't possible. As with model #1, however, we still run into problems of token exposure if the token is present in the URL.

Model #2 treats each individual authentication token not as a session for an entire third-party system, but as a session for a single request on that system. These per-request session tokens require greater effort to implement, but are inherently safer due to the increased granularity of our authentication and our flexibility in allowing ourselves to expire the tokens at will.

Final Thoughts

Security is hard. Even with per-request session tokens, webhooks still aren't as secure as we might like them to be. Some systems allow us to define tokens that will be inserted into the request payload, but more often than not you'll find that only a webhook URL is possible to specify. Ideally we would stuff those tokens right into the POST request payload for all of our third-party systems so they would never be so easily exposed in plaintext in log files, but legacy systems tend to be slow to catch up and newer systems often don't have developers with the security background to consider it.

Still, as far as securing webhooks goes, having some sort of cryptographically secure authentication token is far better than leaving the door wide open for any script kiddie having a bad day to waltz right in and set the whole place on fire. If you're integrating with any third-party system, your job isn't to make it impossible for them to get their hands on a key, but to make it really difficult and to make sure you don't leave any gasoline lying around in case they do.

In the rare chance you haven't heard of Flutter, here's the link: https://flutter.io

Flutter just officially left beta with v1.0 December 4, last year. The code is written in Dart, and deploys on Android, and iOS (and will run natively on the rumored Fuchsia OS).

So for those of you that have used Flutter or are currently using Flutter.

• What are you working on?
• Why'd you choose Flutter?
• What do you like about Flutter?
• And what do you dislike about Flutter?

I'll start:

I'm working on a niche art app. I myself do not do that type of art, but knowing people that do, I wanted to create a tool to fill in the lackluckster market for Chromebooks and Android.
I chose Flutter because:

• I wanted to try something new, and what newer than something that was (at the time) in beta?
• Custom Views in Android are a hassle.
• I will be able to release on both Android and iOS (semi-)natively without having to code it twice.

Here's what I like about Flutter:

• Layouts are really simple.
  (though you can easily let it get clustered if you don't think too much about it.)
• Design isn't an afterthought.
  Animations are built in (and simple), themes aren't hard-coded, and Material Components get more attention here. (Still waiting for Shapes on Android)
• It's fast by design.
  Flutter uses its own custom rendering engine (Skia). I've never experienced any stutter with the built-in components, and when I caused lag (with heavy I/O) Flutter/Dart had tools in place for me to narrow down exactly what was causing it.

What I don't like about Flutter:

• It has poor mouse/trackpad support.
  Right clicks, not a thing. I can workaround this with a double-click/long-click, but for a desktop OS, this isn't optimal. Scrolling, that's panning, this should be differentiated. There's a difference between using a scrollwheel and moving finger around on the screen. According to Flutter there is not. There's also currently no support for mouse hovers which I have needed very much.
  There is a pull-request for adding support for all of these, but the developer hasn't done anything since code review.
• Keyboard support, while there, is lackluster.
  Ctrl, Shift, Alt. These have to be gotten with the meta code. There's no built-in function for checking those. Text fields don't support the tab key to navigate. And text formatting (bold, italic, etc.) isn't possible with text fields without the use of a library (or making it yourself).

I was trying to think of a third dislike, but I can't. My complaints are on missing APIs for Chromebooks. That's it. I really like Flutter, I plan on using it more, and if they won't add support for mouse/keyboard, maybe I'll have to contribute.

I'd love to hear what your thoughts about it is.

Hi everyone, it's been 12 days since last programming challenge. So here's another one. The task is to make an algorithm that'll count how long would it take to fill system of lakes with water.

It's raining in the forest. The forest is full of lakes, which are close to each other. Every lake is below the previous one (so 1st lake is higher than 2nd lake, which is higher than 3rd lake). Lakes are empty at the beginning, and they're filling at rate of 1l/h. Once a lake is full, all water that'd normally fall into the lake will flow to the next lake.

For example, you have lakes A, B, and C. Lake A can hold 1 l of water, lake B can hold 3 l of water and lake C can hold 5 l of water. How long would it take to fill all the lakes?
After one hour, the lakes would be: A (1/1), B (1/3), C(1/5). After two hours, the lakes would be: A(1/1), B(3/3), C(2/5) (because this hour, B received 2l/h - 1l/h from the rain and 1l/h from lake A). After three hours, the lakes would be: A(1/1), B(3/3), C(5/5). So the answer is 3. Please note, that the answer can be any rational number. For example if lake C could hold only 4l instead of 5, the answer would be 2.66666....

Hour 0:

\            /
  ----(A)----
                       \                /
                        \              /
                         \            /
                          ----(B)----
                                             \           /
                                              \         /
                                               \       /
                                               |       |
                                               |       |
                                                --(C)--

Hour 1:

\============/
  ----(A)----
                       \                /
                        \              /
                         \============/
                          ----(B)----
                                             \           /
                                              \         /
                                               \       /
                                               |       |
                                               |=======|
                                                --(C)--

Hour 2:

            ==============
\============/           |
  ----(A)----            |
                       \================/
                        \==============/
                         \============/
                          ----(B)----
                                             \           /
                                              \         /
                                               \       /
                                               |=======|
                                               |=======|
                                                --(C)--

Hour 3:

            ==============
\============/           |
  ----(A)----            |             ========
                       \================/       |
                        \==============/        |
                         \============/         |
                          ----(B)----           |
                                             \===========/
                                              \=========/
                                               \=======/
                                               |=======|
                                               |=======|
                                                --(C)--

Good luck everyone! Tell me if you need clarification or a hint. I already have a solution, but it sometimes doesn't work, so I'm really interested in seeing yours :-)

Preface

Different projects have different use cases that can ultimately result in common solutions not suiting your particular needs. Today I'm going to diverging a bit from my more abstract, generalized topics on code quality and instead focus on a specific project structure example that I encountered.

Background

For a while now, I've found myself being continually frustrated with the state of my project configuration management. I had a single configuration file that would contain all of the configuration options for the various tools I've been using--database, API credentials, etc.--and I kept running into the problem of wanting to test these tools locally while not inadvertently committing and pushing sensitive credentials upstream. For me, part of my security process is ensuring that sensitive access credentials never make it into the repository and to limit access to these credentials to only people who need to be able to access them.

Monolithic Files Cause Monolithic Pain

The first thing I realized was that having a single monolithic configuration file was just terrible practice. There are going to be common configuration options that I want to have in there with default values, such as local database configuration pointing to a database instance running on the same VM as the application. These should always be in the repo, otherwise any dev who spins up an instance of the VM will need to manually tread documentation and copy-paste the missing options into the configuration. This would be incredibly time-consuming, inefficient, and stupid.

I also use different tools which have different configuration options associated with them. Having to dig through a single file containing configuration options for all of these tools to find the ones I need to modify is cumbersome at best. On top of that, having those common configuration options living in the same place that sensitive access credentials do is just asking for a rogue git commit -A to violate the aforementioned security protocol.

Same Problem, Different Structure

My first approach to resolving this problem was breaking the configuration out into separate files, one for each distinct tool. In each file, a "skeleton" config was generated, i.e. each option was given a default empty value. The main config would then only contain config options that are common and shared across the application. To avoid having the sensitive credentials leaked, I then created rules in the .gitignore to exclude these files.

This is where I ran into problem #2. I learned that this just doesn't work. You can either have a file in your repo and have all changes to that file tracked, have the file in your repo and make a local-only change to prevent changes from being tracked, or leave the file out of the repo completely. In my use case, I wanted to be able to leave the file in the repo, treat it as ignored by everyone, and only commit changes to that file when there was a new configuration option I wanted added to it. Git doesn't support this use case whatsoever.

This problem turned out to be really common, but the solution suggested is to have two separate versions of your configuration--one for dev, and one for production--and to have a flag to switch between the two. Given the breaking up of my configuration, I would then need twice as many files to do this, and given my security practices, this would violate the no-upstream rule for sensitive credentials. Worse still, if I had several different kinds of environments with different configuration--local dev, staging, beta, production--then for m such environments and n configuration files, I would need to maintain n*m separate files for configuration alone. Finally, I would need to remember to include a prefix or postfix to each file name any time I needed to retrieve values from a new config file, which is itself an error-prone requirement. Overall, there would be a substantial increase in technical debt. In other words, this approach would not only not help, it would make matters worse!

Borrowing From Linux

After a lot of thought, an idea occurred to me: within Linux systems, there's an /etc/skel/ directory that contains common files that are copied into a new user's home directory when that user is created, e.g. .bashrc and .profile. You can make changes to these files and have them propagate to new users, or you can modify your own personal copy and leave all other new users unaffected. This sounds exactly like the kind of behavior I want to emulate!

Following their example, I took my $APPHOME/config/ directory and placed a skel/ subdirectory inside, which then contained all of the config files with the empty default values within. My .gitignore then looked something like this:

$APPHOME/config/*
!$APPHOME/config/main.php
!$APPHOME/config/skel/
!$APPHOME/config/skel/*
# This last one might not be necessary, but I don't care enough to test it without.

Finally, on deploying my local environment, I simply include a snippet in my script that enters the new skel/ directory and copies any files inside into config/, as long as it doesn't already exist:

cd $APPHOME/config/skel/
for filename in *; do
    if [ ! -f "$APPHOME/config/$filename" ]; then
        cp "$filename" "$APPHOME/config/$filename"
    fi
done

(Note: production environments have a slightly different deployment procedure, as local copies of these config files are saved within a shared directory for all releases to point to via symlink.)

All of these changes ensure that only config/main.php and the files contained within config/skel/ are whitelisted, while all others are ignored, i.e. our local copies that get stored within config/ won't be inadvertently committed and pushed upstream!

Final Thoughts

Common solutions to problems are typically common for a good reason. They're tested, proven, and predictable. But sometimes you find yourself running into cases where the common, well-accepted solution to the problem doesn't work for you. Standards exist to solve a certain class of problems, and sometimes your problem is just different enough for it to matter and for those standards to not apply. Standards are created to address most cases, but edge cases will always exist. In other words, standards are guidelines, not concrete rules.

Sometimes you need to stop thinking about the problem in terms of the standard approach to solving it, and instead break it down into its most abstract, basic form and look for parallels in other solved problems for inspiration. Odds are the problem you're trying to solve isn't as novel as you think it is, and that someone has probably already solved a similar problem before. Parallels, in my experience, are usually a pretty good indicator that you're on the right track.

More importantly, there's a delicate line to tread between needing to use a different approach to solving an edge case problem you have, and needing to restructure your project to eliminate the edge case and allow the standard solution to work. Being able to decide which is more appropriate can have long-lasting repercussions on your ability to manage technical debt.

I'm a student of software engineering. I'm not a programmer yet, but I use software that is common among this crowd, like i3wm, Neovim and Emacs. I know how to find and read documentation. I've read the obnoxious How To Ask Questions the Smart Way. Every time I encounter an issue, I do my diligence. I go through the manuals, I google, I read the docs. My main editor, Emacs, has an extensive manual, with plenty of accurate details. I get that's a huge program (more like a platform, really), but let's just say that a black-and-white 650 pages PDF is not the most ADHD friendly thing in the world.

I'm aware that I chose a career that requires plenty of reading, but I happen to like it a lot and it seems like I have some aptitude for it. I had similar issues in my previous activities anyway. But it's discouraging trying to understand programming and complex software, only to be repelled by people who think everyone has their ability for concentration. Sometimes I completely lose track of time. I can sit on my computer and hyperfocus for up to 48 hours with 20 Chrome tabs open non-stop and Netflix on the background. I may seem productive, but I'm not reading anything. Maybe I read one paragraph or two, and 30 seconds later I can't remember what I was doing. But I still have tasks to accomplish, and sometimes I need help to find useful information on a 700 pages manual.

Luckily I have a great support and determination and have accomplished a lot, but my peers have no idea what I went through to get to where I am. What I don't have in natural born skills I compensate with a lot of raw effort. Everyone has their difficulties and I'm not seeking compassion, but I'd like to suggest people think twice before dismissing as "lazy" someone you know nothing about. That person might have a mental disorder, a reading disorder or even an intellectual disability. Do you wanna be the guy who told a dyslexic to just read the fucking manual?

EDIT: of course I get that time and energy are limited commodities... my point is: don't be an asshole about it. Do what you can and you wanna do, but there's no need to use hostile buzzwords when you communicate with less knowledgeable people. You're not even forced to answer... I much prefer not getting an answer than getting a hostile one.

I've posted a few lengthy topics here outside of programming challenges, and I've noticed that the ones that seem to have spurred the most interest and generated some discussion were ones that were directly related to code quality. To avoid falling for confirmation bias, though, I thought I would ask directly.

Is there generally a greater interest in code quality discussions? If so, then what kind of things are you interested in seeing in those discussions? What do you prefer not to see? If not, then what kinds of programming-related discussions would you prefer to see more of? What about non-programming discussions?

Also, is there any interest in an informal series of topics much like the programming challenges or the a layperson's introduction to... series (i.e. decentralized and available for anyone to participate whenever)? Personally, I'd be interested in seeing more on the subject from others!

Preface

Occasionally I feel the need to touch on the subject of code quality, particularly because of the importance of its impact on technical debt, especially as I continue to encounter the effects of technical debt in my own work and do my best to manage it. It's a subject that is unfortunately not emphasized nearly enough in academia.

Background

As a refresher, technical debt is the long-term cost of the design decisions in your code. These costs can manifest in different ways, such as greater difficulty in understanding what your code is doing or making non-breaking changes to it. More generally, these costs manifest as additional time and resources being spent to make some kind of change.

Sometimes these costs aren't things you think to consider. One such consideration is how difficult it might be to upgrade a specific technology in your stack. For example, what if you've built a back-end system that integrates with AWS and you suddenly need to upgrade your SDK? In a small project this might be easy, but what if you've built a system that you've been maintaining for years and it relies heavily on AWS integrations? If the method names, namespaces, argument orders, or anything else has changed between versions, then suddenly you'll need to update every single reference to an AWS-related tool in your code to reflect those changes. In larger software projects, this could be a daunting and incredibly expensive task, spanning potentially weeks or even months of work and testing.

That is, unless you keep those references to a minimum.

A Toy Example

This is where "wrapping" your external libraries comes into play. The concept of "wrapping" basically means to create some other function or object that takes care of operating the functions or object methods that you really want to target. One example might look like this:

<?php

class ImportedClass {
    public function methodThatMightBecomeModified($arg1, $arg2) {
        // Do something.
    }
}

class ImportedClassWrapper {
    private $class_instance = null;

    private function getInstance() {
        if(is_null($this->class_instance)) {
            $this->class_instance = new ImportedClass();
        }

        return $this->class_instance;
    }

    public function wrappedMethod($arg1, $arg2) {
        return $this->getInstance()->methodThatMightBecomeModified($arg1, $arg2);
    }
}

?>

Updating Tools Doesn't Have to Suck

Imagine that our ImportedClass has some important new features that we need to make use of that are only available in the most recent version, and we're several versions behind. The problem, of course, is that there were a lot of changes that ended up being made between our current version and the new version. For example, ImportedClass is now called NewImportedClass. On top of that, methodThatMightBecomeModified is now called methodThatWasModified, and the argument order ended up getting switched around!

Now imagine that we were directly calling new ImportedClass() in many different places in our code, as well as directly invoking methodThatMightBecomeModified:

<?php

$imported_class_instance = new ImportedClass();
$imported_class_instance->methodThatMightBeModified($val1, $val2);

?>

For every single instance in our code, we need to perform a replacement. There is a linear or--in terms of Big-O notation--a complexity of O(n) to make these replacements. If we assume that we only ever used this one method, and we used it 100 times, then there are 100 instances of new ImportClass() to update and another 100 instances of the method invocation, equaling 200 lines of code to change. Furthermore, we need to remember each of the replacements that need to be made and carefully avoid making any errors in the process. This is clearly non-ideal.

Now imagine that we chose instead to use the wrapper object:

<?php

$imported_class_wrapper = new ImportedClassWrapper();
$imported_class_wrapper->wrappedMethod($val1, $val2);

?>

Our updates are now limited only to the wrapper class:

<?php

class ImportedClassWrapper {
    private $class_instance = null;

    private function getInstance() {
        if(is_null($this->class_instance)) {
            $this->class_instance = new NewImportedClass();
        }

        return $this->class_instance;
    }

    public function wrappedMethod($arg1, $arg2) {
        return $this->getInstance()->methodThatWasModified($arg2, $arg1);
    }
}

?>

Rather than making changes to 200 lines of code, we've now made changes to only 2. What was once an O(n) complexity change has now turned into an O(1) complexity change to make this upgrade. Not bad for a few extra lines of code!

A Practical Example

Toy problems are all well and good, but how does this translate to reality?

Well, I ran into such a problem myself once. Running MongoDB with PHP requires the use of an external driver, and this driver provides an object representing a MongoDB ObjectId. I needed to perform a migration from one hosting provider over to a new cloud hosting provider, with the application and database services, which were originally hosted on the same physical machine, hosted on separate servers. For security reasons, this required an upgrade to a newer version of MongoDB, which in turn required an upgrade to a newer version of the driver.

This upgrade resulted in many of the calls to new MongoId() failing, because the old version of the driver would accept empty strings and other invalid ID strings and default to generating a new ObjectId, whereas the new version of the driver treated invalid ID strings as failing errors. And there were many, many cases where invalid strings were being passed into the constructor.

Even after spending hours replacing the (literally) several dozen instances of the constructor calls, there were still some places in the code where invalid strings managed to get passed in. This made for a very costly upgrade.

The bugs were easy to fix after the initial replacements, though. After wrapping new MongoId() inside of a wrapper function, a few additional conditional statements inside of the new function resolved the bugs without having to dig around the rest of the code base.

Final Thoughts

This is one of those lessons that you don't fully appreciate until you've experienced the technical debt of an unwrapped external library first-hand. Code quality is an active effort, but a worthwhile one. It requires you to be willing to throw away potentially hours or even days of work when you realize that something needs to change, because you're thinking about how to keep yourself from banging your head against a wall later down the line instead of thinking only about how to finish up your current task.

"Work smarter, not harder" means putting in some hard work upfront to keep your technical debt under control.

That's all for now, and remember: don't be fools, wrap your external tools.

The programming challenges have kind of come to a grinding halt recently. I think it's time to get another challenge started!

Given a grid of symbols, representing a simple binary state of "filled" or "unfilled", determine whether or not a square is present on the grid. Squares must be 2x2 in size or larger, must be completely solid (i.e. all symbols in the NxN space are "filled"), and must not be directly adjacent to any other filled spaces.

Example, where 0 is "empty" and 1 is "filled":

000000
011100
011100
011100
000010

// Returns true.

000000
011100
011100
011110
000000

// Returns false.

000000
011100
010100
011100
000000

// Returns false.

For those who want a greater challenge, try any of the following:

1. Get a count of all squares.
2. Detect squares that are touching (but not as a rectangle).
3. Detect other specific shapes like triangles or circles (you will need to be creative).
4. If doing (1) and (3), count shapes separately based on type.
5. Detect shapes within unfilled space as well (a checkerboard pattern is a great use case).

Preface

In a previous topic, I'd covered the subject of a few small lessons regarding code quality. Especially important was the impact on technical debt, which can bog down developer productivity, and the need to pay down on that debt. Today I would like to touch on a practical example that I'd encountered in a production environment.

Background

Before we can discuss the refactor itself, it's important to be on the same page regarding the technologies being used. In my case, I work with PHP utilizing a proprietary back-end framework and MongoDB as our database.

PHP is a server-side scripting language. Like many scripting languages, it's loosely typed. This has some benefits and drawbacks.

MongoDB is a document-oriented database. By default it's schema-less, allowing you to make any changes at will without an update to schema. This can blend pretty well with the loose typing of PHP. Each document is represented using a JSON-like structure and is stored in something called a "collection". For those of you accustomed to using relational database, a "collection" is analogous to a table, each document is a row, and each field in the document is a column. A typical query in the MongoDB shell would look something like this:

db.users.findOne({
    username: "Emerald_Knight"
});

The framework itself has some framework-specific objects that are held in global references. This makes them easily accessible, but naturally littering your code with a bunch of globals is both error-prone and an eyesore.

Unexpected Spaghetti

In my code base are a number of different objects that are designed to handle basic CRUD-like operations on their associated database entries. Some of these objects hold references to other objects, so naturally there is some data validation that occurs to ensure that the references are both valid and authorized. Pretty typical stuff.

What I noticed, however, is that the collection names for these database entries were littered throughout my code. This isn't necessarily a bad thing, except there were some use cases that came to mind: what if it turned out that my naming for one or more of these collections wasn't ideal? What if I wanted to change a collection name for the sake of easier management on the database end? What if I have a tendency to forget the name of a database collection and constantly have to look it up? What if I make a typo of all things? On top of that, the framework's database object was stored in a global variable.

These seemingly minor sources of technical debt end up adding up over time and could cause some serious problems in the worst case. I've had breaking bugs make their way passed QA in the past, after all.

Exchanging Spaghetti for Some Light Lasagna

The problem could be characterized simply: there were scoping problems and too many references to what were essentially magic strings. The solution, then, was to move the database object reference from global to local scope within the application code and to eliminate the problem of magic strings. Additionally, it's a good idea to avoid polluting the namespace with an over-reliance on constants, and using those constants for database calls can also become unsightly and difficult to follow as those constants could end up being generally disconnected from the objects they're associated with.

There turned out to be a nice, object-oriented, very PHP-like solution to this problem: a so-called "magic method" named "__call". This method is invoked whenever an "inaccessible" method is called on the object. Using this method, a database command executed on a non-database object could pass the command to the database object itself. If this logic were placed within an abstract class, the collection could then be specified simply as a configuration option in the inheriting class.

This is what such a solution could look like:

<?php

abstract class MyBaseObject {

    protected $db = null;
    protected $collection_name = null;

    public function __construct() {
        global $db;
        
        $this->db = $db;
    }

    public function __call($method_name, $args) {
        if(method_exists($this->db, $method_name)) {
            return $this->executeDatabaseCommand($method_name, $args);
        }

        throw new Exception(__CLASS__ . ': Method "' . $method_name . '" does not exist.');
    }

    public function executeDatabaseCommand($command, $args) {
        $collection = $this->collection_name;
        $db_collection = $this->db->$collection;

        return call_user_func_array(array($db_collection, $command), $args);
    }
}

class UserManager extends MyBaseObject {
    protected $collection_name = 'users';

    public function __construct() {
        parent::__construct();
    }
}

$user_manager = new UserManager();
$my_user = $user_manager->findOne(array('username'=>'Emerald_Knight'));

?>

This solution utilizes a single parent object which transforms a global database object reference into a local one, eliminating the scope issue. The collection name is specified as a class property of the inheriting object and only used in a single place in the parent object, eliminating the magic string and namespace polluting issues. Any time you perform queries on users, you do so by using the UserManager class, which guarantees that you will always know that your queries are being performed on the objects that you intend. And finally, if the collection name for an object class ever needs to be updated, it's a simple matter of modifying the single instance of the class property $collection_name, rather than tracking down some disconnected constant.

Limitations

This, of course, doesn't solve all of the existing problems. After all, executing the database queries for one object directly from another is still pretty bad practice, violating the principle of separation of concerns. Instead, those queries should generally be encapsulated within object methods and the objects themselves given primary responsibility in handling associated data. It's also incredibly easy to inadvertently override a database method, e.g. defining a findOne() method on UserManager, so there's still some mindfulness required on the part of the programmer.

Still, given the previous alternative, this is a pretty major improvement, especially for an initial refactor.

Final Thoughts

As always, technical debt is both necessary and inevitable. After all, in exchange for not taking the excess time and considering structuring my code this way in the beginning, I had greater initial velocity to get the project off of the ground. What's important is continually reviewing your code as you're building on top of it so that you can identify bottlenecks as they begin to strain your efficiency, and getting those bottlenecks out of the way.

In other words, even though technical debt is often necessary and is certainly inevitable, it's important to pay down on some of that debt once it starts getting expensive!

I have been interested in making the gradual career change to software development from my current humanities field. This stems from a handful of different places. Of course the pay and flexibility are strong drivers but I like the idea of a field that is somewhat of a creative expression; one where you can manifest your knowledge and experience into something tangible.

I have no experience with programming other than SQL use in ArcGIS and am hoping to gain some knowledge about the field; so anything would be helpful. Whether what to expect from this line of work, where someone with no experience should look to get started and what to expect, personal journeys, etc.

Cheers!

Here’s a problem I had to solve at work this week that I enjoyed solving. I think it’s a good programming challenge that will test if you really grok XML.

Your input is some XML such as this:

<DOC>
<TEXT PARTNO="000">
<TAG ID="3">This</TAG> is <TAG ID="0">some *JUNK* data</TAG> .
</TEXT>
<TEXT PARTNO="001">
*FOO* Sometimes <TAG ID="1">tags in <TAG ID="0">the data</TAG> are nested</TAG> .
</TEXT>
<TEXT PARTNO="002">
In addition to <TAG ID="1">nested tags</TAG> , sometimes there is also <TAG ID="2">junk</TAG> we need to ignore .
</TEXT>
<TEXT PARTNO="003">*BAR*-1
<TAG ID="2">Junk</TAG> is marked by uppercase characters between asterisks and can also optionally be followed by a dash and then one or more digits . *JUNK*-123
</TEXT>
<TEXT PARTNO="004">
Note that <TAG ID="4">*this*</TAG> is just emphasized . It's not <TAG ID="2">junk</TAG> !
</TEXT>
</DOC>

The above XML has so-called in-line textual annotations because the XML <TAG> elements are embedded within the document text itself.

Your goal is to convert the in-line XML annotations to so-called stand-off annotations where the text is separated from the annotations and the annotations refer to the text via slicing into the text as a character array with starting and ending character offsets. While in-line annotations are more human-readable, stand-off annotations are equally machine-readable, and stand-off annotations can be modified without changing the document content itself (the text is immutable).

The challenge, then, is to convert to a stand-off JSON format that includes the plain-text of the document and the XML tag annotations grouped by their tag element IDs. In order to preserve the annotation information from the original XML, you must keep track of each <TAG>’s starting and ending character offset within the plain-text of the document. The plain-text is defined as the character data in the XML document ignoring any junk. We’ll define junk as one or more uppercase ASCII characters [A-Z]+ between two *, and optionally a trailing dash - followed by any number of digits [0-9]+.

Here is the desired JSON output for the above example to test your solution:

{
  "data": "\nThis is some data .\n\n\nSometimes tags in the data are nested .\n\n\nIn addition to nested tags , sometimes there is also junk we need to ignore .\n\nJunk is marked by uppercase characters between asterisks and can also optionally be followed by a dash and then one or more digits . \n\nNote that *this* is just emphasized . It's not junk !\n\n",
  "entities": [
    {
      "id": 0,
      "mentions": [
        {
          "start": 9,
          "end": 18,
          "id": 0,
          "text": "some data"
        },
        {
          "start": 41,
          "end": 49,
          "id": 0,
          "text": "the data"
        }
      ]
    },
    {
      "id": 1,
      "mentions": [
        {
          "start": 33,
          "end": 60,
          "id": 1,
          "text": "tags in the data are nested"
        },
        {
          "start": 80,
          "end": 91,
          "id": 1,
          "text": "nested tags"
        }
      ]
    },
    {
      "id": 2,
      "mentions": [
        {
          "start": 118,
          "end": 122,
          "id": 2,
          "text": "junk"
        },
        {
          "start": 144,
          "end": 148,
          "id": 2,
          "text": "Junk"
        },
        {
          "start": 326,
          "end": 330,
          "id": 2,
          "text": "junk"
        }
      ]
    },
    {
      "id": 3,
      "mentions": [
        {
          "start": 1,
          "end": 5,
          "id": 3,
          "text": "This"
        }
      ]
    },
    {
      "id": 4,
      "mentions": [
        {
          "start": 289,
          "end": 295,
          "id": 4,
          "text": "*this*"
        }
      ]
    }
  ]
}

Python 3 solution here.

If you need a hint, see if you can find an event-based XML parser (or if you’re feeling really motivated, write your own).

It's time for another programming challenge!

Given a list of coordinate pairs on a 2D plane that describe the vertices of a polygon, determine whether the polygon is concave or convex.

Since a polygon could potentially be any shape if we don't specify which vertices connect to which, we'll assume that the coordinates are given in strict order such that adjacent coordinates in the list are connected. Specifically, if we call the list V[1, n] and say that V[i] <-> V[j] means "vertex i and vertex j are connected", then for each arbitrary V[i] we have V[i-1] <-> V[i] <-> V[i+1]. Moreover, since V[1] and V[n] are at the ends of the list, V[1] <-> V[n] holds (i.e. the list "wraps around").

Finally, for simplicity we can assume that all coordinates are unique, that all polygon descriptions generate valid polygons with 3 or more non-overlapping sides, and that, yes, we're working with coordinates that exist in the set of real numbers only. Don't over-complicate it :)

For those who want an even greater challenge, extend this out to work with 3D space!

It's been nearly a week, so it's time for another programming challenge!

This time, let's create a calculator that accepts reverse Polish notation (RPN), also known as postfix notation.

For a bit of background, RPN is where you take your two operands in an expression and place the operator after them. For example, the expression 3 + 5 would be written as 3 5 +. A more complicated expression like (5 - 3) x 8 would be written as 5 3 - 8 x, or 8 5 3 - x.

All your program has to do is accept a valid RPN string and apply the operations in the correct order to produce the expected result.