I have found this to be a semi controversial topic. Its almost becoming a required point for getting a new job to have open source work that you can show. Some people just enjoy working on programming side projects and others don't want to do any more after they leave the office.

Whats your opinion on this? Do you work on any side projects? Do you think its reasonable for interviewers to look for open source work when hiring?

Previous challenges

It's time for another coding challenge!

This challenge isn't mine, it's this challenge (year 5, season 3, challenge 3) by ČVUT FIKS.

The task is to design a network communication protocol. You're sending large amount of bits over the network. The problem is that network is not perfect and the message sometimes arrives corrupted. Design a network protocol, that will guarantee that the decoded message will be exactly same as the message that was encoded.

MESSAGE => (encoding) => message corrupted => (decoding) => MESSAGE

Corruption

Transmitting the message might corrupt it and introduce errors. Each error in a message (there might be more than one error in a single message) will flip all following bits of the message.

Example:

011101 => 011|010

(| is place where an error occured).

There might be more than one error in a message, but there are some rules:

• Minimum distance between two errors in a single message is k

• Number of bits between two errors is always odd number

According to these rules, describe a communication protocol, that will encode a message, and later decode message with errors.

Bonus

• Guarantee your protocol will work always - even when errors are as common as possible

• Try to make the protocol as short as possible.

I'm looking for a cheap small factor mechanical keyboard. I love the looks of XD64 or a JJ50. How do i go about it? I Don't want the numpad, don't care about leds and prefer more retro/sober styles than the ones with wings and dragons and lasers blazing across the keys.

I live in Brazil, so i'll have to import everything. I'm looking to buy everything from one place if possible.

I was looking at KPrepublic. What do i need for the xd64?

1. Plastic case
2. Key caps
3. XD64 PCB with gateron switchs (Kit 10)

Do i need plate and stabilizers? It's already $91 without it. If it's necessary, i think it would be a better idea to just buy a Durgod Taurus k320 since it's cheaper.

What do you guys think?

EDIT: Got a Magicforce 68 for $47. Thanks for the help!

Preface

Conceptual models in programming are essential for being able to reason about problems. We see this through code all the time, with implementation details hidden away behind abstractions like functions and objects so that we can ignore the cumbersome details and focus only on the details that matter. Without these abstractions and conceptual models, we might find ourselves overwhelmed by the size and complexity of the problem we’re facing. Of these conceptual models, one of the most easily neglected is that of data and object structure.

Data Types Galore

Possibly one of the most overwhelming aspects of conceptualizing data and object structure is the sheer breadth of data types available. Depending on the programming language you’re working with, you may find that you have more than several dozens of object classes already defined as part of the language’s core; primitives like booleans, ints, unsigned ints, floats, doubles, longs, strings, chars, and possibly others; arrays that can contain any of the objects or primitives, and even other arrays; and several other data structures like queues, vectors, and mixed-type collections, among others.

With so many types of data, it’s incredibly easy to lose track in a sea of type declarations and find yourself confused and unsure of where to go.

Tree’s Company

Let’s start by trying to make these data types a little less overwhelming. Rather than thinking strictly of types, let’s classify them. We can group all data types into one of three basic classifications:

1. Objects, which contain key/value pairs. For example, an object property that stores a string.
2. Arrays, which contain some arbitrary number of values.
3. Primitives, which contain nothing. They’re simply a “flat” data value.

We can also make a couple of additional notes. First, arrays and objects are very similar; both contain references to internal data, but the way that data is referenced differs. In particular, objects have named keys while arrays have numeric, zero-indexed keys. In a sense, arrays are a special case of objects where the keys are more strictly typed. From this, we can condense the classifications of objects and arrays into the more general “container” classification.

With that in mind, we now have the following classifications:

1. Containers.
2. Primitives.

We can now generally state that containers may contain other containers and primitives, and primitives may not contain anything. In other words, all data structures are a composition of containers and/or primitives, where containers may accept containers and/or primitives and primitives may not accept anything. More experienced programmers should notice something very familiar about this description--we’re basically describing a tree structure! Primitive types and empty containers act as the leaves in a tree, whereas objects and arrays act as the nodes.

Trees Help You Breathe

Okay, great. So what’s the big deal, anyway? We’ve now traded a bunch of concrete data types that we can actually think about and abstracted them away into this nebulous mess of containers and primitives. What do we get out of this?

A common mistake many programmers make is planning their data types out from the very beginning. Rather than planning out an abstraction for their data and object architecture, it’s easy to immediately find yourself focusing too much on the concrete implementation details.

Imagine, for example, modeling a user account for an online payment system. A common feature to include is the ability to store payment information for auto-pay, and payment methods typically take the form of some combination of credit/debit cards and bank accounts. If we focus on implementation details from the beginning, then we may find ourselves with something like this in a first iteration:

UserAccount: {
    username: String,
    password: String,
    payment_methods: PaymentMethod[]
}

PaymentMethod: {
    account_name: String,
    account_type: Enum,
    account_holder: String,
    number: String,
    routing_number: String?,
    cvv: String?,
    expiration_date: DateString?
}

We then find ourselves realizing that PaymentMethod is an unnecessary mess of optional values and needing to refactor it. Odds are we would break it off immediately into separate account types and make a note that they both implement some interface. We may also find that, as a result, remodeling the PaymentMethod could result in the need to remodel the UserAccount. For more deeply nested data structures, a single change deeper within the structure could result in those changes cascading all the way to the top-level object. If we have multiple objects, then these changes could propagate to them as well. And what if we decide a type needs to be changed, like deciding that our expiration date needs to be some sort of date object? Or what if we decide that we want to modify our property names? We’re then stuck having to update these definitions as we go along. What if we decide that we don't want an interface for different payment method types after all and instead want separate collections for each type? Then including the interface consideration will have proven to be a waste of time. The end result is that before we’ve even touched a single line of code, we’ve already found ourselves stuck with a bunch of technical debt, and we’re only in our initial planning stages!

To alleviate these kinds of problems, it’s far better to just ignore the implementation details. By doing so, we may find ourselves with something like this:

UserAccount: {
    Username,
    Password,
    PaymentMethods
}

PaymentMethods: // TODO: Decide on this container’s structure.

CardAccount: {
    AccountName,
    CardHolder,
    CardNumber,
    CVV,
    ExpirationDate,
    CardType
}

BankAccount: {
    AccountName,
    AccountNumber,
    RoutingNumber,
    AccountType
}

A few important notes about what we’ve just done here:

1. We don’t specify any concrete data types.
2. All fields within our models have the capacity to be either containers or primitives.
3. We’re able to defer a model’s structural definition without affecting the pace of our planning.
4. Any changes to a particular field type will automatically propagate in our structural definitions, making it trivial to create a definition like ExpirationDate: String and later change it to ExpirationDate: DateObject.
5. The amount of information we need to think about is reduced down to the very bare minimum.
6. By deferring the definition of the PaymentMethods structure, we find ourselves more inclined to focus on the more concrete payment method definitions from the very beginning, rather than trying to force them to be compatible through an interface.
7. We focused only on data representation, ensuring that representation and implementation are both separate and can be handled differently if needed.

SOLIDifying Our Conceptual Model

In object-oriented programming (OOP), there’s a generally recommended set of principles to follow, represented by the acronym “SOLID”:

• Single responsibility.
• Open/closed.
• Liskov substitution.
• Interface segregation.
• Dependency inversion.

These “SOLID” principles were defined to help resolve common, recurring design problems and anti-patterns in OOP.

Of particular note for us is the last one, the “dependency inversion” principle. The idea behind this principle is that implementation details should depend on abstractions, not the other way around. Our new conceptual model obeys the dependency inversion principle by prioritizing a focus on abstractions while leaving implementation details to the future class definitions that are based on our abstractions. By doing so, we limit the elements involved in our planning and problem-solving stages to only what is necessary.

Final Thoughts

The consequences of such a conceptual model extend well beyond simply planning out data and object structures. For example, if implemented as an actual programming or language construct, you could make the parsing of your data fairly simple. By implementing an object parser that performs reflection on some passed object, you can extract all of the publicly accessible object properties of the target object and the data contained therein. Thus, if your language doesn’t have a built-in JSON encoding function and no library yet exists, you could recursively traverse your data structure to generate the appropriate JSON with very little effort.

Many of the most fundamental programming concepts, like data structures ultimately being nothing more than trees at their most abstract representation, are things we tend to take for granted and think very little about. By making ourselves conscious of these fundamental concepts, however, we can more effectively take advantage of them.

Additionally, successful programmers typically solve a programming problem before they’ve ever written a single line of code. Whether or not they’re conscious of it, the tools they use to solve these problems effectively consist largely of the myriad conceptual models they’ve collected and developed over time, and the experience they’ve accumulated to determine which conceptual models need to be utilized to solve a particular problem.

Even when you have a solid grasp of your programming fundamentals, you should always revisit them every now and then. Sometimes there are details that you may have missed or just couldn’t fully appreciate when you learned about them. This is something that I’m continually reminded of as I continue on in my own career growth, and I hope that I can continue passing these lessons on to others.

As always, I'm absolutely open to feedback and questions!

I was fed up with the chores of writing consistent git commit messages, so a while ago I started developing a hook in Emacs which I used with Magit (actually git-commit-mode) which uses some crude heuristics to fill out the COMMIT_EDITMSG buffer for me. Here is what it does (| stands for the cursor):

• If only a single file modified, insert <filename>: |

  • If can figure out function name, insert <filename> (<functionname>): |

• If only a single file added, insert Add <filename>|

• If a TODO added to Readme.org, insert ; TODO <headline>|

• If a TODO was DONE, insert ; DONE <headline>|

• If the files are Readme.org and Readme.org_archive, and no new TODO's were added anywhere, insert ; Archive DONE|

• If the file is .gitignore, insert ; Ignore |

• If the file is TAGS, insert ; Update TAGS|

I extend this when I find new cases where I repeatedly do the same thing. The code is below. It's probably a good idea to use it as a starting point and personalise it because this reflects how I like to write my commit messages (and I like pretending how they do it over at Emacs git repo). It is sloppy and probably buggy, but I don't think it can be destructive.

Final note: I can't figure out how to set this up so that after this takes effect, the buffer is marked as modified. I want to flip the modified bit so that in some cases I can just hit C-c C-c and go. But I need to modify the buffer somehow to commit in some cases (I just type C-o to open a new line in those cases). Here is the function:

(defun gk-git-commit-mode-hook ()
  "Set up git commit buffer."
  ;; If a single file is modified, prefix the message w/ it.
  (let ((modified-re "^#	modified:")
        (new-re "^#	new file:")
        (issue-re "^[+\\- ]\\*+ \\(TODO\\|DONE\\) ")
        current-defun filename addp onlyp issuep)
    (save-excursion
      (with-current-buffer "COMMIT_EDITMSG"
        (goto-char (point-min))
        (re-search-forward "^# Changes to be committed:" nil t)
        (forward-line)
        (beginning-of-line)
        (cond ((looking-at modified-re)
               (re-search-forward ":   " nil t)
               (setf filename (thing-at-point 'filename t)))
              ((looking-at new-re)
               (re-search-forward ":   " nil t)
               (setf filename (thing-at-point 'filename t)
                     addp t)))
        (setq onlyp (progn
                      (forward-line)
                      (not (or (looking-at modified-re)
                               (looking-at new-re)))))
        (when (and onlyp (equal filename "Readme.org"))
          (goto-char (point-min))
          (when-let* ((pos (re-search-forward issue-re nil t)))
            (setq issuep (progn
                           (re-search-backward "\\*" nil t)
                           (buffer-substring (1+ (point))
                                             (line-end-position))))))
        ;; Try to set ‘current-defun’.
        (when onlyp
          (save-excursion
            (goto-char (point-min))
            ;; Error if not found, means verbose diffs
            ;; not enabled.
            (re-search-forward "^diff --git")
            (goto-char (line-beginning-position))
            (let ((str (buffer-substring (point) (point-max)))
                  (default-directory (expand-file-name "..")))
              (with-temp-buffer
                (insert str)
                (diff-mode)
                (goto-char (point-min))
                (setq current-defun (diff-current-defun))))))))
    (if onlyp
        (cond
         ((and issuep (not addp))
          (goto-char (point-min))
          (insert ";" issuep))
         ((equal filename "TAGS")
          (goto-char (point-min))
          (insert "; Update TAGS"))
         ((equal filename ".gitignore")
          (goto-char (point-min))
          (insert "; Ignore "))
         (filename
          (goto-char (point-min))
          (if addp
              (insert "Add " filename)
            (insert
             filename
             (if (and current-defun)
                 (format " (%s)" current-defun)
               "")
             ": "))))
      (when (and (equal filename "Readme.org")
                 (save-excursion
                   (goto-char (point-min))
                   (re-search-forward (concat modified-re " +Readme.org_archive")
                                      nil t))
                 (save-excursion
                   (goto-char (point-min))
                   (re-search-forward "\\-\\*+ DONE" nil t))
                 (not
                  (save-excursion
                    (goto-char (point-min))
                    (re-search-forward "\\+\\*[\\+\\-] TODO" nil t))))
        (goto-char (point-min))
        (insert "; Archive DONE")))))

(add-hook 'git-commit-mode-hook #'gk-git-commit-mode-hook)

Hope you find it useful.

I've been trying to set up a reliable lightweight solution for high quality, low-latency webcam (v4l2) streaming from Linux server to browsers, allowing for small (1-5) number of concurrent viewers.

The obvious choice here is WebRTC, which when used through browser APIs, works wonderfully. It has low latency and automatic quality adjustment depending on network performance.
I also checked out RTSP and RTMP, which are not supported without browser plugins. Next candidates were DASH and HLS, but while they provide high quality, they also have high latency.
For a while I used MPEG1 streaming through Websockets (using jsmpeg library), which worked and had low latency, but the video quality was bad.

Back to WebRTC - It seems like reliable, lightweight and maintained projects are really hard to find. So far I've found a few WebRTC media servers, but they're overkill for my use case:

• Janus
• MediaSoup
• Kurento (unmaintained)

I also tried implementing this functionality using low level Gstreamer elements in Python using PyGObject, but that's proving to be rather complicated with a ton of extremely low level implementation details.

If anyone has tried doing something similar, I'd really like to hear what (if any) problems you had and if you found any sane solutions. Next thing on my list is using headless Chromium in combination with Puppeteer, but I'd really prefer more lightweight solutions.