Discussion: The pros and cons to different approaches to solving a problem.

I can't think of an example from work that'd be sufficiently interesting without having to set up a large amount of technical background first, so that leaves the programming I do at home. Of the projects I've been working on, some community additions to this site are probably the most interesting to others. Note that this is a post more on the design choices to a new feature rather than a computational tradeoff, but I hope it's still in line with this topic.

Background

For anyone who hasn't come to Tildes from Reddit or similar link aggregation site, reposting is when someone submits a link to an external site that someone else has already submitted previously. The user is resubmitting the post, so they're "reposting." On Reddit, there are a few mechanics to avoid this, the most common being the croud of people that will call the poster a bunch of names for doing so. On Tildes, people so far are much more forgiving of this practice. Now that we have search, we have the start of tooling that should cut down on this event. Search is still in its young stage though, and we can't realistically expect every submission to be done by a user who searched the entire site to make sure the article hasn't been posted before. Thus, it'd be nice to have an additional built-in mechanic for detecting and deterring reposting. Enter: issue 181. Note that the breakdown below partially comes from my ideas, and partially from Deimos. If you want to find out who came up with what, have a peak at that link.

Approaching the problem

Unless I'm certain that a requested change involves only a very small amount of work, it's best to start with planning. When writing code for myself, I can more often get away without a dedicated planning phase, as sometimes I'll be happy with something when I see it. When submitting code to an open-source project, it's best to have a planning phase and then pass the results of that on to the code owner to see what they say. Deimos has been wonderful in responding quickly with great ideas. Contributing to an open-source project is very different than working at a job, but this post will be just looking at this open source project.

So, to this problem: lowering the occurance of reposting. First off, from the description, we have a couple of different requests:

• When the user attempts to make a repost, they "should be warned"
• If the repost is made, it should receive a "repost" tag
• The warning should only happen if it's a recent duplicate
• No warning should be given if it was posted a month ago
• A warning should only be given if the link is being posted in the same group

Those are several good ideas, and it's important to note that it is a collection of ideas, not hard requirements. I could implement all of those requests into a single flow, but when working iteratively, it's good to start from the ground up. And, as I'm only planning right now, I have to start at the base: what happens when they click the submit button.

Options

Now onto the part that actually satisfies this wall of text being posted in this topic: pros/cons of different approaches.

First, we break down the page into its components: "what is there that I can use to implement this feature?" Ignoring the other parts, there is the link input, and the submit button. Additionally, not everyone has seen it, but there is a space for validation errors.

I could:

1. watch for the user typing/pasting in the link and check it once they have, or
2. intercept the submit button and bring up an alert popup
3. use the existing errors feature
4. let them submit, but redirect them to a page that says "that's already been posted"

Option 1 won't work because of how onchange jQuery events are sent - the user could paste their link and then click the submit button without clicking anything else and the code wouldn't be called. Additionally, I'd have to block, somehow, the submission of the form, until that callback resolved and cleared the link as not being a repost. As JavaScript is all asynchronous, this could quickly become complicated.

Option 2 is bad because alerts are bad.

Option 3 is bad because there's no way for the user to "submit anyway," ignoring the errors. It is, however, programatically the simplest. I could resolve the entire issue in less than 10 lines of code. It's a bad user experience, though.

Option 4, then, has to work. It has a number of options in itself, depending on what we want to show to the user:

1. page that has text saying that the link has been posted before, with a link to the topic(s), and a button that says "post anyway"
2. like option 1, but has a button that shows up if the user is also submitting a comment that says "post my comment there"
3. like option 1, but shows the warning about the submission being a repost and link(s) to the topic(s) above the existing form

This set of options is a better example of actually looking into pros and cons of each, as the list above was simpler to break down into what's the best.

Option 1 is the simplest for the user and probably the easiest to code. It's also, however, the most restrictive.

Option 2 is an intersting one. At first glance, it can seem to be better than option 1. After all, it has everything that option 1 does, plus an option to "bring along" the comment. I don't have any data to back this up (I could get the data by crawling the site or by asking for it), but option 2 is only good if the user is also submitting a comment. If most users are submitting links as-is, without a comment, then the additional work to make option 2 work is moot. Additionally, in the use case of there being multiple instances of the link having been posted before, to which topic should the user's comment be posted? And finally, would there comment actually work on that topic. This option has a lot of assumptions and a very small use case.

Option 3 has several tradeoffs that can be identified. First, it'll be more visually complicated than option 1, as there's more on the screen. The potential for users being confused by what they're looking at will be (I hope) reduced by the fact that most of what they're seeing is the screen they were just at, just with some more text and a few links at the top. This option gives the user the ability to submit anyway (option 1 couldn't do this), and allows the user to optionally copy the comment they wanted to make so they can post it on the topic (like option 2, but without making assumptions).Done right, this'll likely be the

Plan done

Thus, the plan is to allow the user to submit the form, redirect them to a page that is that form, but with some text at the top that says the link has already been posted, with a link to the topic(s). The user will have the option to "submit anyway," or they can just go to one of the linked topics and comment there. This solves the originally problem statement that users don't know when they're submitting reposted links, and does so in a manner that gives the most options to the user.

Each option had pros and cons. Some had enough cons to be eliminited immediately, which is always nice, but some really had to be deliberated on (both internally and with Deimos, thanks!).

This is more implementation oriented, and certainly a language quirk thing, but I think it might belong here.

I spent much of my summer working on image processing in MATLAB. Now having done only a bit of programming in MATLAB prior to this for homework assignments, I was mostly approaching things from C/Python perspective. Say for a test set of 100, 300x300 images, with 10 points of data at each pixel, I might have written something like this:

for i = 1:100
  for j = 1:300
    for k = 1:300
      maxSet(i,j,k) = max(testSet(i,j,k,:));
    endfor
  endfor
endfor

At first glance, this seems like a perfectly reasonable way of doing things, and would be pretty simple to understand or port to another language. And you might think you would have to port it to another language, because when I tested that snippet it took 154.3 seconds to run. I'm testing in Octave at the moment but if I was in MATLAB I could maybe use a parfor loop to get a ~4x speed improvement, at the cost of memory from having several copies of all the arrays.

However, it's possible to take advantage of how MATLAB processes data to get a speed improvement of ~580x, for no cost in memory. Originally I was going to show off the wonders of reshape to get the 4d data into a 2d array and then process it, but I realized while writing that even that isn't necessary. MATLAB processing is centered around vectorization, so doing an operation on large vectors and matrices is automagically translated into well optimized, multithreaded code. Because of that, I can replace the code above with:

maxSet = max(testSet, [], 4);

This snippet gives me identical results to the previous one, but it runs in ~0.27 seconds. This is a huge performance gain. The tradeoff is that your code can be confusing to someone who hasn't done much MATLAB before. Also, if you want to port your code, you need figure out what the implicit logic is before you can craft the appropriate loop(s).

If you'd like to download Octave and test this for yourself, the code I used is below:

# Prep workspace
clear, clc, close all;

testSet = rand(100, 300, 300, 10);
maxSet = zeros(100, 300, 300);

# Looping implemenation
tic
for i = 1:100
  for j = 1:300
    for k = 1:300
      maxSet(i,j,k) = max(testSet(i,j,k,:));
    endfor
  endfor
endfor
toc

# Vectorized implemenation
tic
maxSet2 = max(testSet, [], 4);
toc

# Accuracy check
sum(sum(sum(maxSet == maxSet2))) == numel(maxSet)