• Activity
  • Votes
  • Comments
  • New
  • All activity
  • Showing only topics with the tag "software development". Back to normal view
    1. What is your framework for back of the envelope/ MVP style software design?

      I suspect many don’t write anything down and do this largely by intuition/experience but I want to tease out some ideas. when it comes to describing and designing a system from a blank piece of...

      I suspect many don’t write anything down and do this largely by intuition/experience but I want to tease out some ideas.

      when it comes to describing and designing a system from a blank piece of paper, what are the parameters you think of?

      I’m thinking napkin sketch level of software design.

      So things like:
      Number of users, are they concurrent users, what load dimensions there are (disk IO, network IO etc.), target platform (everything is a web app these days), how do you design/visualise the data model?

      Any decisions or constraints that impact what and how you build a proof of concept / MVP? How do you document this? How do you test it against the finished software?

      7 votes
    2. Making infinite scrollable lists for web without a constantly expanding DOM

      A common theme in web development, and the crux of the so-called "Web 2.0" is scrolling through dynamic lists of content. Tildes is such an example: you can scroll through about 50 topics on the...

      A common theme in web development, and the crux of the so-called "Web 2.0" is scrolling through dynamic lists of content. Tildes is such an example: you can scroll through about 50 topics on the front page before you reach a "next" button if you want to keep looking.

      There's a certain beauty in the simplicity of the next/previous page. When done right it's fast, it's easy, and fits neatly into a server-side rendered model. However, it does cause that small bit of friction where you need to hit the next button to go forward -- taking you out of the "flow", so-to-speak. It's slick, but it could be slicker. Perhaps more importantly, it's an interesting problem to solve.

      A step up from the next/previous button is to load the next page of content when you reach the end of the list, inserting it below. If the load is pretty fast, this will hardly interrupt your flow at all! The ever-so-popular reddit enhancement suite does precisely that for reddit: instead of a next button, when you reach the bottom, the next page of items simply plops into place. If the loading isn't fast enough, perhaps instead of loading when they reach the last item, you might choose to load when they hit the fifth from last item, etc.

      To try to keep this post more concrete, and more helpful, here's how this type of pagination would work in practice, in typescript and using the Intersection Observer API but otherwise framework agnostic:

      /**
       * Allows the user to scroll forever through the given list by calling the given loadMore()
       * function whenever the bottom element (by default) becomes visible. This assumes that
       * loadMore is the only thing that modifies the list, and that the list is done being modified
       * once the promise returned from loadMore resolves
       *
       * @param list The element which contains the individual items
       * @param loadMore A function which can be called to insert more items into the list. Can return
       *   a rejected promise to indicate that there are no more items to load
       * @param triggerLoadAt The index of the child in the list which triggers the load. Negative numbers
       *   are interpreted as offsets from the end of the list. 
       */
      function handlePagination(list: Element, loadMore: () => Promise<void>, triggerLoadAt: number = -1) {
          manageIntersection();
          return;
      
          function handleIntersection(ele: Element, handler: () => void): () => void {
              let active = true;
              const observer = new IntersectionObserver((entries) => {
                  if (active && entries[0].isIntersecting) {
                      handler()
                  }
              }, { root: null, threshold: 0.5 });
              observer.observe(ele);
              return () => {
                  if (active) {
                      active = false;
                      observer.disconnect();
                  }
              }
          }
      
          function manageIntersection() {
              const index = triggerLoadAt < 0 ? list.children.length + triggerLoadAt : triggerLoadAt;
              if (index < 0 || index >= list.children.length) {
                  throw new Error(`index=${index} is not valid for a list of ${list.children.length} items`);
              }
      
              const child = list.children[index];
              const removeIntersectionHandler = handleIntersection(child, () => {
                  removeIntersectionHandler();
                  loadMore().then(() => {
                      manageIntersection();
                  }).catch((e) => {});
              });
          }
      }
      

      If you're sane, this probably suffices for you. However, there is still one problem: as you scroll,
      the number of elements on the DOM get longer and longer. This means they necessarily take up
      some amount of memory, and browsers probably have to do some amount of work to keep
      track of them. Thus, in theory, if you were to scroll long enough, the page would get slower and
      slower! How long "long enough" is would depend mostly on how complicated each item is: if each one
      is a unique 20k element svg, it'll get slow pretty quickly.

      The trick to avoid this, and to get a constant overhead, is that when adding new items below, remove the same number of items above! Of course, if the user scrolls back up they'll be expecting those items to be there, but no worries, the handlePagination from before works just as well for loading items before the first item.

      However, this simple change is where a key problem arises: inserting elements below doesn't cause any layout shift, but inserting an item above ought to--right?

      The answer is: it depends on the browser! Back in 2017 chrome realized that it's often convenient to be able to insert items into the dom above the viewport, and implemented scroll anchoring, which basically ensures that if you insert an item 50px tall above the viewport, then scroll 50px down so that there's no visual layout shift. Firefox followed suite in 2019, and edge got support in 2020. But alas, safari both on mac and ios does not support scroll anchoring (though they expressed interest in it since 2017)

      Now, there's two responses to this:

      • Surely Safari support is coming soon, they've posted on that bug as recently as April! Just use simpler pagination for now
      • Pshhhh, just implement scroll anchoring ourself!

      Of course, I've gone and done #2, and it almost perfectly works. Here's the idea:

      • Right before loadMore, find the first item in the list which is inside the viewport. This is the item whose position we don't want to move. Use getBoundingClientRect to find it's top position.
      • Perform the DOM manipulation as desired
      • Use getBoundingClientRect again to find the new top of that item.
      • Insert (or remove) the appropriate amount of blank space at the top of the list to offset the change in client rect (note that if there's scroll anchoring support in the browser this should always be zero, which means this effectively works as progressive enhancement)

      Now, the function to do this is a tad too long for this post. I implemented it in React, however, and combined it with some stronger preloading object (we don't need all the items we've fetched from the API on the DOM, so we can use before, onTheDom, after lists to avoid getting a bunch of api requests just from scrolling down and up within the same small number of items).

      What's interesting is that it still works perfectly on chrome even with scroll-anchoring disabled (via overflow-anchor: none), but on Safari there is still, sometimes, 1 frame where it renders the wrong scroll position before immediately adjusting. Because I implemented it in react, however, my current hypothesis is I have a mistake somewhere which causes the javascript to yield to the renderer before all the manipulations are done, and it only shows up on Safari because of the generally higher framerates there

      If it's interesting to people, I could extract the infinite list component outside of this project: I certainly like it, and in my case I do expect people to want to quickly scroll through hundreds to thousands of items, so the lighter DOM feels worth it (though perhaps it wouldn't if I had known, when starting, how painful getting it to work on Safari would be!).

      What do you think of this type of "true" infinite scrolling for web? Good thing, neutral thing, bad thing? Would you use it, if the component were available? Would you remove it, if you saw someone doing this? Are there other questions about how this was accomplished? Is this an appropriate post for Tildes?

      11 votes
    3. Full-stack developers starting a software agency?

      Hey guys, I have been flip-flopping back and forth on this idea for a while, and would love some feedback on whether peeps would find this valuable. Although I still call my self a "software...

      Hey guys,

      I have been flip-flopping back and forth on this idea for a while, and would love some feedback on whether peeps would find this valuable.

      Although I still call my self a "software developer" (and try to code daily), for the last 8 years I have ran a small 5-person agency that I started from the ground up, so my role was really CEO/CTO/CFO/Everything-O. My company focused on delivering high-quality custom software. Not brochure websites, and not Wordpress - our niche was internal business software (or as I like to call it "boring software for boring businesses") - and for a client service company we got very high margins of return.

      Last year my business was acquired by a larger company which was an amazing result after the time and effort I had poured into it. I have realised I now want to help other developers who want to start their own software agency, or maybe they already have and are looking for hints or advice on certain topics.

      So I have started Dev to Agency - a part blog part guidebook for how a full-stack developer can start and successfully run a software development agency, the things to pay attention too (and the things to ignore), and the key-values that I feel helped my business go from nothing, to 7 figures per year, and then to being acquired (if that is a path people would want to take).

      I have just published my first couple of posts, About Dev To Agency that is a rundown of what I hope to achieve with this, then a post about My small custom software development agency - which gives an overview of what I built and where I think my articles will add value, and lastly You are the gold standard which covers how I feel an owner/maker should set the businesses standards and practises based of their personal values.

      I have never written a blog before (or really done any writing before), so it would be fantastic to get some feedback from the community, and if there are any developers that this could interest then please subscribe on the website.

      Cheers,

      Chris.

      15 votes
    4. Looking for advice on a CI / regression testing platform

      Hi all, I'm looking for some advice regarding how to set up a basic CI regression / testing suite. This isn't my full time job, but a side project my group at work wants to spin up to... shall we...

      Hi all,

      I'm looking for some advice regarding how to set up a basic CI regression / testing suite. This isn't my full time job, but a side project my group at work wants to spin up to... shall we say, give us a more real time monitoring of functionality and performance regressions coming out of the underlying software stack development (long story).

      As none of us are particularly automation experts, I was looking for some advice from my fellow Tilderinos. Please forgive me if any of the below is obvious and/or silly.

      A few basic requirements I had in mind:

      1. Can handle different execution environments: essentially different versions of the software stack, both in docker form and (eventually) via lmod or some other module file approach (e.g., TCL), and sensible handling of a node list.

      2. Related to one, supports using the products of builds as execution environments. Ideally we'd like to have a build step compile the stack and install it to a NFS from which we can load it as a module.

      3. Simple to add tests. Again, this isn't our full time job -- we mostly want to add a quick bash script / makefile / source code or the like to the tests when we run into an issue and forgot about it.

      4. Related. We should be able to store the entire thing as a git repo. I have seen this to some extent with Travis, but my experience with Jenkins was... sub-par (is there a history? Changelog? Any way at all of backing up the test config?).

      5. Some sort of post-processing capabilities. At a glance we need to be able to see the top line performance numbers for 20-30 apps over the different build environment. Bonus points if there's a graph showing performance vs build version or the like, but honestly a CSV log file is good enough.

      6. Whatever CI software we get has to be able to run this locally. Lots of these are internal only numbers / codes. FOSS prefered.

      7. A webui for scheduling runs / visualizing results would be nice, but again this could be a bash script and none of us would bat an eye.

      Any thoughts would be greatly appreciated. Thanks!

      7 votes
    5. Programming/software design practice?

      So, I've been going through Project Euler and solving problems as a way to brush up on my programming abilities, but it's mostly a math-focused set of problems. Which is cool..they're nice little...

      So, I've been going through Project Euler and solving problems as a way to brush up on my programming abilities, but it's mostly a math-focused set of problems. Which is cool..they're nice little puzzles that get the gears turning...

      BUT I'm wondering if anyone here has suggestions for a website/course that teaches software design in a piece-wise way. Like... each problem is a nugget of software design that builds off previous problems and eventually you're creating an entire application utilizing different algorithms/design patterns/data structures/etc.

      I'd appreciate any resources similar to that idea. Thanks!

      7 votes
    6. New to Leading a Team of Software Developers

      Hey Tildes, I got a job directly supervising a small team of 4 software developers. I'm very excited at the prospect and would like to put my best foot forward. To that end, I would like to have a...

      Hey Tildes, I got a job directly supervising a small team of 4 software developers. I'm very excited at the prospect and would like to put my best foot forward. To that end, I would like to have a discussion around a few topics. Feel free to expand the scope if you believe the conversation would be beneficial. I'm sure I won't be the last person to be in this position. I've done research, read, and watched videos regarding several of these questions; however, since Tilde prioritizes high-quality discussion, I thought it would be a fun opportunity to chat with others about these topics.

      • As a member of a software development team, what are things that your supervisor has done that has had the greatest (a) positive and (b) negative impact?
      • Supervisors, when you joined your new team, what was your methodology for reviewing the team, projects, and processes? What was the scenarios behind your review and the outcome? What would you do differently?
      12 votes