• Activity
  • Votes
  • Comments
  • New
  • All activity
  • Showing only topics in ~comp with the tag "development". Back to normal view / Search all groups
    1. Non-engineers AI coding & corporate compliance?

      Part of my role at work is in security policy & implementation. I can't figure this out so maybe someone will have some advice. With the advent of AI coding, people who don't know how to code now...

      Part of my role at work is in security policy & implementation. I can't figure this out so maybe someone will have some advice.

      With the advent of AI coding, people who don't know how to code now start to use the AI to automate their work. This isn't new - previously they might use already other low code tools like Excel, UIPath, n8n, etc. but it still require learning the tools to use it. Now, anyone can "vibe coding" and get an output, which is fine for engineers who understand how the output should work and can design how it should be tested (edge cases, etc.)

      I had a team come up with me that they managed to automate their work, which is good, but they did it with ChatGPT and the code works as they expected, but they doesn't fully understand how the code works and of course they're deploying this "to production" which means they're setting up an environment that supposed to be for internal tools, but use real customer data fed in from the production systems.

      If you're an engineer, usually this violates a lot of policies - you should get the code peer reviewed by people who know what it does (incl. business context), the QA should test the code and think about edge cases and the best ways to test it and sign it off, the code should be developed & tested in non-production environment with fake data.

      I can't think of a way non-engineers can do this - they cannot read code (and it get worse if you need two people in the same team to review each other) and if you're outsourcing it to AI, the AI company doesn't accept liability, nor you can retrain the AI from postmortems. The only way is to include lessons learned into the prompt, and I guess at some point it will become one long holy bible everyone has to paste into the limited context window. They are not trained to work on non-production data (if you ever try, usually they'll claim that the data doesn't match production - which I think because they aren't trained to design and test for edge cases). The only way to solve this directly is asking engineers to review them, but engineers aren't cheap and they're best doing something more important.

      So far I think the best way to approach this problem is to think of it like Excel - the formulas are always safe to use - they don't send data to the internet, they don't create malware, etc. The worst think they can do is probably destroy that file or hangs your PC. And people don't know how to write VBA so they never do it. Now you have people copy pasting VBA code that they don't understand. The new AI workspace has to be done by building technical guardrails that the AI are limited to. I think it has to be done in some low-code tools that people using AI has to use (like say n8n). For example, blocks that do computation can be used, blocks that send data to the intranet/internet or run arbitrary code requires approval before use. And engineers can build safe blocks that can be used, such as sending messages to Slack that can only be used to send to corporate workspace only.

      Does your work has adjusted policies for this AI epidemic? or other ideas that you wanted to share?

      23 votes
    2. Seeking advice as a Frontend web developer

      We have this big project at work...an "all hands on deck" kinda thing that has us rank-and-file frontend devs working alongside our manager more closely than I'm used to. And it was fine, because...

      We have this big project at work...an "all hands on deck" kinda thing that has us rank-and-file frontend devs working alongside our manager more closely than I'm used to. And it was fine, because I like the guy and he's been a decent manager. But this project is killing me.

      On multiple occasions now I've written code, had it pass code review (often with his approval after a round of changes/guidance), and then every few days we get these massive re-write PRs from him where he completely rewrites large chunks of what we've done. It's leaving me feeling a few different ways:

      1. Angry because how quickly your code gets replaced is a (imo, bullshit) metric used as a part of our annual reviews and promotion discussions
      2. Doubting myself because in my head a good developer doesn't have their code rewritten that quickly.
      3. Confused because features I thought I understood are constantly being rewritten leaving me wasting time trying to relearn how things work
      4. Wondering what the point of writing code is if it's just going to be thrown in the garbage later in the week?

      And like I'll be the first to admit I'm not the most proficient developer on our team. React and Typescript are relatively new concepts to me, despite a long career in web development. But I've been writing with it for about a year now and I had thought I was finally getting a good grasp on things. But now I'm wondering if I'm just an idiot? Is it imposter syndrome or have I actually somehow coasted through a 15 year career across various stacks and it's just now catching up to me?

      Or is this just the nature of massive projects like this? We had a half-baked product scope to begin with and its getting daily changes with entire chunks of it not very well thought out by our PM. I can see how it would make sense that the more senior developer might see the need to refactor things when things are constantly changing and we're left writing code based on assumptions and half-written requirements. I'm also getting are comments on my PRs that request changes, but mid-comment he's like "I'll just take care of this because it's blocking me".

      It's just really taking a toll on my mental health and how I feel about my job. I've been trying to find another job for a few months now, but I'm not having any luck. Job hunting sucks and when you're already demoralized as hell, it's hard to sell yourself to prospective employers.

      Could really use some insight from other experienced devs, please!

      12 votes
    3. Ideas for a side project I'm working on -- an RPG to help me curb my alcohol consumption

      Preface: I am familiar with Habitica. This idea would probably scratch a similar itch, but I'm also using this as an opportunity to sharpen my Rust skills. My idea came about when I was trying to...

      Preface: I am familiar with Habitica. This idea would probably scratch a similar itch, but I'm also using this as an opportunity to sharpen my Rust skills.

      My idea came about when I was trying to find out some new tactics to curb my alcohol consumption, which isn't quite out of control yet, but I don't want to tempt fate.

      I've also really liked the progression aspect of RPGs. What if I could gamify my quest to not drink alcohol and make it sort of a fun, unique RPG experience at the same time?

      In the broadest sense, it would go something like this:

      • You open the game up, ideally each day. You are instantly prompted: "Did you drink yesterday?" (and perhaps it will go back a few more days if you skipped).
      • For each day you answer "no", you are rewarded with some sort of tokens, credits, etc. -- currency to play the game. If you answer "yes", maybe you get penalized somehow.
      • Then, you pick up your journey, which is sort of a standard RPG experience -- fighting battles, buying gear, learning spells, leveling up, advancing through the world, you name it.
      • The game should get progressively more difficult, but should not have an ending, as "quitting alcohol" does not have an ending either. At the same time, it should scratch the RPG progression itch.

      The initial game concept I came up with is just one that I see as the quickest way to get this off the ground, which would be something CLI-based, where you are presented with a menu ("visit shop, enter arena, view equipment" etc.). You spend battle tokens to enter into arena battles, which reward experience points, money, and gear. You level up, work towards a build (there needs to be a way to respec because restarting isn't really an option), and progress through the arena.

      In total, you would probably spend less than 5 minutes every day playing the game, which is by design. It should be an every day habit. But, there should be enough entertainment value that, if I'm not getting those sweet battle tokens by not drinking, I'm missing out on experiencing the game (or, I could lie, which defeats the purpose of the app).

      So that's where I'm at right now. I'm really interested to hear your thoughts, ideas, critiques, etc. before I spend a free weekend building out a concept.

      Some questions in particular:

      • I was leaning toward just building this in CLI because it will be extremely simple. It could just be a matter of STDINs. However, I'm open to other Rust-based options. Is there a good Rust UI toolkit or web framework that is worth looking into that would make this a little more modern?
      • What about game features? What could make this a really fun experience, while also balancing the whole concept of being built around your life and your habits?

      In the end, this is a deeply personal project that would be built, first and foremost, for my specific needs. But that's not to say I couldn't build it with some scalability in mind. Rather than asking about alcohol, perhaps the "habits" can be customized, and so forth.

      Anyway, have a great weekend!

      23 votes
    4. How do you organize your Linux packages?

      Hello everyone. I am planning to get back into Linux development after working with Mac only for almost a decade. On Mac, one of the most important lessons that I learned was to always use...

      Hello everyone.

      I am planning to get back into Linux development after working with Mac only for almost a decade. On Mac, one of the most important lessons that I learned was to always use Homebrew. Using various package managers (e.g. Homebrew, NPM, Yarn, Pip, etc.) creates situations in which you don't know how to uninstall or upgrade certain pieces of software. Also, it's hard to generate a complete overview.

      How do you Linux folks handle this?

      Bonus question: How do you manage your dotfiles securely? I use Bitwarden, and it's a bit clunky.

      If that helps, I want to try Mint and always use Oh My ZSH!.

      6 votes
    5. Game Development Career Advice

      Hi, I'm curious if anyone in this group has achieved success in game development, whether that's carving out a career or earning any amount of income from it. I'm currently working as a software...

      Hi,

      I'm curious if anyone in this group has achieved success in game development, whether that's carving out a career or earning any amount of income from it.

      I'm currently working as a software developer, but my passion lies in game development. I'm all too aware that achieving any measure of success in this field is next to impossible. Hence, I'm reaching out here, hoping to gather insights and advice from those who have walked this path in the past, or those who are currently walking alongside/behind me.

      One of my specific questions is about the types of games I should focus on creating. Specifically, I've heard differing opinions on whether it's more advantageous to develop a series of small games with advertisements for mobile platforms or to invest in larger, premium games for platforms like Steam. Can anyone share their insights or experiences regarding this dilemma? Is there a clear advantage to one approach over the other?

      Currently I am using godot to make a larger scale game, but I am considering switching to defold and making smaller scale games with ads.

      I saw some folks here discuss making games for the playdate. How much should one consider targeting niche platforms like this? Some of the users I saw discuss this seem to have had good success.

      Some general questions: How did you break into game dev? What were you doing before? Do you see game dev as a viable career, only as a source of side income, or is it just a hobby?

      Any guidance or experiences you can share would be greatly appreciated.

      17 votes
    6. Just got an Nvidia 4090 GPU, looking for local LLM + general generative AI software recommendations

      I was fortunate enough to grab a discounted 4090 while on my travels and just got everything installed. Already having a lot of fun pumping all my games to max settings, but I'm also interested in...

      I was fortunate enough to grab a discounted 4090 while on my travels and just got everything installed. Already having a lot of fun pumping all my games to max settings, but I'm also interested in running generative AI stuff locally to really take advantage of all that VRAM.

      Do you have any newbie-friendly Windows 11 software to recommend for getting started? Thanks!

      20 votes
    7. Newbie here looking for advice on how to get into Programming/CS by building a project

      Been lurking for a week on tildes now and I am really glad this place exists. The crow here is exactly what I have been missing on Reddit for a while now. Having said that, the whole Reddit...

      Been lurking for a week on tildes now and I am really glad this place exists. The crow here is exactly what I have been missing on Reddit for a while now.

      Having said that, the whole Reddit situation has some-what motivated me to get the balls rolling on an idea that I have had for a while and I am looking for advice on the same.

      I have often heard this phrase "Learn programming by building" but whenever I dive in to the resources, I fall flat due to the information overload and the general abstractness that the field has (I appreciate abstractness but here it demotivates me) and I have never found a proper resource that I could follow to actually build something instead of just blindly following tutorials and playing with them.

      So, my question is how do I translate "learn by building a project" into a practical framework.

      I know of 100 days of swift and I really like that approach however I don't think I want to start with swift or build an iOS app right now.

      24 votes
    8. Making infinite scrollable lists for web without a constantly expanding DOM

      A common theme in web development, and the crux of the so-called "Web 2.0" is scrolling through dynamic lists of content. Tildes is such an example: you can scroll through about 50 topics on the...

      A common theme in web development, and the crux of the so-called "Web 2.0" is scrolling through dynamic lists of content. Tildes is such an example: you can scroll through about 50 topics on the front page before you reach a "next" button if you want to keep looking.

      There's a certain beauty in the simplicity of the next/previous page. When done right it's fast, it's easy, and fits neatly into a server-side rendered model. However, it does cause that small bit of friction where you need to hit the next button to go forward -- taking you out of the "flow", so-to-speak. It's slick, but it could be slicker. Perhaps more importantly, it's an interesting problem to solve.

      A step up from the next/previous button is to load the next page of content when you reach the end of the list, inserting it below. If the load is pretty fast, this will hardly interrupt your flow at all! The ever-so-popular reddit enhancement suite does precisely that for reddit: instead of a next button, when you reach the bottom, the next page of items simply plops into place. If the loading isn't fast enough, perhaps instead of loading when they reach the last item, you might choose to load when they hit the fifth from last item, etc.

      To try to keep this post more concrete, and more helpful, here's how this type of pagination would work in practice, in typescript and using the Intersection Observer API but otherwise framework agnostic:

      /**
       * Allows the user to scroll forever through the given list by calling the given loadMore()
       * function whenever the bottom element (by default) becomes visible. This assumes that
       * loadMore is the only thing that modifies the list, and that the list is done being modified
       * once the promise returned from loadMore resolves
       *
       * @param list The element which contains the individual items
       * @param loadMore A function which can be called to insert more items into the list. Can return
       *   a rejected promise to indicate that there are no more items to load
       * @param triggerLoadAt The index of the child in the list which triggers the load. Negative numbers
       *   are interpreted as offsets from the end of the list. 
       */
      function handlePagination(list: Element, loadMore: () => Promise<void>, triggerLoadAt: number = -1) {
          manageIntersection();
          return;
      
          function handleIntersection(ele: Element, handler: () => void): () => void {
              let active = true;
              const observer = new IntersectionObserver((entries) => {
                  if (active && entries[0].isIntersecting) {
                      handler()
                  }
              }, { root: null, threshold: 0.5 });
              observer.observe(ele);
              return () => {
                  if (active) {
                      active = false;
                      observer.disconnect();
                  }
              }
          }
      
          function manageIntersection() {
              const index = triggerLoadAt < 0 ? list.children.length + triggerLoadAt : triggerLoadAt;
              if (index < 0 || index >= list.children.length) {
                  throw new Error(`index=${index} is not valid for a list of ${list.children.length} items`);
              }
      
              const child = list.children[index];
              const removeIntersectionHandler = handleIntersection(child, () => {
                  removeIntersectionHandler();
                  loadMore().then(() => {
                      manageIntersection();
                  }).catch((e) => {});
              });
          }
      }
      

      If you're sane, this probably suffices for you. However, there is still one problem: as you scroll,
      the number of elements on the DOM get longer and longer. This means they necessarily take up
      some amount of memory, and browsers probably have to do some amount of work to keep
      track of them. Thus, in theory, if you were to scroll long enough, the page would get slower and
      slower! How long "long enough" is would depend mostly on how complicated each item is: if each one
      is a unique 20k element svg, it'll get slow pretty quickly.

      The trick to avoid this, and to get a constant overhead, is that when adding new items below, remove the same number of items above! Of course, if the user scrolls back up they'll be expecting those items to be there, but no worries, the handlePagination from before works just as well for loading items before the first item.

      However, this simple change is where a key problem arises: inserting elements below doesn't cause any layout shift, but inserting an item above ought to--right?

      The answer is: it depends on the browser! Back in 2017 chrome realized that it's often convenient to be able to insert items into the dom above the viewport, and implemented scroll anchoring, which basically ensures that if you insert an item 50px tall above the viewport, then scroll 50px down so that there's no visual layout shift. Firefox followed suite in 2019, and edge got support in 2020. But alas, safari both on mac and ios does not support scroll anchoring (though they expressed interest in it since 2017)

      Now, there's two responses to this:

      • Surely Safari support is coming soon, they've posted on that bug as recently as April! Just use simpler pagination for now
      • Pshhhh, just implement scroll anchoring ourself!

      Of course, I've gone and done #2, and it almost perfectly works. Here's the idea:

      • Right before loadMore, find the first item in the list which is inside the viewport. This is the item whose position we don't want to move. Use getBoundingClientRect to find it's top position.
      • Perform the DOM manipulation as desired
      • Use getBoundingClientRect again to find the new top of that item.
      • Insert (or remove) the appropriate amount of blank space at the top of the list to offset the change in client rect (note that if there's scroll anchoring support in the browser this should always be zero, which means this effectively works as progressive enhancement)

      Now, the function to do this is a tad too long for this post. I implemented it in React, however, and combined it with some stronger preloading object (we don't need all the items we've fetched from the API on the DOM, so we can use before, onTheDom, after lists to avoid getting a bunch of api requests just from scrolling down and up within the same small number of items).

      What's interesting is that it still works perfectly on chrome even with scroll-anchoring disabled (via overflow-anchor: none), but on Safari there is still, sometimes, 1 frame where it renders the wrong scroll position before immediately adjusting. Because I implemented it in react, however, my current hypothesis is I have a mistake somewhere which causes the javascript to yield to the renderer before all the manipulations are done, and it only shows up on Safari because of the generally higher framerates there

      If it's interesting to people, I could extract the infinite list component outside of this project: I certainly like it, and in my case I do expect people to want to quickly scroll through hundreds to thousands of items, so the lighter DOM feels worth it (though perhaps it wouldn't if I had known, when starting, how painful getting it to work on Safari would be!).

      What do you think of this type of "true" infinite scrolling for web? Good thing, neutral thing, bad thing? Would you use it, if the component were available? Would you remove it, if you saw someone doing this? Are there other questions about how this was accomplished? Is this an appropriate post for Tildes?

      11 votes
    9. Looking for smallish feature suggestions for an open source project

      I'm thinking about increasing the level of my open source contributions a bit. Instead of searching blindly until I stumble upon an issue that: A) Piques my interest B) I feel somewhat qualified...

      I'm thinking about increasing the level of my open source contributions a bit. Instead of searching blindly until I stumble upon an issue that:
      A) Piques my interest
      B) I feel somewhat qualified to implement

      I figured I'd check with the tildes community. Is there any Open Source software that you use that is missing a feature/capability? Can you give a brief description of it (bonus points for links to an issue tracker with an open ticket :))?

      Can't of course promise anything will come of it, but if I do pick up your suggestion at least I'll give you a ping if I make any progress!

      7 votes
    10. Trying to become a junior developer in Brazil is an uphill battle

      They ask for years of experience, skills that no Jr would know since, well, it is a Jr and the process to apply for jobs are surreal. Thousands of tests, interviews that goes nowhere and lots of...

      They ask for years of experience, skills that no Jr would know since, well, it is a Jr and the process to apply for jobs are surreal. Thousands of tests, interviews that goes nowhere and lots of ghosting. And the pay is not that good. No wonder after 2 or 3 years of experience a lot of develpers starts working for companies outside of Brazil.

      Last one to contact me sent me a test to do it in 1 week. I went above and beyond and learned a lot of things. Before this, i had some small projects in Go and Python. Now i needed to learn Docker, tests, github actions, Postgresql and other things. Not everything was mandatory, but i did my best and did it all. I finished in 5 days since i have a day job.

      Here is the result: https://github.com/crdpa/conservice

      Showing the data in the browser was not necessary, but i think it was a nice touch and well made. If this does not land me a job as a junior developer i don't know what else could.

      I'm glad i already have a job in another area, but me and my SO are separated by a 4 hour drive and i'm tired. I want to work from home to be near her and our dog. Paying rent in two places is becoming a burden.

      I would be happy if you guys could test the application i made. It only needs docker.

      And do you guys have any tips from now on?

      7 votes
    11. A few easy linux commands, and a real-world example on how to use them in a pinch

      This below is a summary of some real-world performance investigation I recently went through. The tools I used are installed on all linux systems, but I know some people don't know them and would...

      This below is a summary of some real-world performance investigation I recently went through. The tools I used are installed on all linux systems, but I know some people don't know them and would straight up jump to heavyweight log analysis services and what not, or writing their own solution.

      Let's say you have request log sampling in a bunch of log files that contain lines like these:

      127.0.0.1 [2021-05-27 23:28:34.460] "GET /static/images/flags/2/54@3x.webp HTTP/2" 200 1806 TLSv1.3 HIT-CLUSTER SessionID:(null) Cache:max-age=31536000
      127.0.0.1 [2021-05-27 23:51:22.019] "GET /pl/player/123456/changelog/ HTTP/1.1" 200 16524 TLSv1.2 MISS-CLUSTER SessionID:(null) Cache:

      You might recognize Fastly logs there (IP anonymized). Now, there's a lot you might care about in this log file, but in my case, I wanted to get a breakdown of hits vs misses by URL.

      So, first step, let's concatenate all the log files with cat *.log > all.txt, so we can work off a single file.

      Then, let's split the file in two: hits and misses. There are a few different values for them, the majority are covered by either HIT-CLUSTER or MISS-CLUSTER. We can do this by just grepping for them like so:

      grep HIT-CLUSTER all.txt > hits.txt; grep MISS-CLUSTER all.txt > misses.txt
      

      However, we only care about url and whether it's a hit or a miss. So let's clean up those hits and misses with cut. The way cut works, it takes a delimiter (-d) and cuts the input based on that; you then give it a range of "fields" (-f) that you want.

      In our case, if we cut based on spaces, we end up with for example: 127.0.0.1 [2021-05-27 23:28:34.460] "GET /static/images/flags/2/54@3x.webp HTTP/2" 200 1806 TLSv1.3 HIT-CLUSTER SessionID:(null) Cache:max-age=31536000.

      We care about the 5th value only. So let's do: cut -d" " -f5 to get that. We will also sort the result, because future operations will require us to work on a sorted list of values.

      cut -d" " -f5 hits.txt | sort > hits-sorted.txt; cut -d" " -f5 misses.txt | sort > misses-sorted.txt
      

      Now we can start doing some neat stuff. wc (wordcount) is an awesome utility, it lets you count characters, words or lines very easily. wc -l counts lines in an input, since we're operating with one value per line we can easily count our hits and misses already:

      $ wc -l hits-sorted.txt misses-sorted.txt
        132523 hits-sorted.txt
        220779 misses-sorted.txt
        353302 total
      

      220779 / 132523 is a 1:1.66 ratio of hits to misses. That's not great…

      Alright, now I'm also interested in how many unique URLs are hit versus missed. uniq tool deduplicates immediate sequences, so the input has to be sorted in order to deduplicate our entire file. We already did that. We can now count our urls with uniq < hits-sorted.txt | wc -l; uniq < misses-sorted.txt | wc -l. We get 49778 and 201178, respectively. It's to be expected that most of our cache misses would be in "rarer" urls; this gives us a 1:4 ratio of cached to uncached URL.

      Let's say we want to dig down further into which URLs are most often hitting the cache, specifically. We can add -c to uniq in order to get a duplicate count in front of our URLs. To get the top ones at the top, we can then use sort, in reverse sort mode (-r), and it also needs to be numeric sort, not alphabetic (-n). head lets us get the top 10.

      $ uniq -c < hits-sorted.txt | sort -nr | head
          815 /static/app/webfonts/fa-solid-900.woff2?d720146f1999
          793 /static/app/images/1.png
          786 /static/app/fonts/nunito-v9-latin-ext_latin-regular.woff2?d720146f1999
          760 /static/CACHE/js/output.cee5c4089626.js
          758 /static/images/crest/3/light/notfound.png
          757 /static/CACHE/css/output.4f2b59394c83.css
          756 /static/app/webfonts/fa-regular-400.woff2?d720146f1999
          754 /static/app/css/images/loading.gif?d720146f1999
          750 /static/app/css/images/prev.png?d720146f1999
          745 /static/app/css/images/next.png?d720146f1999
      

      And same for misses:

      $ uniq -c < misses-sorted.txt | sort -nr | head
           56 /
           14 /player/237678/
           13 /players/
           12 /teams/
           11 /players/top/
      <snip>
      

      So far this tells us static files are most often hit, and for misses it also tells us… something, but we can't quite track it down yet (and we won't, not in this post). We're not adjusting for how often the page is hit as a whole, this is still just high-level analysis.

      One last thing I want to show you! Let's take everything we learned and analyze those URLs by prefix instead. We can cut our URLs again by slash with cut -d"/". If we want the first prefix, we can do -f1-2, or -f1-3 for the first two prefixes. Let's look!

      cut -d'/' -f1-2 < hits-sorted.txt | uniq -c | sort -nr | head
       100189 /static
         5948 /es
         3069 /player
         2480 /fr
         2476 /es-mx
         2295 /pt-br
         2094 /tr
         1939 /it
         1692 /ru
         1626 /de
      
      cut -d'/' -f1-2 < misses-sorted.txt | uniq -c | sort -nr | head
        66132 /static
        18578 /es
        17448 /player
        17064 /tr
        11379 /fr
         9624 /pt-br
         8730 /es-mx
         7993 /ru
         7689 /zh-hant
         7441 /it
      

      This gives us hit-miss ratios by prefix. Neat, huh?

      13 votes
    12. In which a foolish developer tries DevOps: critique my VPS provisioning script!

      I'm attempting to provision two mirror staging and production environments for a future SaaS application that we're close to launching as a company, and I'd like to get some feedback on the...

      I'm attempting to provision two mirror staging and production environments for a future SaaS application that we're close to launching as a company, and I'd like to get some feedback on the provisioning script I've created that takes a default VPS from our hosting provider, DigitalOcean, and readies it for being a secure hosting environment for our application instance (which runs inside Docker, and persists data to an unrelated managed database).

      I'm sticking with a simple infrastructure architecture at the moment: A single VPS which runs both nginx and the application instance inside a containerised docker service as mentioned earlier. There's no load balancers or server duplication at this point. @Emerald_Knight very kindly provided me in the Tildes Discord with some overall guidance about what to aim for when configuring a server (limit damage as best as possible, limit access when an attack occurs)—so I've tried to be thoughtful and integrate that paradigm where possible (disabling root login, etc).

      I’m not a DevOps or sysadmin-oriented person by trade—I stick to programming most of the time—but this role falls to me as the technical person in this business; so the last few days has been a lot of reading and readying. I’ll run through the provisioning flow step by step. Oh, and for reference, Ubuntu 20.04 LTS.

      First step is self-explanatory.

      #!/bin/sh
      
      # Name of the user to create and grant privileges to.
      USERNAME_OF_ACCOUNT=
      
      sudo apt-get -qq update
      sudo apt install -qq --yes nginx
      sudo systemctl restart nginx
      

      Next, create my sudo user, add them to the groups needed, require a password change on first login, then copy across any provided authorised keys from the root user which you can configure to be seeded to the VPS in the DigitalOcean management console.

      useradd --create-home --shell "/bin/bash" --groups sudo,www-data "${USERNAME_OF_ACCOUNT}"
      passwd --delete $USERNAME_OF_ACCOUNT
      chage --lastday 0 $USERNAME_OF_ACCOUNT
      
      HOME_DIR="$(eval echo ~${USERNAME_OF_ACCOUNT})"
      mkdir --parents "${HOME_DIR}/.ssh"
      cp /root/.ssh/authorized_keys "${HOME_DIR}/.ssh"
      
      chmod 700 ~/.ssh
      chmod 600 ~/.ssh/authorized_keys
      chown --recursive "${USERNAME_OF_ACCOUNT}":"${USERNAME_OF_ACCOUNT}" "${HOME_DIR}/.ssh"

sudo chmod 775 -R /var/www
      sudo chown -R $USERNAME_OF_ACCOUNT /var/www
      rm -rf /var/www/html
      

      Installation of docker, and run it as a service, ensure the created user is added to the docker group.

      sudo apt-get install -qq --yes \
          apt-transport-https \
          ca-certificates \
          curl \
          gnupg-agent \
          software-properties-common
      
      curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
      sudo apt-key fingerprint 0EBFCD88
      
      sudo add-apt-repository --yes \
         "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
         $(lsb_release -cs) \
         stable"
      
      sudo apt-get -qq update
      sudo apt install -qq --yes docker-ce docker-ce-cli containerd.io
      
      # Only add a group if it does not exist
      sudo getent group docker || sudo groupadd docker
      sudo usermod -aG docker $USERNAME_OF_ACCOUNT
      
      # Enable docker
      sudo systemctl enable docker
      
      sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
      sudo chmod +x /usr/local/bin/docker-compose
      sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
      docker-compose --version
      

      Disable root logins and any form of password-based authentication by altering sshd_config.

      sed -i '/^PermitRootLogin/s/yes/no/' /etc/ssh/sshd_config
      sed -i '/^PasswordAuthentication/s/yes/no/' /etc/ssh/sshd_config
      sed -i '/^ChallengeResponseAuthentication/s/yes/no/' /etc/ssh/sshd_config
      

      Configure the firewall and fail2ban.

      sudo ufw default deny incoming
      sudo ufw default allow outgoing
      sudo ufw allow ssh
      sudo ufw allow http
      sudo ufw allow https
      sudo ufw reload
      sudo ufw --force enable && sudo ufw status verbose
      
      sudo apt-get -qq install --yes fail2ban
      sudo systemctl enable fail2ban
      sudo systemctl start fail2ban
      

      Swapfiles.

      sudo fallocate -l 1G /swapfile && ls -lh /swapfile
      sudo chmod 0600 /swapfile && ls -lh /swapfile
      sudo mkswap /swapfile
      sudo swapon /swapfile && sudo swapon --show
      echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
      

      Unattended updates, and restart the ssh daemon.

      sudo apt install -qq unattended-upgrades
      sudo systemctl restart ssh
      

      Some questions

      You can assume these questions are cost-benefit focused, i.e. is it worth my time to investigate this, versus something else that may have better gains given my limited time.

      1. Obviously, any critiques of the above provisioning process are appreciated—both on the micro level of criticising particular lines, or zooming out and saying “well why don’t you do this instead…”. I can’t know what I don’t know.

      2. Is it worth investigating tools such as ss or lynis (https://github.com/CISOfy/lynis) to perform server auditing? I don’t have to meet any compliance requirements at this point.

      3. Do I get any meaningful increase in security by implementing 2FA on login here using google authenticator? As far as I can see, as long as I'm using best practices to actually ssh into our boxes, then the likeliest risk profile for unwanted access probably isn’t via the authentication mechanism I use personally to access my servers.

      4. Am I missing anything here? Beyond the provisioning script itself, I adhere to best practices around storing and generating passwords and ssh keys.

      Some notes and comments

      1. Eventually I'll use the hosting provider's API to spin up and spin down VPS's on the fly via a custom management application, which gives me an opportunity to programmatically execute the provisioning script above and run some over pre- and post-provisioning things, like deployment of the application and so forth.

      2. Usage alerts and monitoring is configured within DigitalOcean's console, and alerts are sent to our business' Slack for me to action as needed. Currently, I’m settling on the following alerts:
        1. Server CPU utilisation greater than 80% for 5 minutes.
        2. Server memory usage greater than 80% for 5 minutes.
        3. I’m also looking at setting up daily fail2ban status alerts if needed.
      9 votes
    13. Deploying containerized Docker instances in production?

      Hello! After spending many development hours in my past years running on Virtualbox/Vagrant-style setups, I've decided to take the plunge into learning Docker, and after getting a few containers...

      Hello! After spending many development hours in my past years running on Virtualbox/Vagrant-style setups, I've decided to take the plunge into learning Docker, and after getting a few containers working, I'm now looking to figure out how to deploy this to production. I'm not a DevOps or infrastructure guy, my bread and butter is software, and although I've become significantly better at deploying & provisioning Linux VPS's, I'm still not entirely confident in my ability to deploy & manage such systems at scale and in production. But, I am now close to running my own business, so these requirements are suddenly going from "nice to have" to "critical".

      As I mentioned, in the past when I've previously developed applications that have been pushed onto the web, I've tended to develop on my local machine, often with no specific configuration environment. If I did use an environment, it'd often be a Vagrant VM instance. From here, I'd push to GitHub, then from my VPS, pull down the changes, run any deployment scripts (recompile, restart nginx, etc), and I'm done.

      I guess what I'm after with Docker is something that's more consistent between dev, testing, & prod, and is also more hands off in the deployment process. Yet, what I'm currently developing still does have differing configuration needs between dev and prod. For example, I'd like to use a hosted DB solution such as DigitalOcean Managed Databases in production, yet I'm totally fine using a Docker container for MySQL for local development. Is something like this possible? Does anyone have any recommendations around how to accomplish this, any do's and dont's, or any catches that are worth mentioning?

      How about automating deployment from GitHub to production? I've never touched any CI/CD tools in my life, yet I know it's a hugely important part of the process when dealing with software in production, especially software that has clients dependent on it to function. Does anything specifically work well with Docker? Or GitHub? Ideally I want to be avoiding manual processes where I have to ssh in, and pull down the latest changes, half-remembering the commands I need to write to recompile and run the application again.

      10 votes
    14. How much testing do you guys do?

      Pretty straight forward question, but basically I was watching a discussion panel the other day talking about the ethics of Self-Driving cars. A topic came up about people writing crappy code, and...

      Pretty straight forward question, but basically I was watching a discussion panel the other day talking about the ethics of Self-Driving cars. A topic came up about people writing crappy code, and more than that, people not testing their code. And if they do, they do point testing. I am in my last semester of uni and I am working with some companies where we are doing pretty extensive testing, happy flows and a lot of alternate flows, as well as UI/UX testing. I wanted to extend this question to you, do you guys do testing, what type? How much do you focus on it? And if u love it/hate it?

      12 votes