• Activity
  • Votes
  • Comments
  • New
  • All activity
  • Showing only topics with the tag "development". Back to normal view
    1. Ideas for a side project I'm working on -- an RPG to help me curb my alcohol consumption

      Preface: I am familiar with Habitica. This idea would probably scratch a similar itch, but I'm also using this as an opportunity to sharpen my Rust skills. My idea came about when I was trying to...

      Preface: I am familiar with Habitica. This idea would probably scratch a similar itch, but I'm also using this as an opportunity to sharpen my Rust skills.

      My idea came about when I was trying to find out some new tactics to curb my alcohol consumption, which isn't quite out of control yet, but I don't want to tempt fate.

      I've also really liked the progression aspect of RPGs. What if I could gamify my quest to not drink alcohol and make it sort of a fun, unique RPG experience at the same time?

      In the broadest sense, it would go something like this:

      • You open the game up, ideally each day. You are instantly prompted: "Did you drink yesterday?" (and perhaps it will go back a few more days if you skipped).
      • For each day you answer "no", you are rewarded with some sort of tokens, credits, etc. -- currency to play the game. If you answer "yes", maybe you get penalized somehow.
      • Then, you pick up your journey, which is sort of a standard RPG experience -- fighting battles, buying gear, learning spells, leveling up, advancing through the world, you name it.
      • The game should get progressively more difficult, but should not have an ending, as "quitting alcohol" does not have an ending either. At the same time, it should scratch the RPG progression itch.

      The initial game concept I came up with is just one that I see as the quickest way to get this off the ground, which would be something CLI-based, where you are presented with a menu ("visit shop, enter arena, view equipment" etc.). You spend battle tokens to enter into arena battles, which reward experience points, money, and gear. You level up, work towards a build (there needs to be a way to respec because restarting isn't really an option), and progress through the arena.

      In total, you would probably spend less than 5 minutes every day playing the game, which is by design. It should be an every day habit. But, there should be enough entertainment value that, if I'm not getting those sweet battle tokens by not drinking, I'm missing out on experiencing the game (or, I could lie, which defeats the purpose of the app).

      So that's where I'm at right now. I'm really interested to hear your thoughts, ideas, critiques, etc. before I spend a free weekend building out a concept.

      Some questions in particular:

      • I was leaning toward just building this in CLI because it will be extremely simple. It could just be a matter of STDINs. However, I'm open to other Rust-based options. Is there a good Rust UI toolkit or web framework that is worth looking into that would make this a little more modern?
      • What about game features? What could make this a really fun experience, while also balancing the whole concept of being built around your life and your habits?

      In the end, this is a deeply personal project that would be built, first and foremost, for my specific needs. But that's not to say I couldn't build it with some scalability in mind. Rather than asking about alcohol, perhaps the "habits" can be customized, and so forth.

      Anyway, have a great weekend!

      23 votes
    2. How do you organize your Linux packages?

      Hello everyone. I am planning to get back into Linux development after working with Mac only for almost a decade. On Mac, one of the most important lessons that I learned was to always use...

      Hello everyone.

      I am planning to get back into Linux development after working with Mac only for almost a decade. On Mac, one of the most important lessons that I learned was to always use Homebrew. Using various package managers (e.g. Homebrew, NPM, Yarn, Pip, etc.) creates situations in which you don't know how to uninstall or upgrade certain pieces of software. Also, it's hard to generate a complete overview.

      How do you Linux folks handle this?

      Bonus question: How do you manage your dotfiles securely? I use Bitwarden, and it's a bit clunky.

      If that helps, I want to try Mint and always use Oh My ZSH!.

      6 votes
    3. Game Development Career Advice

      Hi, I'm curious if anyone in this group has achieved success in game development, whether that's carving out a career or earning any amount of income from it. I'm currently working as a software...

      Hi,

      I'm curious if anyone in this group has achieved success in game development, whether that's carving out a career or earning any amount of income from it.

      I'm currently working as a software developer, but my passion lies in game development. I'm all too aware that achieving any measure of success in this field is next to impossible. Hence, I'm reaching out here, hoping to gather insights and advice from those who have walked this path in the past, or those who are currently walking alongside/behind me.

      One of my specific questions is about the types of games I should focus on creating. Specifically, I've heard differing opinions on whether it's more advantageous to develop a series of small games with advertisements for mobile platforms or to invest in larger, premium games for platforms like Steam. Can anyone share their insights or experiences regarding this dilemma? Is there a clear advantage to one approach over the other?

      Currently I am using godot to make a larger scale game, but I am considering switching to defold and making smaller scale games with ads.

      I saw some folks here discuss making games for the playdate. How much should one consider targeting niche platforms like this? Some of the users I saw discuss this seem to have had good success.

      Some general questions: How did you break into game dev? What were you doing before? Do you see game dev as a viable career, only as a source of side income, or is it just a hobby?

      Any guidance or experiences you can share would be greatly appreciated.

      17 votes
    4. The decline of username and password on the same page

      Web devs: what's up with this trend? For enterprise apps, I get it…single sign-on needs to detect what your email domain is to send you to your identity provider. For consumers, I feel like it's...

      Web devs: what's up with this trend? For enterprise apps, I get it…single sign-on needs to detect what your email domain is to send you to your identity provider. For consumers, I feel like it's gotta be one of these reasons:

      • Users don't know about the tab key being able to move to other fields on a page
      • Mobile users don't really have a tab key, despite there being "previous/next field" arrows on the stock iOS keyboard since its inception (Android users, help me out please)
      • Users tend to hit Enter after typing in their username, leading to a form submission with a blank password
      • Security, maybe? In the past I have sent a link and a password in separate emails or separate communication methods entirely. Are you hashing/salting these separately for better MITM mitigation?

      Did your UX team make a decision? Are my password managers forever doomed to need a "keyboard combo" value for every entry from now on?

      Non-devs: do you prefer one method over the other? If so, why?

      Tildes maintainers: selfishly, thanks for keeping these together :)

      71 votes
    5. Resources and help for setting up a Tildes dev environment

      I've been trying to set up a dev enviornment for Tildes, mainly so that I can actually test my MR (!136), and I've been running into a few issues. However, since we also have a new influx of...

      I've been trying to set up a dev enviornment for Tildes, mainly so that I can actually test my MR (!136), and I've been running into a few issues.

      However, since we also have a new influx of people who might be interested in contributing to Tildes, it seems like a good time to collect resources on setting up the dev environment, as well as helping anyone running into issues.

      So, if you have issues or advice, post them here! I'll be adding my questions in a comment shortly.

      Relevant wiki pages:


      Edit: A more recent post on setting up the dev environment on Apple Silicon / M1 Macs

      36 votes
    6. Just got an Nvidia 4090 GPU, looking for local LLM + general generative AI software recommendations

      I was fortunate enough to grab a discounted 4090 while on my travels and just got everything installed. Already having a lot of fun pumping all my games to max settings, but I'm also interested in...

      I was fortunate enough to grab a discounted 4090 while on my travels and just got everything installed. Already having a lot of fun pumping all my games to max settings, but I'm also interested in running generative AI stuff locally to really take advantage of all that VRAM.

      Do you have any newbie-friendly Windows 11 software to recommend for getting started? Thanks!

      20 votes
    7. Tildes fundraiser June 2023: Encourage an app developer (me) to work on a Tildes app faster, by donating to Tildes (not me)!

      Hey Tildes, with the renewed interest in the site, it got me thinking that we should hold a fundraiser for the not-for-profit company—which currently consists of just one person—that runs Tildes....

      Hey Tildes, with the renewed interest in the site, it got me thinking that we should hold a fundraiser for the not-for-profit company—which currently consists of just one person—that runs Tildes. It's overdue.

      Disclaimer: These are my words as a member of the community. I haven't run this message by the admin before posting. I may have gotten some details wrong.

      Where to donate

      History

      A bit of history: The site admin, @Deimos ran the first three years of the site working full-time on it, paid only by donations, plus a $5000 GitHub sponsor match one year, which I'm not even sure was fully achieved, or only just barely.

      For that time period 2018-2020, a lowball salary as a software engineer with his experience would have been $100,000 USD per year not including benefits.

      If he received $5000 in donations per year (almost certainly an overestimate for more recent years) plus the $5000 GitHub match for the first year—for the 5 years of Tildes' life, that's about $30,000.

      The remaining opportunity cost of $270,000 was essentially paid out of pocket by himself, as a donation to the community. Plus remember there are server expenses, legal incorporation expenses, etc. And, y'know, rent.

      In recent years he had to take a full-time job because the situation was, of course, unsustainable.

      App?

      I announced in April that a mobile app is under development. Originally, I was planning to take my time and release a first alpha by the end of 2023.

      How about if we struck a deal: get the donation numbers up and I will devote more time to the app, as opposed to splitting my time between it and contract work and other projects.

      What's the deal?

      • 150 active donors combined on GitHub Sponsors and Patreon—I'll release an alpha by November.
        GOAL REACHED
      • 300 active donors—I'll release an alpha by October.
        GOAL REACHED
      • 500 active donors—I'll release an alpha by September.

      The dollar amounts don't matter.

      As of writing, we are at 46 active donors.

      What's in it for you, though?

      Feeling like I did a good deed, I guess? I'm not looking for a "slice of the pie," to be clear. In some sense I'd be matching your donations with my time, aka opportunity cost.

      If I donate, can I bother the admin to work more on the site?

      No.

      Again, I haven't run this fundraiser by the admin. He will certainly keep his full-time employment for the foreseeable future, and will not magically have more hours in the day to devote to Tildes.

      With a sustainable budget, though, a lot can happen in the future. Contracting out work to others, for example.

      But the point of this fundraiser is more to make a small dent in the past debt we owe the admin, not making any promises whatsoever on the future of the site and how it's run.

      Let's go, my fellow Tilderinos!

      313 votes
    8. Any Bevy (the rust game engine) users here?

      Bevy just released their version 0.11, so I figured it would be a nice opportunity to ask the Tildes gamedevs if they were using it :) Bevy is a rust game engine - more like a set of libraries...

      Bevy just released their version 0.11, so I figured it would be a nice opportunity to ask the Tildes gamedevs if they were using it :)

      Bevy is a rust game engine - more like a set of libraries actually - that's been gaining popularity the last few years. It has become the de facto toolset if you want to make a game in rust. It is very opinionated towards Entity-Component-System (ECS), and uses the pattern to facilitate parallelism and multi-threading.

      Personally, I'm using the bevy-ecs lib (not the whole engine) to write a roguelike and hone my skills in rust. I enjoy it but it's not really beginner-friendly. The official docs are lacking, and you'll have to dig in the auto-generated api docs to make the most out of it. However, I appreciate that each release not only brings new features, but also refines existing ones. The engine is getting better - not only bigger - release after release.

      16 votes
    9. How to contribute a theme to Tildes

      Want to contribute a theme to Tildes but don't know where to start? Let's fix that. Before we start, get yourself a development environment setup and do a quick read through of the general...

      Want to contribute a theme to Tildes but don't know where to start? Let's fix that.

      Before we start, get yourself a development environment setup and do a quick read through of the general development info to get acquainted with how Tildes works (or at least the HTML and CSS section).

      For this walkthrough I'll be using tildexample as the example name for the theme, but if you decide to contribute a theme for real, make sure it uses the proper name of your theme. :P

      Step 1: Sassy _Sass

      Open the Tildes codebase using your text editor of choice and navigate to the themes directory at tildes/scss/themes. Then create a copy of _default.scss at _tildexample.scss. The default White theme is the canonical source of all colors used, so it's the best place to start from.

      Below is an annotated example of all the things you need to change in your new theme file.

      Annotated example theme
      // Add a small description of the theme here with maybe a link to its website.
      // Check the other themes for examples. https://example.org/tildexample
      
      // Change the theme variable to $theme-tildexample
      // ↓ ↓ ↓ ↓ ↓ ↓ 
      $default-theme: (
        // A whole bunch of color definitions, edit as your theme demands.
        // ...
      );
      
      // Append ".theme-tildexample" to the body selector.
      // ↓ ↙
      body {
        // Don't forget to update the theme variable here too.
        //                  ↓ ↓ ↓ ↓ ↓ ↓ ↓
        @include use-theme($default-theme);
      }
      
      @include theme-preview-block(
        // Change the text to tildexample.
        // ↓ ↓
        "white",
        // And again update the theme variable here.
        //       ↓ ↓ ↓ ↓ ↓ ↓ ↓
        map-get($default-theme, "foreground-primary"),
        map-get($default-theme, "background-primary")
        //       ↑ ↑ ↑ ↑ ↑ ↑ ↑
      );
      

      Once that's done, head to tildes/scss/styles.scss and at the bottom of the file add your theme import:

      @import "themes/tildexample";
      

      Step 2: Hardcoding a TheMe coLor

      Boy that title is a stretch just to say, we need to add 2 lines to the HTML base template.

      Inside the tildes/tildes/templates/base.jinja2 file is a section of if/elif/elif/elif/... statements to set the theme color meta element. Add yourself an elif block and add your theme color.

      For this you probably want to use the background-primary color you used in your theme definition. I've used #ff00dd below because it spells food. I'm such a jokester.

      {% elif request.current_theme == "tildexample" %}
      <meta name="theme-color" content="#ff00dd">
      {% endif %}
      

      Step 3: Snakey Wakey

      Finally the last step is to grab your trusty pungi and give it a blow.

      Head to tildes/tildes/views/settings.py and find the THEME_OPTIONS constant. Here you want to add the theme class you used in body.theme-<this part> and a proper name that will be shown in the theme dropdown.

      THEME_OPTIONS = {
          "white": "White",
          # Many other themes...
          "tildexample": "Tildes Theme Example",
      }
      

      Once that's all been done, check it out in your development site and see if it works.

      Now git!

      Commit. Push. Merge request. Have some water. Deimos reviews, merges and deploys your theme. Job's done.

      26 votes
    10. Where to ask Tildes dev questions?

      If I have a quick Tildes dev question, where should I ask it? For an example that is not actually just an example, but the actual question which drove me to post, where can I found logging output?...

      If I have a quick Tildes dev question, where should I ask it?

      For an example that is not actually just an example, but the actual question which drove me to post, where can I found logging output? Specifically, if I want to print out something every time a request comes in, how should I do so?
      18 votes
    11. Newbie here looking for advice on how to get into Programming/CS by building a project

      Been lurking for a week on tildes now and I am really glad this place exists. The crow here is exactly what I have been missing on Reddit for a while now. Having said that, the whole Reddit...

      Been lurking for a week on tildes now and I am really glad this place exists. The crow here is exactly what I have been missing on Reddit for a while now.

      Having said that, the whole Reddit situation has some-what motivated me to get the balls rolling on an idea that I have had for a while and I am looking for advice on the same.

      I have often heard this phrase "Learn programming by building" but whenever I dive in to the resources, I fall flat due to the information overload and the general abstractness that the field has (I appreciate abstractness but here it demotivates me) and I have never found a proper resource that I could follow to actually build something instead of just blindly following tutorials and playing with them.

      So, my question is how do I translate "learn by building a project" into a practical framework.

      I know of 100 days of swift and I really like that approach however I don't think I want to start with swift or build an iOS app right now.

      24 votes
    12. Tildes dev environment on Apple Silicon // Apple M1

      I'm having some trouble getting the local developer environment set up on Apple hardware, specifically it seems because Virtualbox, which is used as the provider, is not properly functioning on...

      I'm having some trouble getting the local developer environment set up on Apple hardware, specifically it seems because Virtualbox, which is used as the provider, is not properly functioning on Apple hardware.

      Is there anyone here who has managed to get it up and running?

      29 votes
    13. Tildes is still in alpha-testing. It’s an unfinished product. Set your expectations accordingly.

      Someone mentioned elsewhere that they signed up for Tildes “years ago during the beta”. That reminded me: Tildes hasn’t reached beta-testing yet. Officially, Tildes is still in alpha-testing...

      Someone mentioned elsewhere that they signed up for Tildes “years ago during the beta”. That reminded me: Tildes hasn’t reached beta-testing yet.

      Officially, Tildes is still in alpha-testing phase.

      The login page says “Tildes is currently in invite-only alpha...” And the Contact page says “To request an invite to the Tildes alpha...”

      We’re still in alpha-testing. Alpha-testing of software usually happens on an incomplete product before it is released to the customer.

      This is a very important point. Tildes is not feature-complete yet: there are literally hundreds of feature requests yet to work on before Tildes will be what people want it to be – and even that list is far from complete. In Agile software development terms, Tildes is a minimum viable product, or, in other words, “a version of a product with just enough features to be usable by early customers who can then provide feedback for future product development”.

      Tildes works as it is, but it’s a bare-bones forum: you can post, and comment… and that’s about it. It’s a proof of concept. There are a few minor tweaks here and there, which give the impression that Tildes is more complex than it is, but they’re misleading. There are plans to make Tildes a more complex website but, right now, most of that complexity of Tildes exists only in people’s imaginations (and there have been some very imaginative people contributing to that list of future features!).

      Most questions about “Why doesn’t Tildes do X?” or “Can Tildes do X?” can be answered simply by saying “Tildes is incomplete and X hasn’t been built yet.” There are some questions about missing features which can be answered by saying “Tildes was never intended to do X”, but those are far and away in the minority. Most flaws, drawbacks, and problems with Tildes exist because Tildes is still a proof of concept, rather than a finished product.

      It’s also worth noting that Tildes’ current feature set is absolutely not up to the task if the user base and site activity increase too quickly. There’s too much manual tinkering required at the moment to make things work properly: for one thing, there are no significant moderation tools on Tildes (that’s almost all done manually at the moment). There are still a lot of features yet to be built - and we don’t even know what some of those features are yet!

      To pre-empt the people who will rightly point out that Tildes is 5 years old: Tildes’ feature set was intended to grow gradually over time, in line with a gradual growth in users, activity, and the need for those features to exist. However, Tildes has not undergone much growth over the past few years, so the existing features were sufficient to manage the existing activity. Basically, the site didn’t need a lot of fancy features to handle the low traffic here.

      This sudden surge of new users might change that. But it will take time to build more features. That was always the intention, and it hasn’t changed now.

      Until then: Tildes is still in alpha-testing. It’s an unfinished product. Set your expectations accordingly.


      EDIT:

      If you're one of the many people who seem to be replying to this topic, saying "it's okay, I like this simple bare-bones site as it is"... then you're probably not part of my original intended audience for this topic. This topic was aimed at all the people who are arriving here, being taken aback at how simple Tildes is, and wondering where the advanced features are.

      However, we can still take the "set your expectations accordingly" message and apply it to you: "Tildes is an unfinished product, so you can expect it to change in the future. It won't be like this forever."

      Either way, "set your expectations accordingly" is the message here, whether you're expecting more features and not finding them, or whether you're expecting simplicity and enjoying what you see. Either way, you should know that things will change around here. Maybe slowly. Maybe quickly. Maybe they'll get better from your point of view. Maybe they'll get worse from your point of view. But, change they will.

      151 votes
    14. Looking for smallish feature suggestions for an open source project

      I'm thinking about increasing the level of my open source contributions a bit. Instead of searching blindly until I stumble upon an issue that: A) Piques my interest B) I feel somewhat qualified...

      I'm thinking about increasing the level of my open source contributions a bit. Instead of searching blindly until I stumble upon an issue that:
      A) Piques my interest
      B) I feel somewhat qualified to implement

      I figured I'd check with the tildes community. Is there any Open Source software that you use that is missing a feature/capability? Can you give a brief description of it (bonus points for links to an issue tracker with an open ticket :))?

      Can't of course promise anything will come of it, but if I do pick up your suggestion at least I'll give you a ping if I make any progress!

      7 votes
    15. Trying to become a junior developer in Brazil is an uphill battle

      They ask for years of experience, skills that no Jr would know since, well, it is a Jr and the process to apply for jobs are surreal. Thousands of tests, interviews that goes nowhere and lots of...

      They ask for years of experience, skills that no Jr would know since, well, it is a Jr and the process to apply for jobs are surreal. Thousands of tests, interviews that goes nowhere and lots of ghosting. And the pay is not that good. No wonder after 2 or 3 years of experience a lot of develpers starts working for companies outside of Brazil.

      Last one to contact me sent me a test to do it in 1 week. I went above and beyond and learned a lot of things. Before this, i had some small projects in Go and Python. Now i needed to learn Docker, tests, github actions, Postgresql and other things. Not everything was mandatory, but i did my best and did it all. I finished in 5 days since i have a day job.

      Here is the result: https://github.com/crdpa/conservice

      Showing the data in the browser was not necessary, but i think it was a nice touch and well made. If this does not land me a job as a junior developer i don't know what else could.

      I'm glad i already have a job in another area, but me and my SO are separated by a 4 hour drive and i'm tired. I want to work from home to be near her and our dog. Paying rent in two places is becoming a burden.

      I would be happy if you guys could test the application i made. It only needs docker.

      And do you guys have any tips from now on?

      7 votes
    16. Where/how should I acquire a .com domain for three years in advance?

      So I wanna purchase a domain for my personal website (just a WordPress thing), and I wanna pay for three years in advance (I have my reasons). Which domain sellers are reasonably priced,...

      So I wanna purchase a domain for my personal website (just a WordPress thing), and I wanna pay for three years in advance (I have my reasons). Which domain sellers are reasonably priced, trustworthy, and more likely to assist a less technical, non-developer user like myself?

      Thanks!

      13 votes
    17. A few easy linux commands, and a real-world example on how to use them in a pinch

      This below is a summary of some real-world performance investigation I recently went through. The tools I used are installed on all linux systems, but I know some people don't know them and would...

      This below is a summary of some real-world performance investigation I recently went through. The tools I used are installed on all linux systems, but I know some people don't know them and would straight up jump to heavyweight log analysis services and what not, or writing their own solution.

      Let's say you have request log sampling in a bunch of log files that contain lines like these:

      127.0.0.1 [2021-05-27 23:28:34.460] "GET /static/images/flags/2/54@3x.webp HTTP/2" 200 1806 TLSv1.3 HIT-CLUSTER SessionID:(null) Cache:max-age=31536000
      127.0.0.1 [2021-05-27 23:51:22.019] "GET /pl/player/123456/changelog/ HTTP/1.1" 200 16524 TLSv1.2 MISS-CLUSTER SessionID:(null) Cache:

      You might recognize Fastly logs there (IP anonymized). Now, there's a lot you might care about in this log file, but in my case, I wanted to get a breakdown of hits vs misses by URL.

      So, first step, let's concatenate all the log files with cat *.log > all.txt, so we can work off a single file.

      Then, let's split the file in two: hits and misses. There are a few different values for them, the majority are covered by either HIT-CLUSTER or MISS-CLUSTER. We can do this by just grepping for them like so:

      grep HIT-CLUSTER all.txt > hits.txt; grep MISS-CLUSTER all.txt > misses.txt
      

      However, we only care about url and whether it's a hit or a miss. So let's clean up those hits and misses with cut. The way cut works, it takes a delimiter (-d) and cuts the input based on that; you then give it a range of "fields" (-f) that you want.

      In our case, if we cut based on spaces, we end up with for example: 127.0.0.1 [2021-05-27 23:28:34.460] "GET /static/images/flags/2/54@3x.webp HTTP/2" 200 1806 TLSv1.3 HIT-CLUSTER SessionID:(null) Cache:max-age=31536000.

      We care about the 5th value only. So let's do: cut -d" " -f5 to get that. We will also sort the result, because future operations will require us to work on a sorted list of values.

      cut -d" " -f5 hits.txt | sort > hits-sorted.txt; cut -d" " -f5 misses.txt | sort > misses-sorted.txt
      

      Now we can start doing some neat stuff. wc (wordcount) is an awesome utility, it lets you count characters, words or lines very easily. wc -l counts lines in an input, since we're operating with one value per line we can easily count our hits and misses already:

      $ wc -l hits-sorted.txt misses-sorted.txt
        132523 hits-sorted.txt
        220779 misses-sorted.txt
        353302 total
      

      220779 / 132523 is a 1:1.66 ratio of hits to misses. That's not great…

      Alright, now I'm also interested in how many unique URLs are hit versus missed. uniq tool deduplicates immediate sequences, so the input has to be sorted in order to deduplicate our entire file. We already did that. We can now count our urls with uniq < hits-sorted.txt | wc -l; uniq < misses-sorted.txt | wc -l. We get 49778 and 201178, respectively. It's to be expected that most of our cache misses would be in "rarer" urls; this gives us a 1:4 ratio of cached to uncached URL.

      Let's say we want to dig down further into which URLs are most often hitting the cache, specifically. We can add -c to uniq in order to get a duplicate count in front of our URLs. To get the top ones at the top, we can then use sort, in reverse sort mode (-r), and it also needs to be numeric sort, not alphabetic (-n). head lets us get the top 10.

      $ uniq -c < hits-sorted.txt | sort -nr | head
          815 /static/app/webfonts/fa-solid-900.woff2?d720146f1999
          793 /static/app/images/1.png
          786 /static/app/fonts/nunito-v9-latin-ext_latin-regular.woff2?d720146f1999
          760 /static/CACHE/js/output.cee5c4089626.js
          758 /static/images/crest/3/light/notfound.png
          757 /static/CACHE/css/output.4f2b59394c83.css
          756 /static/app/webfonts/fa-regular-400.woff2?d720146f1999
          754 /static/app/css/images/loading.gif?d720146f1999
          750 /static/app/css/images/prev.png?d720146f1999
          745 /static/app/css/images/next.png?d720146f1999
      

      And same for misses:

      $ uniq -c < misses-sorted.txt | sort -nr | head
           56 /
           14 /player/237678/
           13 /players/
           12 /teams/
           11 /players/top/
      <snip>
      

      So far this tells us static files are most often hit, and for misses it also tells us… something, but we can't quite track it down yet (and we won't, not in this post). We're not adjusting for how often the page is hit as a whole, this is still just high-level analysis.

      One last thing I want to show you! Let's take everything we learned and analyze those URLs by prefix instead. We can cut our URLs again by slash with cut -d"/". If we want the first prefix, we can do -f1-2, or -f1-3 for the first two prefixes. Let's look!

      cut -d'/' -f1-2 < hits-sorted.txt | uniq -c | sort -nr | head
       100189 /static
         5948 /es
         3069 /player
         2480 /fr
         2476 /es-mx
         2295 /pt-br
         2094 /tr
         1939 /it
         1692 /ru
         1626 /de
      
      cut -d'/' -f1-2 < misses-sorted.txt | uniq -c | sort -nr | head
        66132 /static
        18578 /es
        17448 /player
        17064 /tr
        11379 /fr
         9624 /pt-br
         8730 /es-mx
         7993 /ru
         7689 /zh-hant
         7441 /it
      

      This gives us hit-miss ratios by prefix. Neat, huh?

      13 votes
    18. In which a foolish developer tries DevOps: critique my VPS provisioning script!

      I'm attempting to provision two mirror staging and production environments for a future SaaS application that we're close to launching as a company, and I'd like to get some feedback on the...

      I'm attempting to provision two mirror staging and production environments for a future SaaS application that we're close to launching as a company, and I'd like to get some feedback on the provisioning script I've created that takes a default VPS from our hosting provider, DigitalOcean, and readies it for being a secure hosting environment for our application instance (which runs inside Docker, and persists data to an unrelated managed database).

      I'm sticking with a simple infrastructure architecture at the moment: A single VPS which runs both nginx and the application instance inside a containerised docker service as mentioned earlier. There's no load balancers or server duplication at this point. @Emerald_Knight very kindly provided me in the Tildes Discord with some overall guidance about what to aim for when configuring a server (limit damage as best as possible, limit access when an attack occurs)—so I've tried to be thoughtful and integrate that paradigm where possible (disabling root login, etc).

      I’m not a DevOps or sysadmin-oriented person by trade—I stick to programming most of the time—but this role falls to me as the technical person in this business; so the last few days has been a lot of reading and readying. I’ll run through the provisioning flow step by step. Oh, and for reference, Ubuntu 20.04 LTS.

      First step is self-explanatory.

      #!/bin/sh
      
      # Name of the user to create and grant privileges to.
      USERNAME_OF_ACCOUNT=
      
      sudo apt-get -qq update
      sudo apt install -qq --yes nginx
      sudo systemctl restart nginx
      

      Next, create my sudo user, add them to the groups needed, require a password change on first login, then copy across any provided authorised keys from the root user which you can configure to be seeded to the VPS in the DigitalOcean management console.

      useradd --create-home --shell "/bin/bash" --groups sudo,www-data "${USERNAME_OF_ACCOUNT}"
      passwd --delete $USERNAME_OF_ACCOUNT
      chage --lastday 0 $USERNAME_OF_ACCOUNT
      
      HOME_DIR="$(eval echo ~${USERNAME_OF_ACCOUNT})"
      mkdir --parents "${HOME_DIR}/.ssh"
      cp /root/.ssh/authorized_keys "${HOME_DIR}/.ssh"
      
      chmod 700 ~/.ssh
      chmod 600 ~/.ssh/authorized_keys
      chown --recursive "${USERNAME_OF_ACCOUNT}":"${USERNAME_OF_ACCOUNT}" "${HOME_DIR}/.ssh"

sudo chmod 775 -R /var/www
      sudo chown -R $USERNAME_OF_ACCOUNT /var/www
      rm -rf /var/www/html
      

      Installation of docker, and run it as a service, ensure the created user is added to the docker group.

      sudo apt-get install -qq --yes \
          apt-transport-https \
          ca-certificates \
          curl \
          gnupg-agent \
          software-properties-common
      
      curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
      sudo apt-key fingerprint 0EBFCD88
      
      sudo add-apt-repository --yes \
         "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
         $(lsb_release -cs) \
         stable"
      
      sudo apt-get -qq update
      sudo apt install -qq --yes docker-ce docker-ce-cli containerd.io
      
      # Only add a group if it does not exist
      sudo getent group docker || sudo groupadd docker
      sudo usermod -aG docker $USERNAME_OF_ACCOUNT
      
      # Enable docker
      sudo systemctl enable docker
      
      sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
      sudo chmod +x /usr/local/bin/docker-compose
      sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
      docker-compose --version
      

      Disable root logins and any form of password-based authentication by altering sshd_config.

      sed -i '/^PermitRootLogin/s/yes/no/' /etc/ssh/sshd_config
      sed -i '/^PasswordAuthentication/s/yes/no/' /etc/ssh/sshd_config
      sed -i '/^ChallengeResponseAuthentication/s/yes/no/' /etc/ssh/sshd_config
      

      Configure the firewall and fail2ban.

      sudo ufw default deny incoming
      sudo ufw default allow outgoing
      sudo ufw allow ssh
      sudo ufw allow http
      sudo ufw allow https
      sudo ufw reload
      sudo ufw --force enable && sudo ufw status verbose
      
      sudo apt-get -qq install --yes fail2ban
      sudo systemctl enable fail2ban
      sudo systemctl start fail2ban
      

      Swapfiles.

      sudo fallocate -l 1G /swapfile && ls -lh /swapfile
      sudo chmod 0600 /swapfile && ls -lh /swapfile
      sudo mkswap /swapfile
      sudo swapon /swapfile && sudo swapon --show
      echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
      

      Unattended updates, and restart the ssh daemon.

      sudo apt install -qq unattended-upgrades
      sudo systemctl restart ssh
      

      Some questions

      You can assume these questions are cost-benefit focused, i.e. is it worth my time to investigate this, versus something else that may have better gains given my limited time.

      1. Obviously, any critiques of the above provisioning process are appreciated—both on the micro level of criticising particular lines, or zooming out and saying “well why don’t you do this instead…”. I can’t know what I don’t know.

      2. Is it worth investigating tools such as ss or lynis (https://github.com/CISOfy/lynis) to perform server auditing? I don’t have to meet any compliance requirements at this point.

      3. Do I get any meaningful increase in security by implementing 2FA on login here using google authenticator? As far as I can see, as long as I'm using best practices to actually ssh into our boxes, then the likeliest risk profile for unwanted access probably isn’t via the authentication mechanism I use personally to access my servers.

      4. Am I missing anything here? Beyond the provisioning script itself, I adhere to best practices around storing and generating passwords and ssh keys.

      Some notes and comments

      1. Eventually I'll use the hosting provider's API to spin up and spin down VPS's on the fly via a custom management application, which gives me an opportunity to programmatically execute the provisioning script above and run some over pre- and post-provisioning things, like deployment of the application and so forth.

      2. Usage alerts and monitoring is configured within DigitalOcean's console, and alerts are sent to our business' Slack for me to action as needed. Currently, I’m settling on the following alerts:
        1. Server CPU utilisation greater than 80% for 5 minutes.
        2. Server memory usage greater than 80% for 5 minutes.
        3. I’m also looking at setting up daily fail2ban status alerts if needed.
      9 votes