11 votes

What programming/technical projects have you been working on?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

18 comments

  1. DMBuce
    Link
    I used python's transliterate package to write a simple transliteration script, then used that script to build a rofi-based menu that can easily convert text to look π”Šπ”¬π”±π”₯𝔦𝔠 or π’Έπ“Šπ“‡π“ˆπ’Ύπ“‹β„― or...

    I used python's transliterate package to write a simple transliteration script, then used that script to build a rofi-based menu that can easily convert text to look π”Šπ”¬π”±π”₯𝔦𝔠 or π’Έπ“Šπ“‡π“ˆπ’Ύπ“‹β„― or π–ΌΜΆπ—‹ΜΆπ—ˆΜΆπ—ŒΜΆπ—ŒΜΆπ–ΎΜΆπ–½ΜΆ π—ˆΜΆπ—ŽΜΆπ—ΜΆ or uʍop-ǝpᴉsdn or π•π•šπ•œπ•– π•₯π•™π•šπ•€, for example.

    6 votes
  2. [4]
    ali
    (edited )
    Link
    After 5 weeks of writing I handed my masters thesis last night. The sub area was natural language processing (just woke up when I first wrote this) Some more info: I wanted to write a classifier...

    After 5 weeks of writing I handed my masters thesis last night. The sub area was natural language processing (just woke up when I first wrote this)

    Some more info:
    I wanted to write a classifier to extract object information from natural language answers. That tool is supposed to be used for explanatory learning, to get neural networks to be β€šright for the right reasonsβ€˜

    6 votes
    1. Grendel
      Link Parent
      Congratulations! Finishing your Masters is such a huge accomplishment.

      Congratulations! Finishing your Masters is such a huge accomplishment.

      3 votes
    2. [2]
      joplin
      Link Parent
      Sweet! Congratulations on getting it done!

      Sweet! Congratulations on getting it done!

      1 vote
      1. ali
        Link Parent
        Thank you and @grendel It was such a huge challenge for me. Especially the writing and then, after working on it for months it feels like "did I really do anything worthwhile?" Because you compare...

        Thank you and @grendel

        It was such a huge challenge for me. Especially the writing and then, after working on it for months it feels like "did I really do anything worthwhile?"
        Because you compare yourself to all this novel research you see all the time.
        I’m just glad it’s over to be honest.
        First break I have had in a long time

        2 votes
  3. teaearlgraycold
    Link
    Currently in the middle of a re-write of our web app at work. The new app is built as a full-stack TypeScript monorepo. Coming from the Rails world it's really nice being able to share code...

    Currently in the middle of a re-write of our web app at work. The new app is built as a full-stack TypeScript monorepo. Coming from the Rails world it's really nice being able to share code between the front-end and the back-end. I have it set up so that the front-end can import a module from the back-end that defines a set of types for all of the HTTP endpoints. So when you write a fetch call for the browser you have compile-time guarantees on the path, HTTP method, POST body, query parameters and response structure. This means when you refactor the back-end API you'll immediately get compiler errors on the front end telling you what you need to update.

    I'm also able to share all POST body validators between the two. Basically - it's impossible for the two to be out of sync if you use the types correctly.

    5 votes
  4. [2]
    rogue_cricket
    (edited )
    Link
    At work it's "innovation week', which is a much less fun version of a hackathon. Regardless, I finally (finally!!) have the opportunity to make up a proof-of-concept that we can do our data...

    At work it's "innovation week', which is a much less fun version of a hackathon. Regardless, I finally (finally!!) have the opportunity to make up a proof-of-concept that we can do our data analytics services in-house rather than using the third-party analytics and visualization solution we have been using. Our flexible analytics is one of the big draws of our product - we store a lot of information, but our data is just inherently complex. We want to give our customers access to this information, and we want people to be able to generate their own reports and metrics for the stuff that matters to them - it needs to be easy-to-use and performant.

    We've tried a couple off-the-shelf solutions for this but none of them have really worked. Chiefly the issue has been slow performance or bugs or just a straight-up inability to customize to the degree we need to. But also these solutions are just... not fun to work with.

    So what I'm working on right now is a proof-of-concept for a denormalized data warehouse. The performance should be better than our current solution by a factor of like, literally hundreds. Originally the plan was just to create a basic interface for this data, but the front-end team wanted to do "something" with Dart/Flutter for their innovation week and the three of us on back-end duty were like, well, why not some data visualization? So it's going to be a good-looking demo, hopefully!

    The other two people on my team (one of whom is an actual SQL wizard) are handling the data query and testing its correctness, and I'm translating everything into Clojure (our existing webserver stuff is 90% written in Clojure, which has easy access to JDBC, running as it does on the JVM) and providing the API for the front-end team. I think the other two will finalize their queries today so I should get some help on that soon.

    I'm honestly super excited to be writing in Clojure again! It's been a while so I'm a bit rusty, but it's coming back quickly. I think writing reducers is just fun. :)

    5 votes
    1. rogue_cricket
      Link Parent
      As an update to this: our goal was to time the return of about 300,000 rows of data in a particular structure to compare with an issue in the 3rd party solution where a customer wants about that...

      As an update to this: our goal was to time the return of about 300,000 rows of data in a particular structure to compare with an issue in the 3rd party solution where a customer wants about that much data. The third party solution just fails at this and I don't think we can do much else to help, honestly. We've already optimized what we can in the queries we give it given the table structure. It takes 10 minutes and times out.

      With the new structure, we've got the same information coming back now in about 10 seconds on our first go. This is just using a denormalized table in our existing database and our first pass at a query, not even a dedicated database instance with more appropriate settings.

      If the execs don't love this and let us keep working on it eventually I'm going to have a full fit! It is literally at least six hundred times faster and we will be able to filter it more flexibly. Not to mention right now we can't do any kind of migration on a user-generated report, which causes all kinds of issues.

      Feels vindicating that we're not crappy at our jobs. We just have an unreasonable product and structure to work with.

      4 votes
  5. [5]
    skybrian
    Link
    More 3D printing with flex materials. It's trickier than PLA and there seems to be a lot of trial and error involved, but I'm getting the hang of it.

    More 3D printing with flex materials. It's trickier than PLA and there seems to be a lot of trial and error involved, but I'm getting the hang of it.

    3 votes
    1. [4]
      joplin
      Link Parent
      Does "flex materials" mean flexible materials, or just a different type of material?

      Does "flex materials" mean flexible materials, or just a different type of material?

      1. [3]
        skybrian
        Link Parent
        It does mean flexible materials and they are a different type of material, sometimes called TPU for "thermoplastic polyurethane" or "flex" for short. There are a variety of filaments that you can...

        It does mean flexible materials and they are a different type of material, sometimes called TPU for "thermoplastic polyurethane" or "flex" for short. There are a variety of filaments that you can buy to print rubber-like objects. They vary in hardness and the softer ones are harder to print with since they tend to result in stringing or clogging the extruder. Generally speaking you have to print slower, so it takes longer.

        (I am trying Ninjatek Cheetah, which they claim is one of the easier ones.)

        3 votes
  6. [2]
    joplin
    Link
    I'm working on Truchet Tiles. I had created a rectangular Truchet tile renderer years ago, but I never bothered with hexagonal tiles. I always wanted to, but never had the time. Well, now I do,...

    I'm working on Truchet Tiles. I had created a rectangular Truchet tile renderer years ago, but I never bothered with hexagonal tiles. I always wanted to, but never had the time. Well, now I do, thanks to a handy little tutorial on hexagonal tiling from Red Blob Games. I've got a basic renderer up and running with hexagonal tiles, but they're not yet patterned. Hopefully, I'll get that working tonight.

    2 votes
    1. JRandomHacker
      Link Parent
      The Red Blob hex-grid articles are a shining example of an educational/informative website.

      The Red Blob hex-grid articles are a shining example of an educational/informative website.

      2 votes
  7. Artemix
    Link
    I've been refining my barebones git repository manager daemon, which aims to provide an interface similar to gitweb in simplicity, allowing to create / mirror repositories, edit their description,...

    I've been refining my barebones git repository manager daemon, which aims to provide an interface similar to gitweb in simplicity, allowing to create / mirror repositories, edit their description, category, default HEAD, and such.

    1 vote
  8. dedime
    (edited )
    Link
    I made a website scraper at work. The company is working on migrating our client's websites to a new platform, using an external development agency. We wanted an easy way to provide the agency...

    I made a website scraper at work. The company is working on migrating our client's websites to a new platform, using an external development agency. We wanted an easy way to provide the agency with a list of pages that a website has, and a list of images that page has. My scraping tool, given a site's base URL, will reach out to that site, fetch the sitemap.xml, parse all the available URLs, visit each one and save all the images to a named folder on the local filesystem. We'll then zip that folder up and send it over to the agency.

    I used golang to accomplish this. It's just an absolute pleasure to work with. It's extremely fast, and adding concurrency as an afterthought seriously took about 5 minutes.

    1 vote
  9. DMBuce
    (edited )
    Link
    Not exactly a "project" but I just set up uBlock Origin to block Tildes comments on non-text posts except for top-level comments by the OP. The CSS is simple enough but took a while to figure out,...

    Not exactly a "project" but I just set up uBlock Origin to block Tildes comments on non-text posts except for top-level comments by the OP. The CSS is simple enough but took a while to figure out, so for anyone else who's interested:

    ##.topic-full-link ~ .topic-comments article:not(.is-comment-by-op)
    ##article:not(.topic-with-excerpt) .topic-info-comments-new
    ##article:not(.topic-with-excerpt) .topic-info-comments > *