14 votes

What programming/technical projects have you been working on?

This is a recurring post to discuss programming or other technical projects that we've been working on. Tell us about one of your recent projects, either at work or personal projects. What's interesting about it? Are you having trouble with anything?

10 comments

  1. Pioneer
    Link
    Not programming. But I've been getting my head around Databricks, Lakehouses and Data Vault architecture recently. My new role uses all of these and they have my head spinning with a million...

    Not programming.

    But I've been getting my head around Databricks, Lakehouses and Data Vault architecture recently. My new role uses all of these and they have my head spinning with a million possiblities.

    5 votes
  2. [7]
    public
    Link
    As one of my tiamasomo 2023 goals, I want to turn my rPi4 into a streaming server. If anyone has tips to break analysis paralysis, that would be massively appreciated. I keep making zero progress...

    As one of my tiamasomo 2023 goals, I want to turn my rPi4 into a streaming server. If anyone has tips to break analysis paralysis, that would be massively appreciated. I keep making zero progress due to researching too much.

    Here are the completed steps and bolded decisions I expect in the process.

    1. Format external 4TB HDD (chose EXT4 b/c it was the default, and I don't think most Linux file systems are meaningfully different until you're using multiple discs)
    2. Set up a network drive. SMB or NFS? The clients are my Macs. I don't expect I'll need Windows compatibility.
    3. Set up dynamic DNS on one of the many domains I've hoarded.
    4. Copy files over to the drive
    5. Install Navidrome
    6. Configure Navidrome
    7. Is a video media manager fulfilling the following requirements? If not, I'll default to Jellyfin.
      1. It's just a bunch of folders, not some smart library view
      2. Can be configured for differing access for different users
    8. Set up a VPN? Somehow, I need to link the ddns I configured to these services. Is this the most secure way?

    Really could use some pointers on my overall plan here.

    4 votes
    1. [5]
      Comment deleted by author
      Link Parent
      1. [3]
        public
        Link Parent
        Thank you for your thorough response. Now I've got my template. I should've mentioned the two motivating factors for this project in my OP: Apple's extortionate SSD pricing—not only does my music...
        1. Thank you for your thorough response. Now I've got my template.
        2. I should've mentioned the two motivating factors for this project in my OP:
          1. Apple's extortionate SSD pricing—not only does my music collection not fit an iPhone (at least without using iTunes' "convert to 128k .aac" option), it'd also overflow the base tier SSD on new Macs.
          2. A substantial minority of my music is not in iPhone-friendly formats (mostly Ogg Opus and FLAC), so I want Navidrome (or Jellyfin) to handle conversion & streaming. I can be my own Spotify.
        3. Is the order of steps I presented reasonable?

        ext4 served over NFS

        Works for me. IIRC, NFS will auto-mount on Macs while SMB needs to be manually reconnected each time.

        DNS host

        For the domains I want to use for streaming (thinking of setting up one for Navidrome and one for Jellyfin), Pork Bun; for most of the rest, Cloudflare. I believe they both have dDNS clients.

        why VPN?

        So I can offload excess files while on the road. I'm planning on some extended travel (again), so I'd like to be able to connect and move files over to the NFS drive.

        Need VPS accessible by others?

        No, just me on two iPhones and two Macs (and perhaps an iPad in the future). If I set it up for just me and later change my mind, is this a scrap it and start over change or merely adding another user to the list of authorized accounts?

        1 vote
        1. [3]
          Comment deleted by author
          Link Parent
          1. public
            Link Parent
            As always, thanks for your informative answers. I hope Tildes is indexed so someone else can learn from you correcting my ignorance. Neat, setting this up is like a slightly faster Tailscale but...

            As always, thanks for your informative answers. I hope Tildes is indexed so someone else can learn from you correcting my ignorance.

            Wireguard

            Neat, setting this up is like a slightly faster Tailscale but without a SAAS in between.

            static DHCP reservation

            Already have that set up for the PiHole. Should I also set those up for my Macs? As I understand it, NFS works better with authorized IPs than addresses like public-macbook.local. The Macs also get two 192.168.whatever addresses if they're connected to both Wi-Fi and ethernet—would that make a difference for assigning reservations on the local network?

            checkpoints

            Sounds like my order of operations is

            1. Set up NFS (inc. assigning static DHCP to my Macs, if needed)
            2. Copy files over
            3. Set up basic Navidrome/Jellyfin, for testing
            4. Configure and check Wireguard (see your list of checkpoints). I presume a properly configured Wireguard means that my VNC would also start working here.
            5. Configure Navidrome+Jellyfin for "regular" access (a.k.a. login w/ user+pass over the internet), but this step is optional for October.
          2. public
            Link Parent
            I presume this is to make it unambiguous which is the overlay and which is the local network—useful to keep devices not on the wireguard network separate even when on the local physical network....

            these are very intentionally different from the 192.168.0 addresses above. I can go into more detail about this if you want.

            I presume this is to make it unambiguous which is the overlay and which is the local network—useful to keep devices not on the wireguard network separate even when on the local physical network. Is that right?

            I've gotten a bit stuck getting NFS properly setup. When I try to connect, it times out.

      2. UP8
        Link Parent
        Don’t mess with new filesystems, the risks are real but the benefits aren’t. That said, my media server has a two disk ZFS array with 2 14TB disks, it is certain the disks will be out of warrantee...

        Don’t mess with new filesystems, the risks are real but the benefits aren’t. That said, my media server has a two disk ZFS array with 2 14TB disks, it is certain the disks will be out of warrantee before they get filled.

    2. [2]
      Shmiggles
      Link Parent
      If you're not using Windows, stick to NFS for directory sharing - it's just easier. If you're just using this to get the files onto the Pi, forget NFS and just rsync them over. Jellyfin is great -...

      If you're not using Windows, stick to NFS for directory sharing - it's just easier. If you're just using this to get the files onto the Pi, forget NFS and just rsync them over.

      Jellyfin is great - it's a nice, slick Netflix-style UI. It can also handle TV tuners, if you get the Pi TV hat.

      I expose my Jellyfin install to the Internet, with no VPN, so my parents can access it. Jellyfin has password-protected user accounts, so that's fine for my needs.

      1. public
        Link Parent
        Setting that up to work remotely isn't much different than ensuring I can SSH remotely [setup DDNS first]. Even if I'm copying off the server, I can rsync. I was thinking of NFS b/c I do prefer a...

        NFS / just rsync

        Setting that up to work remotely isn't much different than ensuring I can SSH remotely [setup DDNS first]. Even if I'm copying off the server, I can rsync. I was thinking of NFS b/c I do prefer a graphical environment for file management.

        I presume the VNC can also work remotely once DDNS is set up. I want to use that so I can do things like log into Firefox on the Pi and directly download my Bandcamp purchases instead of downloading them to my Mac, then transferring over [or ripping discs].

  3. CuriosityGobble
    Link
    I've been learning perl. It's a weird fun language.

    I've been learning perl. It's a weird fun language.

    2 votes
  4. UP8
    Link
    3-sided cards: For each design “card”, each “print” has been getting a unique URL for a few weeks, now there is a sqllite database for each card that keeps track of the prints and has a sequence...

    3-sided cards:

    For each design “card”, each “print” has been getting a unique URL for a few weeks, now there is a sqllite database for each card that keeps track of the prints and has a sequence number that gets incremented if a print is made successfully, otherwise the print is treated as defective or a work print. The back side visible here is “print #344.2” and is the second card to have a sequence number.

    https://mastodon.social/@UP8/111168528593969946

    There is likely to be a public database so I can do things like give somebody a business card and register a portrait I take against it, or let people register cards and trade them like NFTs (no money though!)

    I updated my stereogram developer to output side-by-side stereo pairs in addition to the red-cyan anaglyphs I make because people on dpreview are asking for them. I am planning on stepping up stereogram production so there will be more updates in this area, I really need something to adjust exposures, crop, etc also I am still trying to produce stereograms with a light-field camera, the Lytro Desktop isn’t cutting it, and I may need to look at open source plentopic processing software.

    Blog: no progress

    YOShInOn RSS reaser: I have a long list of “little” things like an edit dialog for the HN submission queue, a submission queue for Mastodon (as opposed to immediate posts) but the thing that is looming is reworking the ingestion system so I can keep track which feed every item has come from, the processing of tearing the ingestion system down and rebuilding it could be a hassle so I am avoiding it, but a bunch of features I want will become easier to code up once I get it done so I should “just do it.”