• Activity
  • Votes
  • Comments
  • New
  • All activity
  • Showing only topics in ~comp with the tag "homeserver". Back to normal view / Search all groups
    1. Cosmos Cloud Writeup

      I'm just copy pasting my reddit writeup since that's where the creator is active. For those curious the basic idea of cosmos (https://cosmos-cloud.io/) is home server with a push towards default...

      I'm just copy pasting my reddit writeup since that's where the creator is active. For those curious the basic idea of cosmos (https://cosmos-cloud.io/) is home server with a push towards default safety stuff. Reverse proxy over your docker containers configured to not see beyond their world sort of thing so you can safely control access. I believe it's a one person project and still very much in development, but given that so many people just drop "roll your own, you just need to learn...." as the solution I find this to be vastly preferable, and maybe better than things like CasaOS

      Post:
      I've had less time than I hoped to really poke at this, so it's a bit rambly/stream of consciousness. Figured I'd put this up as a data point for anyone either considering cosmos, or maybe as some feedback. If anyone wants more detail on a specific part I'll gladly dive in, but for now if I don't put this up I never will. A very large thanks to the various people who guided me on the discord.

      Techstack/layout/hardware:

      1. Cloudflare domain with proxy active
      2. Ubiquiti UDM Pro router
      3. MS01 on Unbuntu, in default DMZ vlan
      4. Client devices on other vlans(a secure VLAN, technically not the default but similar) or external to network

      Personal skill level: I code for a living, but that's probably overstating my skill. Mostly light CRUD apps. Network is a MASSIVE blindspot that I know very little about. This project was in part to help fix that by getting me some practical experience. It's also GROSSLY overspecc'd for my skill level with some hope I can eventually do some more ambitious stuff.

      Setup: I had installed Cosmos before and run it locally unsecured/self signed (as provided by just clicking on the button in cosmos), just to make sure I understood "intended" behavior.

      My initial hiccups mostly revolved around me setting up port forwarding incorrectly in the router, so i'll skip most of that. Short version is misread something, went down the out of date documentation rabbit hole and then doubled down with some AI hallucinations. In the end it's MUCH easier than I was making it.

      All i needed to do was setup a 443 port forward to the static IP of my Cosmos box. It's even limited to cloudflare IPs only, which was just taking the list provided by cloud flare and copy pasting it in. There's a section in ubiquitis network interface for this and it's very straight forward.

      From there it was configuring the right tokens so I could do the cloudflare DNS Challenge, which is well documented (went the double token route rather than full key.) Once I found the right pages for that it was simple.

      Made my tokens, but was confused as hell because in Comsos it says "you don't need to fill everything out" for cloudflare, and there's CLEARLY duplicate entries, so I wasn't sure if I needed to fill out both.

      From what I can tell, you need to fill out the duplicates (so you will double enter your email and your key/tokens). You can leave blank things like timeouts or whatever you're not using (key if using tokens, token if using key). Some clarity on the dupe thing might help.

      I do think a small guide on bare minimum DNS config would also help. I was using a root A record and a CNAME wildcard record, and I never got it to working with cosmos. Unsure if that's my fault or not, but when I changed the wildcard to another A record (so A record for root and A record for *), it started working. For someone like me who knows fuck all about any of this, there was a lot of stumbling around with DNS.

      Of note I did select allow wildcard domains and .local domains on all attempts. No insecure http local access.

      From there it, mostly, started working. Https enabled and everyone can connect....exceeeept .local domains.

      This is the part i'm still struggling with. There's not a lot of documentation on .local, just "it will work if you check the box". I'm not sure if it clashes with https, or if i need to self sign, or if it really should be that easy.

      My understanding is I just make new url for an app, call it whatever.local, and boom I should be able to connect so long as i'm one the same network.

      In practice, I see no traffic hitting the server when I try this(unless on the server itself), and get timeouts from local clients (server does work). I got it to work once from a client on another vlan after trying to curl the https://whatever.local, but the next morning with nothing changed (went to bed right after and just left the machines running), it no longer worked.

      I did 100% confirm this worked because I used filebrowser to transfer some large data at speeds that NEVER would have been possible if it wasn't over my local network(everything is wired, no wifi, hence the desire for .local access). Also worth noting that I CAN ping the server locally and ssh to it from my other network, so i'm confident the firewall/vlans are configured correctly for that.

      Even for that brief moment when it was working, I STILL couldn't hit domain.local. It clearly exists, but if I can hit it (again from the server box or for that one moment from my other machine) I get the "you should use your domain address" text and cannot continue.

      I suspect router shenanigans (i do have mdns enabled on all VLANS), but I'm having a hard time finding logs and what not for this. I'm also unsure if I don't know enough and am doing some config that obviously shouldn't work. I have toggled the "allow insecure local access" option in testing once or twice, but it doesn't seem to change anything. Not sure how long the delay should be.

      Small things I noticed that might need fixing/expanding: 1. The initial admin account creation "your passwords do not match" help text is not in English. 2. Small thing but while browsing the market it seems there's a few configs that no longer work or aren't supported. EmulatorJS was the main one that seemed clearly done. 3. Hitting the domain, after logging in but not having touched it since forever, just gives you a "user unauthorized" warning but still lets you putter around the setup. 4. Related to that, it does sorta suck that right now even normal users see so much. I would like to hide a LOT of the interface for some of my users(just show them installed visible apps?), and while I can hide something like a new URL, I can't hide the URL screen, or the market, or whatever. It's "fine" but several test members had to be told "yes i know you can see that, no its fine, no you can't delete or edit, yes i know it looks like you can, yes i've tested, etc, etc" 5. In my testing, I did manage to get my domain IP banned by smart shield due to all the logging in and out. Was easy enough to bounce the box and get back in, but maybe a "heavy testing" mode an admin can enable that has smart shield chill for 30 minutes? Dunno how sane that is given the security first focus and I'm sure I could've whitelisted the IP briefly/neutered smart shield somewhere. 6. When entering your license key, you instantly see a "manage your license" button pop up. I emailed about it because I was confused and thought my license was busted, but just needed to scroll to the bottom and hit save. Just a flow thing that might wan to change. 7. Maybe an early "what is your goal" question? Local only vs using a domain vs using a domain and local access with adjusted config process to skip/auto handle things that could go wrong?
      8. The "make admin only" checkbox on every app i've installed, that has it, doesn't appear to work. I have to go into the URL config and manually make it admin only from there. Maybe i'm misunderstanding where/how it's doing this, but some light testing seems to confirm that non admin accounts can access until I do that.

      Side issues:

      At some point in all this my Ubuntu took a spirited attempt at destroying itself and would let me login and then just show me a cursor and nothing else. Couldn't get to the terminal through the recommended ways, but after sshing to the box locally and changing uhh...the display driver I think?, it's mostly been working, but I cannot restart the machine without issues until I hard shutdown (hold the power button). I doubt this is related to cosmos (either caused by, or affecting behavior), but figure I should mention it just in case. Planning a full reinstall later.

      Overall:

      I do love it. Cosmos is trying to be something that I think should exist and yet for some reason does not. There's so many ways to screw something like this up and the "well just roll your own" approach is hellishly easy to screw up with extreme consequences. I have a few more upgrades/tweaks to do (get .local working, maybe reinstall the OS and the thus resetup from scratch, NAS for storage of some family videos/photos we want backed up in more than one spot), and I have mostly enjoyed how clear Cosmos has been.

      7 votes
    2. Home network help part 2, SSH and Server

      Edit: I've made some progress if you want to read the edits at the end. Last year I started slowly planning out a home server setup with help from Tildes. I've gotten a few things up and running,...

      Edit: I've made some progress if you want to read the edits at the end.

      Last year I started slowly planning out a home server setup with help from Tildes. I've gotten a few things up and running, but have been bouncing off a variety of walls trying to get to the next step.

      The first goal was-
      "Ok i've got Cosmos up and running for local access using self signed certs. I'd like to get it up and running using lets encrypt and a domain so I can eventually start giving a few family and friends proper logins and external access". Of note, ideally,

      This led to a second goal of-
      "Gosh it sure would be nice if I didn't have to be sitting at the physical server to do testing and could instead be at another computer in my house. I should probably configure ssh locally (working) and get it to forward windows so I can work in other rooms (not working...)"

      "The stack":

      Server - MS01 running LTS Ubuntu with Cosmos Cloud installed (well it was, but is currently not)

      Router - Ubiquiti Dream Machine Pro (of note i've done some minimal guided config of this to try and harden it at a basic level so my cameras and IoT devices are better isolated. Not fully default, but the server is, for now, in the same network/vlan as the rest of my main computers so don't think this should matter.)

      Clients - All local windows 10/11 machines for now, although in the off off chance it matters, i'm running nushell in the terminal

      Domain Provider - Cloudflare

      The SSH Problems:

      I have a friend who's set SSH up for themselves with their home server, however they haven't had time to come over and troubleshoot. My rough understanding is "setup VcXsrv, change some configs, then it just works.". Windows these days has ssh built in, and I can SSH to the machine just fine with my key.

      ssh -X...less so. I've read some docs, followed some guides, tried copilot, and it all leads to "yeah should work" and it just doesn't. I have configured a ssh config on both machines to allow X11 forwarding, i've started the XLaunch making sure I disable access control, made sure my unbuntu login isn't on wayland and so on. So far, no dice.

      If someone has an end to end guide they trust to link, i'll gladly read and start from scratch. I've been cobbling together so many sources at this point i'm very lost. Lots of things jump quickly to "well just use WSL", which yeah ok i probably should test that next, but I was hoping I wouldn't need to (and am unclear if that'll even help).

      The HTTPS/Domain Problems:

      So..cosmos cloud.

      I like the theory behind this software in that it helps enforce best practices so you don't blow your own head off when you screw something up. Maybe it's not the absolute best starting place, but getting it running without a domain was trivial, and more importantly, shockingly well documented. Not perfect, but for what I understand is mostly a one man show it's better than a lot of professional grade stuff i've dealt with.

      And so I figured it'd be easy to just do the setup from scratch but choose https and point to my domain. There's been two attempts here, no DNS challenge and DNS challenge

      No DNS Challenge Method

      Per their docs it seemed easy enough. I'd never touched a DNS screen before but I configured an A record pointing at my WAN IP (eventually...) and disabled the cloudflare proxy.

      Well going to that domain took me to my router login. Hmm. After screwing around with port forwarding and router DNS records I never got it to work and felt like I was playing with fire, so undid everything I'd done and decided I'd try the DNS challenge. Of note I could still access the cosmos cloud page from http directly to the IP, where it confirmed it failed to get the TLS cert, but https to the domain wasn't having it.

      DNS Challenge Method

      This seemed like I was close, and then nothing. I have no idea if i need to do internal routing on the router for this, it just sorta says "Do the DNS challenge, here's a form, you don't need to fill out all of it" which uh...ok.

      I filled out what I think I needed to after setting up a token(not an API key) in cloudflare. I'm pretty certain I got that correct as I saw text files with keys created on cloudflare's DNS page and had I screwed that I'm guessing it couldn't have.

      However from what I can tell, that's as far as it got. The files nuked themselves 2 minutes later when the TTL expired, and going to the domain locally gave me the cloudflare "our shit's fine, the server is timing out" page. From what I could tell diving into logs, cosmos had the same error, and I couldn't hit cosmos at all, even using the IP and http.

      I do however wonder if maybe it did work BUT since I undid the router DNS record before trying this maybe that killed it? dunno.

      Any ideas?

      That's basically my situation. Figured i'd throw it here and see if anyone has some guidance or troubleshooting they'd recommend. Aforementioned friend who's done some of this before should be free one of these weekends and can probably help, and I haven't tried again since the second attempt. I've thrown some of the questions i've had on the discord and gotten minimal response(although I'm kinda using the thread as a rubber ducking spot as well). Next attempt is probably just DNS challenge again after more research on it and seeing if that works if I put back on the router DNS record, but i feel like logically that shouldn't work.

      Oh also if anyone has some general recommended reading so that I can really understand what the hell it is I'm doing I'd love that. There's a ton of networking books/articles/etc, and in general I'd like to learn more about the subject, but I'm curious if there's a go to for people who are techy and trying to dip their toe in all of it the same way I am and setting up a proper home network and server.

      Edit:
      So after lots of testing, doc reading, and help from the cosmos discord I:

      1. Got the DNS challenge to work according to the cosmos logs.
      2. narrowed down that the main issue was my UDM pro router policies. Needed a firewall rule and a port forward, and had only done one of those at a time in my various attempts and not realized they were really different.

      Now once that was all working and I could hit the site i was getting "likely a false cert" errors, but since i've got all the pieces I'm probably going to try another clean install later and see what we get. Hurrah for troubleshooting, good docs, rubber ducking, and helpful humans.

      Edit 2:

      Eventually required:

      1. Port forward rule in UDM pro
      2. Firewall rule in UDM pro
      3. Static IP and DNS entry in UDM pro.

      One I’d done those things started working. Killed it after that as now I need to think about architecture

      14 votes
    3. A variety of beginner home server questions

      I will soon have a home and figured now's the time to do a proper home server, especially since it's going to come with cat 6 run from the main panel to just about every room. I code for a living,...

      I will soon have a home and figured now's the time to do a proper home server, especially since it's going to come with cat 6 run from the main panel to just about every room. I code for a living, but at the same time network is a massive gap in my knowledge, as are servers, and I was hoping to use this as a learning moment as well as just a way to optimize things. I've been doing research for a few weeks now on and off and feel like I've got more questions than I started with, so I'll just vomit them out and if anyone has some guidance I'd really appreciate it.

      Some information:

      1. I'm willing/able to spend to get quality/simplicity. Time is the much bigger crunch for me right now, and I'd much rather buy something that works even if it costs more than cobbling together some deals.

      2. Related to 1, I'd like this to not become my fulltime second job/hobby. I will at some point try to expand to a full home lab, and do want to use this to learn about things I feel I should understand better for general knowledge and my career, but i'd love for core functionality to mostly "just work" after configuring so when I don't have time to do that I'm not stuck telling everyone "oh yeah it'll be broken until I find time to fix it".

      Things I know I want-

      1. Some sort of NAS. From my research Synology comes up a lot as the "it's expensive but it'll just work" option, and I probably want something like a 4 bay of NAS specific several TB HDD's in something like raid 5/6/10. Pricey as hell but I'm most willing to spend on this as the cost might very well be split by the family members who want me to guinea pig all this.

      2. I will have a camera system and would prefer to not have it sending data outside my network. This is the area i've looked at the least, as it's a little farther down the road, but I know others who have things like Arlo and lets just say i'm not super impressed. Obviously this brings up question like remote access to said camera's and where I'm storing the data (nas? Somewhere else?)

      3. I'd like to mess with a media server. Plex/Jellyfin constantly come up in my research, so I'll be looking into those, but I've also got a bunch of audiobooks that I'd love to be able to easily share, and I think there's software for that stuff as well.

      4. Pihole strikes me as the other "well if you're going to do this, you might as well" option that i'm aware of. Realllly need to better understand networking in general, but I hear these days it can kinda be installed and quickly configured and then left to do its job.

      5. Related to all of this, Casa OS keeps coming up as a very good tool for a beginner like me, since it streamlines the handling of docker containers and also file sharing. However it's not really an OS, since it must actually run on Debian (i think?) for now (zima OS still in testing?).

      Stuff I'd like to mess with but doesn't have to happen right away.

      1. Eventually the aforementioned NAS would be backed up offsite to another NAS at another family members house, once I know what the hell I'm doing.

      2. Proxmox constantly comes up as THE tool to use, but it leaves a lot of questions for me. Obviously if I start trying to do lab environments and screw with VM's it's going to be great, but my understanding is that I probably don't, as a beginner, want to say load up a device with proxmox and then have it host debian which installs CasaOS as it'll get a little more tricky to have everything talk right? Unsure on this part.

      3. Anything else I'm forgetting. One issue I keep having with this is a LOT of the information out there is either too complex for me to really grok or just says "well yeah you could do ANYTHING with this" and it just sorta assumes I know what the options are. If there's anything else worth checking out I'd love to know.

      Hardware I've come across-

      1. Synology - Already mentioned but seems like they're a common go to for a "more money than skill/time" situation like mine.

      2. Zimaboard - My understanding is it's underpowered for its price, but the main draws are that it's VERY low power, small, and quiet. What it could actually do from my list above is where i'm unsure. I see people are supposedly using it for Plex servers and what not, and I'm pretty sure it's not going to make any kick ass lab environments, but being quiet, small, and maybe a bit closer to plug and play seems tempting (I know they make the blade and a few other products but it all seems greek to me).

      3. Various mini computers - I've got a minisforum machine from several years ago that I currently use as a living room computer for light gaming and mostly playing movies and the like. Not sure if i could just wipe it and convert it to be the starting point (more on that later). I know used 1 liter mini pc's from companies like HP are also popular.

      4. The MS-01 - Similar pile as the last one but my understanding is this is the kind of thing that's probably really cool if you actually know what the hell you're doing. I'm 99% positive it is vast overkill for my purposes, but I'd like to eventually get to the point where I could understand why I might want something like this. My understanding is if I knew what I was doing I could probably drop proxmox on this and do everything I could ever want and more, but I feel quite far from that.

      Some general questions I have -

      1. The thing that kicked this all off is my new place likely having fiber, and cat 6 drops throughout the building. Architecture is something I'm still a little shaky on. I assume i'm going to need my own modem/router (just because the cox routers are meh and not really configurable from last I checked), and then that routes to the server first???...or something(seems like a must if you want the pihole to do anything)? I've seen lots of niffty network diagrams at this point but they're all from people WAAAAAAAAY beyond my skill level doing much more ambitious stuff, so it gets hard to understand. If anyone has a simple home network diagram/guide to look at I'd really appreciate it.

      2. I'm just in general going to need to learn more about networking, especially in a home environment. Should I eventually get those camera's set up, I want to understand how to let them talk to internal storage and what not ,but not get out to the web...or..something (again remote access seems nice, but also like a massive security concern). I know speed is also a big factor i'm going to need to better understand. Having a fiber connection in only to be bottle necked by a crappy router or a 1gigabit port is just a waste of money, so that's something else I'd like to better understand.

      3. I'm a little unclear on how to deliver the media in a media server to the various screens throughout the building. I've got cat 6 to all of them, but I suspect i'm still going to need, at the very least, a cheap computer to hook up to it and then display the image to the monitor/TV? This is why I assume I can't just wipe my current mini PC and reuse it as a server, because I still need it to receive the data from the home server (or at least a web browser?). A part of me feels like if I got a powerful enough server it should be able to server the media direction to the screen, but then you'd need some sort of HDMI/DP drops as well from the server to all your screens?...or something?

      Sorry for all the rambling but I've got an odd mix of knowledge and ignorance so it's been a little difficult to research when half the video is stuff I already get, and the other half blows past me or just assumes I know about the parts i'm trying desperately to learn about.

      27 votes
    4. What unified login to use?

      I'm setting up a server with nextcloud, plex, matrix and some other things I don't yet know, for some friends and family, (about 20 people if I get lucky) and now I heard of a thing called single...

      I'm setting up a server with nextcloud, plex, matrix and some other things I don't yet know, for some friends and family, (about 20 people if I get lucky)
      and now I heard of a thing called single sign on/unified login. (Login to different services with the same user/pw and/or login once, access to all services)

      so far I found out about Keycloak https://en.wikipedia.org/wiki/Keycloak

      is this what I'm looking for? does anybody have experience in this? Are there other/better/simpler solutions for this?

      12 votes