16 votes

Hosting my own Tildes

Is there a guide to hosting Tildes? The docs site has development setup but I couldn't get anywhere with that (not a Python developer)

37 comments

  1. clerical_terrors
    Link
    I would recommend against hosting unstable software in production, if you do not fully understand how it works. Because that's going to mean you won't be able to fix any kind of problem yourself....

    I would recommend against hosting unstable software in production, if you do not fully understand how it works. Because that's going to mean you won't be able to fix any kind of problem yourself. Remember that Tildes is in it's Alpha version (i.e. not stable nor feature complete) and despite it running surprisingly smoothly you can expect some level of technical troubles.

    24 votes
  2. [10]
    Comment deleted by author
    Link
    1. [9]
      haykam821
      Link Parent
      I'm triyng to get an actual production environment so a few people can chat around in it.

      I'm triyng to get an actual production environment so a few people can chat around in it.

      4 votes
      1. [8]
        orangse
        Link Parent
        Not trying to be "that guy", but why not just use another piece of software designed for an end user? It would be way easier to just set up a server using Discord or a chatroom with Signal. Or...

        Not trying to be "that guy", but why not just use another piece of software designed for an end user? It would be way easier to just set up a server using Discord or a chatroom with Signal. Or even a private subreddit, if you need the link aggregation setup.

        12 votes
        1. [6]
          hungariantoast
          (edited )
          Link Parent
          I'd recommend Discourse over all of those proprietary systems. Not only does it not require any sort of app or extra software, but it's free and open source software as well. Plus, if OP doesn't...

          I'd recommend Discourse over all of those proprietary systems. Not only does it not require any sort of app or extra software, but it's free and open source software as well.

          Plus, if OP doesn't want to self host, they have options for that.

          EDIT: Signal is free and open source software, and I'm a goof who can't use search engines. It can also be self hosted.

          22 votes
          1. [2]
            Crestwave
            Link Parent
            Isn't Signal also FOSS? You also don't need to self-host it, although it doesn't have a website client.

            Isn't Signal also FOSS? You also don't need to self-host it, although it doesn't have a website client.

            9 votes
            1. hungariantoast
              Link Parent
              Hm, it totally is. I actually looked it up when I wrote that comment just to make sure, but obviously didn't look close enough. I'll update my comment to reflect reality.

              Hm, it totally is. I actually looked it up when I wrote that comment just to make sure, but obviously didn't look close enough. I'll update my comment to reflect reality.

              5 votes
          2. [3]
            haykam821
            Link Parent
            It doesn't have threaded comments from what I've seen.

            It doesn't have threaded comments from what I've seen.

            2 votes
            1. [2]
              Crestwave
              Link Parent
              Do you need nested comment threads like here or just one level? Mattermost has the latter.

              Do you need nested comment threads like here or just one level? Mattermost has the latter.

              8 votes
              1. haykam821
                Link Parent
                I was looking for nested comment threads.

                I was looking for nested comment threads.

        2. haykam821
          Link Parent
          Because that's not self-hosted.

          Because that's not self-hosted.

          2 votes
  3. [25]
    haykam821
    Link
    I mentioned this to @Deimos as well but I never was able to find out how.

    I mentioned this to @Deimos as well but I never was able to find out how.

    3 votes
    1. [24]
      Amarok
      Link Parent
      Sometime I think we'll have a guide for how to build a Tildes site, but as it's still in alpha and all that info is in Deimos' head at the moment (and only there) I doubt it's high on the priority...

      Sometime I think we'll have a guide for how to build a Tildes site, but as it's still in alpha and all that info is in Deimos' head at the moment (and only there) I doubt it's high on the priority list for him.

      6 votes
      1. [23]
        Deimos
        (edited )
        Link Parent
        No, I do actually have a document written up that's a list of everything that needs to be done to get it running on a fresh server. I could just paste it for you if you want, @haykam821, but I'd...

        No, I do actually have a document written up that's a list of everything that needs to be done to get it running on a fresh server. I could just paste it for you if you want, @haykam821, but I'd agree with the other people here that it's probably not a great idea if you're not quite experienced with administration-type tasks.

        I'm sure you'd be able to get it running, but it wouldn't be a complete list of everything you need to do to set up the server, and I don't have explanations for how to do a bunch of general administration tasks that you would need to.

        For example:

        • Are you familiar with how to secure a Linux server, including doing things like configuring user account permissions, firewall, etc.?
        • Are you familiar enough with SQL to be able to connect to the database, examine the tables and run update statements? The only way to do things like set a user as an admin is to connect to the database and run an UPDATE to give them the correct permissions.
        • Are you familiar with git to be able to add the site's repo as a remote, and know how to pull down the code, update it after I make future changes, etc.?
        • If a service fails to launch or crashes for some reason, do you know to use systemd to figure out what's wrong and restart it?

        (There's certainly more than that too, the overall point is that it's very much not a "turnkey" system)

        20 votes
        1. [22]
          hungariantoast
          Link Parent
          If you wouldn't mind sharing it, I'd love to see it. Some of the administration tasks you mentioned are things I've never done, but it sounds like an excellent opportunity to learn. Plus, I too...

          If you wouldn't mind sharing it, I'd love to see it. Some of the administration tasks you mentioned are things I've never done, but it sounds like an excellent opportunity to learn.

          Plus, I too have kind of, sorta, maybe thought about the idea of using a fork of Tildes for a discussion forum about a specific subject, like a video game that I've always wanted to make.

          10 votes
          1. [20]
            Deimos
            (edited )
            Link Parent
            Here's what's in the document. As I said, these should be most of the steps to solely get Tildes to work on a fresh Ubuntu 16.04 server, but doesn't cover all the other things like setting up...
            • Exemplary

            Here's what's in the document. As I said, these should be most of the steps to solely get Tildes to work on a fresh Ubuntu 16.04 server, but doesn't cover all the other things like setting up permissions correctly, ssh security, firewall, etc. Most are the actual commands to run, but comment lines (starting with #) are descriptions of something that needs to be done but not the specific commands to do it.

            adduser tildes
            usermod -aG sudo tildes
            
            # clone tildes and tildes-static-sites repos into /home/tildes
            
            sudo ln -s /home/tildes/tildes/tildes /opt/tildes
            
            sudo ln -s /home/tildes/tildes/salt/salt /srv/salt
            sudo ln -s /home/tildes/tildes/salt/pillar /srv/pillar
            
            sudo ln -s /home/tildes/tildes-static-sites/output /opt/tildes-static-sites
            
            sudo mkdir /etc/salt
            sudo cp /home/tildes/tildes/salt/minion /etc/salt/minion
            
            # edit /etc/salt/minion and change id to "prod"
            
            wget -O - https://repo.saltstack.com/apt/ubuntu/16.04/amd64/latest/SALTSTACK-GPG-KEY.pub | sudo apt-key add -
            echo 'deb http://repo.saltstack.com/apt/ubuntu/16.04/amd64/latest xenial main' | sudo tee /etc/apt/sources.list.d/saltstack.list
            sudo apt-get update
            sudo apt-get install salt-minion
            sudo systemctl stop salt-minion
            sudo systemctl disable salt-minion
            
            # copy /opt/tildes/production.ini.example to production.ini and replace placeholders
            
            sudo add-apt-repository ppa:certbot/certbot
            sudo apt-get update
            sudo apt-get install python-certbot-nginx
            
            sudo certbot certonly --manual -d tildes.net -d *.tildes.net -d tild.es -d *.tild.es --preferred-challenges dns-01 --server https://acme-v02.api.letsencrypt.org/directory
            
            sudo salt-call --local state.highstate
            

            And these are commands that usually need to be done after every time you pull down new code updates:

            sudo salt-call --local state.apply -l info
            alembic upgrade head
            sudo systemctl reload gunicorn.service
            
            18 votes
            1. [4]
              adrianbblk
              Link Parent
              There's an error with the nginx service journalctl -xe Sep 01 10:29:13 AMS-148750 nginx[30716]: nginx: configuration file /etc/nginx/nginx.conf test failed Sep 01 10:29:13 AMS-148750 systemd[1]:...

              There's an error with the nginx service

              journalctl -xe
              Sep 01 10:29:13 AMS-148750 nginx[30716]: nginx: configuration file /etc/nginx/nginx.conf test failed
              Sep 01 10:29:13 AMS-148750 systemd[1]: nginx.service: Control process exited, code=exited status=1
              Sep 01 10:29:13 AMS-148750 systemd[1]: Failed to start A high performance web server and a reverse proxy server.
              -- Subject: Unit nginx.service has failed
              -- Defined-By: systemd
              -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
              -- 
              -- Unit nginx.service has failed.
              -- 
              -- The result is failed.
              Sep 01 10:29:13 AMS-148750 systemd[1]: nginx.service: Unit entered failed state.
              Sep 01 10:29:13 AMS-148750 systemd[1]: nginx.service: Failed with result 'exit-code'.
              Sep 01 10:29:13 AMS-148750 sudo[30709]: pam_unix(sudo:session): session closed for user root
              Sep 01 10:29:14 AMS-148750 sshd[30707]: Failed password for invalid user wa from 35.202.27.205 port 46046 ssh2
              Sep 01 10:29:14 AMS-148750 sshd[30707]: Received disconnect from 35.202.27.205 port 46046:11: Bye Bye [preauth]
              Sep 01 10:29:14 AMS-148750 sshd[30707]: Disconnected from 35.202.27.205 port 46046 [preauth]
              Sep 01 10:29:32 AMS-148750 sshd[30728]: Invalid user david from 145.239.169.177
              Sep 01 10:29:32 AMS-148750 sshd[30728]: input_userauth_request: invalid user david [preauth]
              Sep 01 10:29:32 AMS-148750 sshd[30728]: pam_unix(sshd:auth): check pass; user unknown
              Sep 01 10:29:32 AMS-148750 sshd[30728]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=145.239.169.177
              Sep 01 10:29:34 AMS-148750 sshd[30728]: Failed password for invalid user david from 145.239.169.177 port 1781 ssh2
              Sep 01 10:29:34 AMS-148750 sshd[30728]: Received disconnect from 145.239.169.177 port 1781:11: Bye Bye [preauth]
              Sep 01 10:29:34 AMS-148750 sshd[30728]: Disconnected from 145.239.169.177 port 1781 [preauth]
              
              systemctl status nginx.service
              ● nginx.service - A high performance web server and a reverse proxy server
                 Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
                 Active: failed (Result: exit-code) since Sun 2019-09-01 10:32:53 CEST; 9s ago
                Process: 30614 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)
                Process: 30817 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=1/FAILURE)
               Main PID: 12526 (code=exited, status=0/SUCCESS)
              
              Sep 01 10:32:53 AMS-148750 systemd[1]: Starting A high performance web server and a reverse proxy server...
              Sep 01 10:32:53 AMS-148750 nginx[30817]: nginx: [emerg] getpwnam("nginx") failed in /etc/nginx/nginx.conf:1
              Sep 01 10:32:53 AMS-148750 nginx[30817]: nginx: configuration file /etc/nginx/nginx.conf test failed
              Sep 01 10:32:53 AMS-148750 systemd[1]: nginx.service: Control process exited, code=exited status=1
              Sep 01 10:32:53 AMS-148750 systemd[1]: Failed to start A high performance web server and a reverse proxy server.
              Sep 01 10:32:53 AMS-148750 systemd[1]: nginx.service: Unit entered failed state.
              Sep 01 10:32:53 AMS-148750 systemd[1]: nginx.service: Failed with result 'exit-code'.
              
              1. [3]
                unknown user
                Link Parent
                The nginx user is missing. Whether this is because it should exist and doesn't, or because nginx is configured incorrectly, I can't say.
                nginx: [emerg] getpwnam("nginx") failed in /etc/nginx/nginx.conf:1
                

                The nginx user is missing. Whether this is because it should exist and doesn't, or because nginx is configured incorrectly, I can't say.

                1. adrianbblk
                  Link Parent
                  There was a problem with the ssl certificate, was missing. not Its working.

                  There was a problem with the ssl certificate, was missing. not Its working.

                  1 vote
                2. [2]
                  Comment deleted by author
                  Link Parent
                  1. Deimos
                    Link Parent
                    They're set up in the SaltStack "pillar" (kind of like global config): https://gitlab.com/tildes/tildes/blob/master/salt/pillar/prod.sls That file should be in /srv/pillar if you set up the...

                    They're set up in the SaltStack "pillar" (kind of like global config): https://gitlab.com/tildes/tildes/blob/master/salt/pillar/prod.sls

                    That file should be in /srv/pillar if you set up the symlinks shown above, and after editing it you'll need to run salt again to update the files:

                    sudo salt-call --local state.apply -l info
                    
                    1 vote
            2. [15]
              adrianbblk
              Link Parent
              How to create the admin user? Or what it is ? 😁

              How to create the admin user? Or what it is ? 😁

              1. [14]
                Deimos
                Link Parent
                Connect to the database and run UPDATE users SET permissions = '"admin"' WHERE username = 'whatever';

                Connect to the database and run UPDATE users SET permissions = '"admin"' WHERE username = 'whatever';

                2 votes
                1. [3]
                  adrianbblk
                  Link Parent
                  some help to connect to the database, the whole installation was almost automatically :D And yeah a server noob here too :D Thanks a lot 🙏

                  some help to connect to the database, the whole installation was almost automatically :D
                  And yeah a server noob here too :D

                  Thanks a lot 🙏

                  1. [2]
                    Deimos
                    Link Parent
                    psql -U tildes tildes will open a command-line client to the database.

                    psql -U tildes tildes will open a command-line client to the database.

                    2 votes
                    1. adrianbblk
                      Link Parent
                      Amazing. Thanks. Is there a way to open the registrations without and invite? 🤔

                      Amazing. Thanks.

                      Is there a way to open the registrations without and invite? 🤔

                2. [10]
                  laurcrisst
                  Link Parent
                  Hello, This line UPDATE users SET permissions = '"admin"' WHERE username = 'whatever'; isn't working on my side. UPDATE 0 Also how to set a password for the admin user?

                  Hello, This line UPDATE users SET permissions = '"admin"' WHERE username = 'whatever'; isn't working on my side. UPDATE 0 Also how to set a password for the admin user?

                  1. [9]
                    Deimos
                    Link Parent
                    You have to change the 'whatever' to a username that exists. It will be easiest to register the admin user through the site itself, and change its password through the site instead of trying to do...

                    You have to change the 'whatever' to a username that exists. It will be easiest to register the admin user through the site itself, and change its password through the site instead of trying to do it manually.

                    1. [8]
                      laurcrisst
                      Link Parent
                      I see, but I'm still a bit confused. Tildes registration is not open, so I'm unable to create a new user. There's already a user "tildes" into the database, I make it the admin, but how to login...

                      I see, but I'm still a bit confused.
                      Tildes registration is not open, so I'm unable to create a new user. There's already a user "tildes" into the database, I make it the admin, but how to login if I don't know the password and it cannot be changed from the database side since it is using the hash method?

                      1. [7]
                        Deimos
                        Link Parent
                        You shouldn't be using the "Tildes" user, that's intended for attributing actions performed automatically by the site, such as automatically-scheduled topics. You should create a new user and make...

                        You shouldn't be using the "Tildes" user, that's intended for attributing actions performed automatically by the site, such as automatically-scheduled topics. You should create a new user and make it an admin.

                        1 vote
                        1. [6]
                          laurcrisst
                          Link Parent
                          How to create a new user? I should have to insert the new user directly into the database? It is safe?

                          How to create a new user? I should have to insert the new user directly into the database? It is safe?

                          1. [5]
                            Deimos
                            Link Parent
                            You can use pshell to do it, as shown here (ignore the parts related to the group if you don't also want to add a group):...

                            You can use pshell to do it, as shown here (ignore the parts related to the group if you don't also want to add a group): https://docs.tildes.net/instructions/development-setup#set-up-a-custom-user-and-group-optional

                            1. [4]
                              adrianbblk
                              Link Parent
                              don't waste your time. pshell does not work on ubuntu, at least I wasn't able to make it works.

                              don't waste your time. pshell does not work on ubuntu, at least I wasn't able to make it works.

                              1. [3]
                                Deimos
                                Link Parent
                                You were probably trying to run it outside of the Vagrant VM, instead of ssh-ing in and running it inside.

                                You were probably trying to run it outside of the Vagrant VM, instead of ssh-ing in and running it inside.

                                1 vote
                                1. [2]
                                  adrianbblk
                                  Link Parent
                                  No, I did that from the Vagrant VM Here's what the terminal print (tildes) vagrant@ubuntu-xenial:/opt/tildes$ pshell development.ini Traceback (most recent call last): File...

                                  No, I did that from the Vagrant VM

                                  Here's what the terminal print

                                  (tildes) vagrant@ubuntu-xenial:/opt/tildes$ pshell development.ini
                                  Traceback (most recent call last):
                                    File "/opt/venvs/tildes/bin/pshell", line 10, in <module>
                                      sys.exit(main())
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/pyramid/scripts/pshell.py", line 22, in main
                                      return command.run()
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/pyramid/scripts/pshell.py", line 142, in run
                                      self.env = self.bootstrap(config_uri, options=config_vars)
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/pyramid/paster.py", line 111, in bootstrap
                                      app = get_app(config_uri, options=options)
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/pyramid/paster.py", line 30, in get_app
                                      return loader.get_wsgi_app(name, options)
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/plaster_pastedeploy/__init__.py", line 125, in get_wsgi_app
                                      return loadapp(
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 253, in loadapp
                                      return loadobj(APP, uri, name=name, **kw)
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 275, in loadobj
                                      context = loadcontext(
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 299, in loadcontext
                                      return _loaders[scheme](
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 326, in _loadconfig
                                      return loader.get_context(object_type, name, global_conf)
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 457, in get_context
                                      context = self._context_from_use(
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 480, in _context_from_use
                                      context = self.get_context(
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 410, in get_context
                                      return loadcontext(object_type, name,
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 299, in loadcontext
                                      return _loaders[scheme](
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 334, in _loadegg
                                      return loader.get_context(object_type, name, global_conf)
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 624, in get_context
                                      entry_point, protocol, ep_name = self.find_egg_entry_point(
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/paste/deploy/loadwsgi.py", line 645, in find_egg_entry_point
                                      pkg_resources.require(self.spec)
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
                                      needed = self.resolve(parse_requirements(requirements))
                                    File "/opt/venvs/tildes/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
                                      raise DistributionNotFound(req, requirers)
                                  pkg_resources.DistributionNotFound: The 'tildes' distribution was not found and is required by the application
                                  (tildes) vagrant@ubuntu-xenial:/opt/tildes$ 
                                  
                                  
                                  1 vote
          2. Amarok
            Link Parent
            I'd like to see this. Part of the attraction of making Tildes open source is the satisfaction of seeing what uses others find for the code.

            I'd like to see this. Part of the attraction of making Tildes open source is the satisfaction of seeing what uses others find for the code.

            7 votes
  4. [2]
    adrianbblk
    Link
    The login says "Unknown error" Any idea what could cause this error?

    The login says "Unknown error"
    Any idea what could cause this error?

    1. Soptik
      Link Parent
      Try to look into logs, everything should be there. You might be able to see in which part of the source code is the issue. After examining source code, you should be able to understand what went...

      Try to look into logs, everything should be there. You might be able to see in which part of the source code is the issue. After examining source code, you should be able to understand what went wrong and how to fix it.

      2 votes