47 votes

Tildee — A python library for interacting with Tildes

Update! After a few hours of struggling I managed to set up Read the docs for Tildee, it should help using the library significantly.

After getting some inspiration from TAPS I thought that maybe I try to work on something vaguely similar on my own. And after… some? hours of coding today I came up with this: tildee.py (source)
It's a wrapper for the Tildes Public/Web API that is already used by the site internally to make it work. The obvious problem with that is that it will at one point break when this unstable API is changed. It can do basically all things a normal user can do with the notable exception of applying comment labels (because I haven't gotten around to that yet).

Example of usage for a DM reply bot (result):

import sys
from tildee import TildesClient
import time

# Initialize client and log in, 2FA isn't supported yet and will probably break in horrible ways
t = TildesClient("username", "password", base_url="https://localhost:4443", verify_ssl=False)

while True:
    # Retrieve the "unread messages" page and get a list of the conversations there
    unread_message_ids = t.fetch_unread_message_ids()
    for mid in unread_message_ids:
        # Access the conversation history page; this also clears the "unread" flag 
        conversation = t.fetch_conversation(mid)
        # Get the text of the last message
        text = conversation.entries[-1].content_html
        # Abort if it's from the current user (I don't think this could actually happen)
        if conversation.entries[-1].author == t.username:
            break
        print(f"Found a message by {conversation.entries[-1].author}")
        # If the message contains a reference, reply in kind
        if "hello there" in text.lower():
            print("Replying…")
            t.create_message(mid, f"General {conversation.entries[-1].author}! You are a bold one.")
        # Delay before processing next unread message
        time.sleep(3)
    # Delay before next unread check
    time.sleep(60)

This has a lot of potential. Haven't yet figured out potential for what, but I'll take what I can get.
I'd be really grateful if someone with a little more experience than me (that's not exactly a high bar :P) could give me some pointers on the project's structure and the "API design", hence the ask tag. Other creative ideas for what to use this for are appreciated, too.

23 comments

  1. [2]
    Silbern
    Link
    Nice job! :) One potential I see for this is allowing the use of clients other than the web version. For example, with a Python backend, it's be relatively easy to write a CLI frontend to handle...

    Nice job! :) One potential I see for this is allowing the use of clients other than the web version. For example, with a Python backend, it's be relatively easy to write a CLI frontend to handle the display and user input, so it'd be possible to interact via a terminal instead of needing a graphical web browser for the web client.

    19 votes
    1. mrbig
      Link Parent
      Tildes on Emacs would be awesome! And considering emacsers are over represented right now, it may really happen.

      Tildes on Emacs would be awesome! And considering emacsers are over represented right now, it may really happen.

  2. [16]
    Wes
    Link
    Very cool. An API is instrumental in creating tooling around a project. And as much as I prefer a mobile-browser version to dedicated clients, it's nice to have that option as well. I just hope...

    Very cool. An API is instrumental in creating tooling around a project. And as much as I prefer a mobile-browser version to dedicated clients, it's nice to have that option as well.

    I just hope nobody tries creating auto-posting bots ala reddit.

    5 votes
    1. [16]
      Comment deleted by author
      Link Parent
      1. [15]
        Wes
        Link Parent
        I was actually thinking more along the lines of comment bots. Reddit is flooded with them, and I was just hoping the same doesn't happen here. I think topic-posting bots could be okay within...

        I was actually thinking more along the lines of comment bots. Reddit is flooded with them, and I was just hoping the same doesn't happen here.

        I think topic-posting bots could be okay within reason. They'd probably need to be to specific groups so they're opt-in for readers. eg. there's a HN subreddit that just crossposts HN submissions. It's a pretty dead subreddit, though.

        5 votes
        1. [14]
          Deimos
          Link Parent
          It won't. I'll ban any bots that don't contribute anything useful and just clutter up threads like all the ones on reddit.
          • Exemplary

          Reddit is flooded with them, and I was just hoping the same doesn't happen here.

          It won't. I'll ban any bots that don't contribute anything useful and just clutter up threads like all the ones on reddit.

          10 votes
          1. [13]
            Algernon_Asimov
            Link Parent
            "Useful" is in the eye of the beholder, though. Some people would argue that a bot which corrects misspellings is useful. Some people would argue that a bot which posts mobile versions of web...

            "Useful" is in the eye of the beholder, though. Some people would argue that a bot which corrects misspellings is useful. Some people would argue that a bot which posts mobile versions of web links, or full versions of mobile links, is useful. Some people would argue that a bot which posts the first paragraph of a linked article is useful. And so on. I've seen these arguments made on Reddit.

            So, who decides what's "useful"?

            For my own personal purposes, I've never seen a bot which is useful to me as a general user of Reddit. There are exactly three bots I've seen on Reddit which were useful, but they were useful to me only in my role as a moderator.

            1 vote
            1. [10]
              cfabbro
              (edited )
              Link Parent
              IMO a lot of the problems with bots on reddit is simply due to reddit's inherent limitations and lack of features, not with the bots themselves. E.g. RemindMeBot can be incredibly useful to users,...

              IMO a lot of the problems with bots on reddit is simply due to reddit's inherent limitations and lack of features, not with the bots themselves. E.g. RemindMeBot can be incredibly useful to users, but is essentially forced to be annoying because the only means reddit provides to interact with it is via comments which get treated the same as every other comment which leads to it creating noise in threads. However if reddit had a CLI (command line interface) or aside comment system, then interacting with it wouldn't lead to nearly as much noise and it wouldn't be so annoying.

              And that's not the only useful bot I am aware of either. There are also a bunch of useful twitter, mirror, archive and link bots on reddit as well. E.g. /r/SCP's Marvin.

              5 votes
              1. [9]
                Algernon_Asimov
                Link Parent
                If there are bots which work directly bot-to-user in private, then I won't see them and I won't care. But keep them out of public threads (we're talking about comment bots here, not private bots)....

                E.g. RemindMeBot can be incredibly useful to users, but is essentially forced to be annoying because the only means reddit provides to interact with it is via comments

                If there are bots which work directly bot-to-user in private, then I won't see them and I won't care. But keep them out of public threads (we're talking about comment bots here, not private bots).

                There are also a bunch of useful twitter, mirror, archive and link bots

                That's exactly what I was talking about: bots which clutter up threads with information that's contained in the post itself. It's duplication and clutter.

                1 vote
                1. [8]
                  cfabbro
                  Link Parent
                  If someone on the other end deletes a particular submission (e.g. deletes their tweet) or the site goes down, then "the information" is no longer "contained in the post itself", which is the...

                  If someone on the other end deletes a particular submission (e.g. deletes their tweet) or the site goes down, then "the information" is no longer "contained in the post itself", which is the entire point of archive and mirror bots... and link bots like Marvin exist entirely because the information, in that case a hyperlink to the relevant SCP article, do not exist in the post.

                  It's duplication and clutter.

                  In your opinion. In mine those bots can and often do provide much needed backup, and can make things easier and more convenient for users, if done well.

                  5 votes
                  1. [3]
                    Comment deleted by author
                    Link Parent
                    1. [2]
                      cfabbro
                      Link Parent
                      From a follow up comment further down the thread.

                      From a follow up comment further down the thread.

                      The archive/mirror bots on reddit generally don't copy the content onto reddit itself, they typically archive using screenshots and host them on other sites like imgur... so I don't imagine they would operate any differently on Tildes. And yes, they do potentially violate a person's right to delete their data, but sometimes a backup can also serve the public good, especially when the person is a public figure or the reason the content is gone is not because of intentional deletion but because of a technical difficulty. So as long as the backup isn't being done for malicious purposes and doesn't break any Canadian laws, I think the benefit of allowing them to exist potentially outweighs the downsides. Trust people, but punish abusers.

                      1. [2]
                        Comment deleted by author
                        Link Parent
                        1. cfabbro
                          Link Parent
                          Ah, I see. And yeah, I would agree with that. I don't think mirror or backup bots should be allowed for Tildes specific content either. E.g. If a topic or comment is deleted, it should remain that...

                          Ah, I see. And yeah, I would agree with that. I don't think mirror or backup bots should be allowed for Tildes specific content either. E.g. If a topic or comment is deleted, it should remain that way. I was mostly referring to mirroring and backing up of 3rd party sites submitted here via bots as having potential value.

                          1 vote
                  2. [5]
                    Algernon_Asimov
                    Link Parent
                    And here we come back to my original point: "useful" is in the eye of the beholder. There's no objective standard for when a bot's usefulness outweighs its clutter.

                    In your opinion.

                    And here we come back to my original point: "useful" is in the eye of the beholder. There's no objective standard for when a bot's usefulness outweighs its clutter.

                    1. [4]
                      cfabbro
                      (edited )
                      Link Parent
                      And again, IMO the only reason why they produce clutter on reddit is because reddit basically forces them to, and they're also extra annoying there because blocking them is a PITA, as it can only...

                      And again, IMO the only reason why they produce clutter on reddit is because reddit basically forces them to, and they're also extra annoying there because blocking them is a PITA, as it can only be done on an individual basis after encountering one in the wild. However, if on Tildes bots were required to be formally registered as bots and were clearly distinguished as such, with a mechanism allowing users to easily opt-out of ever seeing any (or a particular one), and also provided alternate means for the bots to make comments and users to interact with them, they wouldn't produce clutter or be as annoying here.

                      You dislike bots... and that's fine. So now imagine if you could just tick a box in your Tildes user settings to never have to ever see one here. Or alternatively, if you could go to ~tildes.bots and scroll down a list of all the active bots on the site to opt out of seeing particular ones. That's basically what I am suggesting should be possible here. And by going that route, rather than banning them all, others such as myself who often do find the bots helpful can still have them available to us. That solution to the problem of bots also fits nicely into the "Let users make their own decisions about what they want to see" goal of Tildes.

                      4 votes
                      1. [3]
                        Algernon_Asimov
                        Link Parent
                        Would that box also stop me seeing the inevitable off-topic replies that bot-comments produce? Would bots reply to my comments/posts for everyone else to see, while I remain ignorant of what the...

                        So now imagine if you could just tick a box in your Tildes user settings to never have to ever see one here.

                        Would that box also stop me seeing the inevitable off-topic replies that bot-comments produce? Would bots reply to my comments/posts for everyone else to see, while I remain ignorant of what the bots are saying about my comments/posts? Would I miss conversations because a bot replies to my comment/post, and someone replies to the bot but not to me? Or would I still see all that bot-triggered clutter, but without the context of the bot-comment that produced it?

                        if you could go to ~tildes.bots and scroll down a list of all the active bots on the site to opt out of seeing particular ones.

                        There had better be a "Select All" option on that page, because I have never seen a bot that added anything of value to a discussion. Ever. As I've often said, bots are a blight on Reddit - and I see no reason to infect Tildes with the same disease.


                        By the way... the bot you mentioned, which copies content from other sites onto Tildes and thereby retains a copy of that off-site content here even after it's deleted from the other site... that contradicts the content owner's right to delete their own content if they want to.

                        1 vote
                        1. [2]
                          cfabbro
                          (edited )
                          Link Parent
                          All the problems you're bringing up about blocking bots apply equally to blocking other user accounts as well, and they are not unsolvable. I imagine so, yes. I imagine no, you would not. Similar...

                          All the problems you're bringing up about blocking bots apply equally to blocking other user accounts as well, and they are not unsolvable.

                          Would that box also stop me seeing the inevitable off-topic replies that bot-comments produce?

                          I imagine so, yes.

                          Or would I still see all that bot-triggered clutter, but without the context of the bot-comment that produced it?

                          I imagine no, you would not. Similar to how noise hides all the replies, blocking a bot would probably act similarly but also hide the initial comment as well.

                          Would bots reply to my comments/posts for everyone else to see, while I remain ignorant of what the bots are saying about my comments/posts? Would I miss conversations because a bot replies to my comment/post, and someone replies to the bot but not to me?

                          Yes, I imagine you would be ignorant of any replies like that, although I can certainly see the merits of the argument that if you block a bot they should not be able to reply to you, so as to avoid all that potential mess.

                          There had better be a "Select All" option on that page...

                          There likely would be, or even a blanket opt-out setting in the user settings.

                          By the way... the bot you mentioned, which copies content from other sites onto Tildes and thereby retains a copy of that off-site content here even after it's deleted from the other site... that contradicts the content owner's right to delete their own content if they want to.

                          Huh? The archive/mirror bots on reddit generally don't copy the content onto reddit itself, they typically archive using screenshots and host them on other sites like imgur... so I don't imagine they would operate any differently on Tildes. And yes, they do potentially violate a person's right to delete their data, but sometimes a backup can also serve the public good, especially when the person is a public figure or the reason the content is gone is not because of intentional deletion but because of a technical difficulty. So as long as the backup isn't being done for malicious purposes and doesn't break any Canadian laws, I think the benefit of allowing them to exist potentially outweighs the downsides. Trust people, but punish abusers.

                          I have never seen a bot that added anything of value to a discussion. Ever. As I've often said, bots are a blight on Reddit - and I see no reason to infect Tildes with the same disease.

                          Yeah, well many people think the opposite, myself included. So how about instead of constantly taking such hardline stances against things all the time, you instead try to help come up with solutions that can accommodate both sides? You don't want bots on Tildes... we get that... but lots of people do, so if they were allowed here what features or bot related policies could Tildes provide that would make the situation more tolerable for you?

                          p.s. Rather coincidentally, a mirror bot could have been useful a bit earlier today:
                          https://tildes.net/~music/f28/richard_cheese_people_equals_shit_slipknot_cover#comment-3j90

                          1 vote
                          1. Algernon_Asimov
                            (edited )
                            Link Parent
                            Because I don't want bots on Tildes, so I see no reason to assist people to put bots on Tildes. Why would I help people do something that I see as bad? Bots must be opt-in. Bots must work only by...

                            So how about instead of constantly taking such hardline stances against things all the time, you instead try to help come up with solutions that can accommodate both sides?

                            Because I don't want bots on Tildes, so I see no reason to assist people to put bots on Tildes. Why would I help people do something that I see as bad?

                            if they were allowed here what features or bot related policies could Tildes provide that would make the situation more tolerable for you?

                            Bots must be opt-in. Bots must work only by direct request from a user. Bots must not post merely in response to an event on Tildes. Someone must go out of their way to summon a bot before it does anything. Where possible, bots should work in private, without posting public comments. If public comments are absolutely necessary, then bots must reply only to the person who summoned them. Bot comments can not be replied to. EDIT: Bot comments must be automatically labelled "Noise", so they are collapsed by default.

                            1 vote
            2. [2]
              unknown user
              Link Parent
              Bots are a patch over a lack of functionality in the core system, as @cfabbro explained. For example: Fixed via community post editing a la Stack Overflow. Alternative resource detection and...

              Bots are a patch over a lack of functionality in the core system, as @cfabbro explained. For example:

              Some people would argue that a bot which corrects misspellings is useful.

              Fixed via community post editing a la Stack Overflow.

              Some people would argue that a bot which posts mobile versions of web links, or full versions of mobile links, is useful.

              Alternative resource detection and multiple links per Tildes submission fix this one.

              Some people would argue that a bot which posts the first paragraph of a linked article is useful.

              We have this already! :) Bots aren't useful. Their functionality is. The bot implementation itself is a shim over the primary issue.

              1 vote
              1. Algernon_Asimov
                Link Parent
                I agree. If there's a need for an automated feature, it should be built into Tildes itself, and not implemented as an external bot - with the proviso that these automated features should not...

                I agree. If there's a need for an automated feature, it should be built into Tildes itself, and not implemented as an external bot - with the proviso that these automated features should not produce comments in threads. That's clutter. Automated features must do their job without creating clutter.

                1 vote
  3. [5]
    Algernon_Asimov
    Link
    So, for the benefit of this non-coder... You've just created an interface so people can write bots for Tildes. Is that right?

    So, for the benefit of this non-coder...

    You've just created an interface so people can write bots for Tildes.

    Is that right?

    1 vote
    1. [4]
      Sybil_Fleming
      Link Parent
      Not just bots, but as @Silbern pointed out, any kind of third party client is possible (mobile apps, desktop apps, autoposting bots, etc).

      Not just bots, but as @Silbern pointed out, any kind of third party client is possible (mobile apps, desktop apps, autoposting bots, etc).

      11 votes
      1. [3]
        Algernon_Asimov
        Link Parent
        Thanks for explaining. To be honest, I didn't understand @Silbern's comment.

        Thanks for explaining. To be honest, I didn't understand @Silbern's comment.

        3 votes
        1. [2]
          hamstergeddon
          Link Parent
          Basically what @Silbern was saying is that you could use it to create a command line based interface for tildes. Some folks (especially programmers) really like doing things entirely from the...

          Basically what @Silbern was saying is that you could use it to create a command line based interface for tildes. Some folks (especially programmers) really like doing things entirely from the keyboard and/or through the command line (like DOS or 80s home computers). Definitely something I'd enjoy playing around with, but not likely something I'd use very often. Although on the upside it would look like I was doing work while I was goofing off on tildes :P

          3 votes
          1. Emerald_Knight
            Link Parent
            Often times it's not that programmers necessarily like doing things from their keyboard, but that we like being able to automate tedious tasks or tasks we're likely to forget. A command line...

            Often times it's not that programmers necessarily like doing things from their keyboard, but that we like being able to automate tedious tasks or tasks we're likely to forget. A command line interface is perfect for this because you can usually "pipe" (feed) output from a command into a completely different command as input. You can build entirely new tools on top of other tools by making them work together. There are also some tasks that you just can't accomplish from a web browser, forcing you to use the command line instead, so having a CLI that you can work with is really handy for avoiding the need to switch back and forth between the command line and the web browser.

            But, let's use a practical example, because that's more tangible. For this specific API, you could build something that posts a new topic on a schedule. For example, if Tildes had a ~sudoku group, then you could have a tool that generates a random sudoku puzzle every day, starting with the easiest difficulty on Sunday and gradually increasing in difficulty until reaching the hardest difficulty on Saturday, and have the sudoku problem submitted for you. Maybe you have a bunch of ideas for submissions to a particular group and you don't want to flood that group with those submissions all at once, so you store each idea in a separate file in a specific folder and you create a tool that grabs one of those files at random and submits its contents. Maybe you just want to delete your entire comment history (without deleting your account), but you don't want to deal with all of the clicking that entails, so you instead write a quick script that goes through your comment history automatically and deletes everything for you.

            The possibilities are practically endless :)

            2 votes