41 votes

Scraping Tildes as an exercise and statistics

TLDR: Album of all 3 graphs

Edit 3: Added some more graphs in my top level comment here.

Hi all, over the past few days I've slowly been working on a scraper for Tildes with the goal of getting some statistics and as an exercise in learning Puppeteer. I may or may not put the source for it online later but it's surprisingly easy to do (so maybe you can learn too, wink wink nudge nudge). Also, I'm not sure whether I want to publish the full data file as I want to respect the privacy of the people, I don't know. You let me know.

On to the specifics of what I'm scraping then, currently the only things are the topics in the topic listing, which give me:

export interface ITopic {
  author: string;   // Author of the topic
  comments: number; // Amount of comments the topic has
  date: Date;       // Date the topic was posted
  group: string;    // Group the topic was posted in
  id: string;       // The base 36 ID of the topic
  link: string;     // A link to the topic, primarily for easier Markdown formatting
  tags: string[];   // Tags of the topic, currently not doing anything with them
  title: string;    // Title of the topic, also not using this for anything
  type: TopicTypes; // The type of topic, ie: Link or Text
  votes: number;    // Amount of votes the topic has
}

And now that the boring stuff is out of the way, actual data:

Topics

Line graph of topics per week and line graph of topics per month

Year Week # of Topics - Year Month # of Topics
2018 17 119 - 2018 4 42
2018 18 143 - 2018 5 1306
2018 19 188 - 2018 6 2045
2018 20 451 - 2018 7 1440
2018 21 440 - 2018 8 2016
2018 22 831 - 2018 9 1405
2018 23 429 - 2018 10 1370
2018 24 403 - 2018 11 940
2018 25 298 - 2018 12 728
2018 26 281 - 2019 1 334
2018 27 241 -
2018 28 336 -
2018 29 406 -
2018 30 418 -
2018 31 506 -
2018 32 484 -
2018 33 479 -
2018 34 350 -
2018 35 345 -
2018 36 345 -
2018 37 354 -
2018 38 305 -
2018 39 306 -
2018 40 289 -
2018 41 326 -
2018 42 286 -
2018 43 324 -
2018 44 231 -
2018 45 207 -
2018 46 243 -
2018 47 180 -
2018 48 173 -
2018 49 199 -
2018 50 159 -
2018 51 150 -
2018 52 152 -
2019 1 143 -
2019 2 106 -

Users

This table only lists users with more than 100 topics, otherwise it would be far too big. All votes listed are only votes received on topics.

User # of Topics # of Votes Most Topics In Group (Topics) Most Votes In Group (Votes)
Algernon_Asimov 728 5863 news (288) news (1828)
Catt 280 2829 news (40) talk (440)
Deimos 884 15837 tech (225) tildes.official (6385)
EightRoundsRapid 472 3068 news (146) news (977)
EscReality 128 1066 food (41) food (328)
Neverland 165 1911 news (79) news (922)
Whom 119 1179 music (87) music (864)
asteroid 169 1316 tech (50) tech (365)
boredop 117 862 music (52) music (321)
cfabbro 275 1725 music (71) music (326)
dubteedub 513 6263 news (151) news (2120)
ducks 107 1411 misc (23) tech (285)
jmillikin 134 1585 news (30) news (351)
patience_limited 124 1350 tech (19) tech (236)
rkcr 127 1406 misc (29) tech (323)
spit-evil-olive-tips 386 3926 news (66) news (735)

Groups

The top users' numbers here are how many topics/votes they have in that specific group. And of course Deimos is listed at the bottom with the highest number of topics/votes.

Graph of this table, excluding top users.

Group # of Topics # of Votes # of Comments Top User (Topics) Top User (Votes)
anime 116 974 877 Cleb (19) Cleb (193)
books 239 2203 2012 Catt (19) cadadr (211)
comp 671 8890 6150 Deimos (41) Deimos (626)
creative 259 2586 1294 Bishop (49) Bishop (364)
enviro 252 2757 1136 dubteedub (42) dubteedub (523)
food 239 2096 1502 spit-evil-olive-tips (45) spit-evil-olive-tips (368)
games 891 10914 9038 Deimos (193) Deimos (2230)
health 174 1635 1062 Catt (26) Catt (238)
hobbies 85 1100 1403 spit-evil-olive-tips (3) RedditRefugee (48)
humanities 254 2291 1206 Algernon_Asimov (97) Algernon_Asimov (796)
lgbt 173 1919 1124 Algernon_Asimov (61) Algernon_Asimov (535)
life 245 3291 2324 Catt (27) Catt (319)
misc 884 9286 5332 EightRoundsRapid (69) dubteedub (855)
movies 263 2331 1722 dubteedub (42) dubteedub (381)
music 1295 7676 5029 EightRoundsRapid (134) Whom (864)
news 1700 18115 8817 Algernon_Asimov (288) dubteedub (2120)
science 519 4952 1551 Algernon_Asimov (66) Algernon_Asimov (556)
sports 250 1602 722 boredop (24) dubteedub (155)
talk 600 11406 18618 Catt (37) Catt (440)
tech 1363 19199 12104 Deimos (225) Deimos (3462)
tildes 730 11513 14360 Algernon_Asimov (15) Kat (331)
tildes.official 132 6385 6814 Deimos (132) Deimos (6385)
tv 292 3200 2733 dubteedub (40) dubteedub (367)
Totals: 11626 136321 106930 Deimos (884) Deimos (15837)

I don't want to draw any conclusions or make guesses so I'll leave that up to the reader, I just wanted to share some of the data.

Edit: minor typos here and there.

Edit 2: some plans for the future (mainly for myself):

  • Get user's highest voted topic (total and in specific group)
  • Get user's highest commented on topic (total and in specific group)
  • Do stuff with tags like which tags are used a lot etc.
  • Figure out when topics are posted in the week and in the day.
  • Do stuff with title length like average, shortest and longest, etc.
  • Scrape text topics' word counts and get average, shortest/longest, etc.

67 comments

  1. [12]
    Deimos Link
    Very cool, thanks Bauke! As part of getting ready for public visibility, I've been doing a lot of work lately on getting my own monitoring/statistics working better so that I can notice (and set...

    Very cool, thanks Bauke! As part of getting ready for public visibility, I've been doing a lot of work lately on getting my own monitoring/statistics working better so that I can notice (and set up alerts) if there are issues (both technical and community ones).

    Here's a screenshot of some of the stats I've been tinkering with (using Grafana) - these charts are all displaying data from the last 6 hours: https://i.imgur.com/mPCchSK.png

    One interesting thing you can see in there related to this topic is that the markdown processing quantiles made a huge jump upwards right at the time that you made this post, because it's quite complex (full of tables), so its long processing time had a big effect.

    I'm hoping that I'll be able to make all these charts/statistics public eventually, so at some point hopefully we'll have some live-updating stats like this that people will be able to look at whenever they want.

    26 votes
    1. [2]
      Adys Link Parent
      How is the markdown processing so visible on the graph just for the one topic? The generated HTML is cached, right? Looking forward to the graphs being public. Gitlab has a public grafana as well....

      How is the markdown processing so visible on the graph just for the one topic? The generated HTML is cached, right?

      Looking forward to the graphs being public. Gitlab has a public grafana as well. I'm a sucker for monitoring.

      8 votes
      1. Deimos (edited ) Link Parent
        It's cached, yes. A post's markdown is only processed when it's posted, and then only again if it's edited. There are a few factors why it's so visible. It's using quantiles for that chart, with...

        It's cached, yes. A post's markdown is only processed when it's posted, and then only again if it's edited.

        There are a few factors why it's so visible. It's using quantiles for that chart, with hour-long windows, so they represent something like "90% of all markdown processing in the hour finished inside this amount of time". Since there just aren't that many posts being made/edited right now, it's pretty easy for that one post to end up affecting the 90% quantile, especially if it's edited a few times. Once the site is more active, an outlier like that would only maybe affect the 99% one.

        8 votes
    2. [9]
      deing Link Parent
      Just out of sheer interest, would you mind to share the approximate frequency of Malice labels labels in general?

      Just out of sheer interest, would you mind to share the approximate frequency of Malice labels labels in general?

      5 votes
      1. [8]
        Deimos Link Parent
        Just in general? It's probably pretty normal to have about 30 labels used a day, but it's been more like 50/day over the last few days with the overall higher activity level.

        Just in general? It's probably pretty normal to have about 30 labels used a day, but it's been more like 50/day over the last few days with the overall higher activity level.

        6 votes
        1. [7]
          Algernon_Asimov Link Parent
          Woah! Really? Even on a relatively inactive site such as Tildes has been? Wow! That's... astounding. We didn't get that many reports per day (of all types!) running a subreddit with 10,000-ish...

          It's probably pretty normal to have about 30 labels used a day

          Woah! Really? Even on a relatively inactive site such as Tildes has been? Wow! That's... astounding. We didn't get that many reports per day (of all types!) running a subreddit with 10,000-ish subscribers.

          Are they all justified? Are there actually 30 instances of truly malicious commenting here every day? It would be extremely disappointing if that were true.

          4 votes
          1. [2]
            Deimos Link Parent
            Yeah, all labels. Malice labels is more around the 0-3/day range.

            Yeah, all labels. Malice labels is more around the 0-3/day range.

            6 votes
          2. [4]
            Bauke Link Parent
            That is, labels applied in total, I don't think there's 30-50 malice labels being applied daily... That would be worrisome.

            That is, labels applied in total, I don't think there's 30-50 malice labels being applied daily... That would be worrisome.

            4 votes
            1. [3]
              Algernon_Asimov Link Parent
              But @deing asked about Malice labels specifically, so I assumed that's what @Deimos was talking about.

              But @deing asked about Malice labels specifically, so I assumed that's what @Deimos was talking about.

              2 votes
              1. deing Link Parent
                I actually ninja-edited that. Deimos replied here with Malice amounts.

                I actually ninja-edited that. Deimos replied here with Malice amounts.

                4 votes
              2. Bauke Link Parent
                Take another look, you'll see that malice labels is crossed out and instead he asked for "labels in general". Initially it was to ask about malice labels but then he edited it.

                Take another look, you'll see that malice labels is crossed out and instead he asked for "labels in general". Initially it was to ask about malice labels but then he edited it.

                3 votes
  2. [23]
    teaearlgraycold Link
    Might as well just make /u/Algernon_Asimov the community leader at this point.

    Might as well just make /u/Algernon_Asimov the community leader at this point.

    12 votes
    1. [6]
      Algernon_Asimov Link Parent
      "If drafted, I will not run; if nominated, I will not accept; if elected, I will not serve." :) I don't think posting lots of articles counts in any way as "leadership"! haha

      "If drafted, I will not run; if nominated, I will not accept; if elected, I will not serve." :)

      I don't think posting lots of articles counts in any way as "leadership"! haha

      12 votes
      1. [5]
        Amarok Link Parent
        Except here, someday it will. That's going to factor into the trust system. Don't think of it like being elected - it's more like being drafted against your will. :D

        Except here, someday it will. That's going to factor into the trust system. Don't think of it like being elected - it's more like being drafted against your will. :D

        9 votes
        1. [4]
          Algernon_Asimov Link Parent
          Bugger! Hoist by my own petard. :)

          Bugger! Hoist by my own petard. :)

          6 votes
          1. [3]
            kiddico Link Parent
            Admittedly I'm a little late to this thread. Just trying to do my due diligence and read through previous tilde posts. Gotta get that community lore. Anyways. I was really hoping this was an...

            Admittedly I'm a little late to this thread. Just trying to do my due diligence and read through previous tilde posts. Gotta get that community lore.

            Anyways. I was really hoping this was an Important If True reference. Petard based jokes don't seem to come up much. Unless that was actually a hamlet reference... in which case I think this site may be too well cultured for me to be allowed ;P

            2 votes
            1. [2]
              Algernon_Asimov Link Parent
              "Hoist by his own petard" is just an old saying that a lot of people use. I didn't know it came from Shakespeare until your reply prompted me to look it up. I don't know what that 'Important If...

              "Hoist by his own petard" is just an old saying that a lot of people use. I didn't know it came from Shakespeare until your reply prompted me to look it up.

              I don't know what that 'Important If True' thing is that you're talking about.

              3 votes
              1. kiddico Link Parent
                It's a podcast whose hosts have a tendency to place petards on the ground then proudly stand over them. The phrase "hoisted by my own petard" was used enough that they'd just say "hoisted!" It was...

                It's a podcast whose hosts have a tendency to place petards on the ground then proudly stand over them. The phrase "hoisted by my own petard" was used enough that they'd just say "hoisted!"

                It was the first/only place I'd heard the phrase so it's got a pretty hard link in my mind. And obviously everyone else on the planet has the same memories as me and thinks just like me, so surely that's where you learned about it too haha

                1 vote
    2. [16]
      cadadr Link Parent
      I'm glad Algernon's here, one of the best persons I've interacted online!

      I'm glad Algernon's here, one of the best persons I've interacted online!

      5 votes
      1. [15]
        Algernon_Asimov Link Parent
        Thank you, but there are quite a few people who would disagree with you... strongly. I have it on good authority I've managed to accumulate more "malice" tags than any other person here. (I have...

        Thank you, but there are quite a few people who would disagree with you... strongly. I have it on good authority I've managed to accumulate more "malice" tags than any other person here. (I have been trying to improve my behaviour since I found that out a few weeks ago.)

        14 votes
        1. [9]
          Nephrited Link Parent
          You're genuinely the reason I tend to read and not comment. I'm terrified of you, and you're everywhere, so I'm nowhere. It takes something I'm reasonably invested in to make me speak on the site...

          You're genuinely the reason I tend to read and not comment. I'm terrified of you, and you're everywhere, so I'm nowhere. It takes something I'm reasonably invested in to make me speak on the site these days. :P

          15 votes
          1. [8]
            Algernon_Asimov Link Parent
            I'm sincerely sorry that you feel that way. What can I do to make you less terrified to comment? This is another reason for Deimos to open up the site - to dilute the influence and impact of any...

            I'm sincerely sorry that you feel that way.

            What can I do to make you less terrified to comment?

            This is another reason for Deimos to open up the site - to dilute the influence and impact of any individual user (I'm not the only person who's ubiquitous!).

            9 votes
            1. [7]
              Nephrited Link Parent
              I never responded to this because I couldn't think of a way you can do anything. I don't like arguing online much. It never goes anywhere, because the kind of people who like to continually engage...
              • Exemplary

              I never responded to this because I couldn't think of a way you can do anything. I don't like arguing online much. It never goes anywhere, because the kind of people who like to continually engage aren't kind of people who admit when they're wrong, or can be swung by discussion, and you're very forceful with your arguments, so I'm scared of saying anything that could potentially be disagreed with.

              This is a problem on Reddit as well which is why I only comment there once every two or three months at best. When the site goes public shortly, that'll probably me wiping my hands of it. I don't feel like it's worked out.

              I'm wiping my comment history of most of my comments shortly as it is. Just in case I've said anything objectionable to the masses.

              14 votes
              1. Algernon_Asimov Link Parent
                Thank you for your honesty. I'm truly sorry you're so intimidated by me. That's really not my intention. I know I'm opinionated and forceful, but there's no malice on my side. My intention in...

                Thank you for your honesty. I'm truly sorry you're so intimidated by me. That's really not my intention. I know I'm opinionated and forceful, but there's no malice on my side.

                My intention in asking for feedback was not to argue with it. Even I'm not that stupid!

                Sorry.

                8 votes
              2. [5]
                Amarok Link Parent
                I've wondered a bit about this. Is there some way we could create some form of 'disengage' system that'd basically allow the user to not be bothered by replies to threads/comments? Something...

                I've wondered a bit about this. Is there some way we could create some form of 'disengage' system that'd basically allow the user to not be bothered by replies to threads/comments? Something that's basically a 'walk away' mechanism, and the site respects that somehow by not drawing that particular user back to the thread again through the usual mechanisms (reply notifications, username mentions, etc). Perhaps the plans for anonymous commenting could serve that role as well.

                We all get sucked in, and sometimes, that's more trouble than it's worth.

                7 votes
                1. [4]
                  hungariantoast Link Parent
                  Aside from deleting our comments and topics, a mechanic that allows a user to "disown" a piece of their content, effectively leaving it in-place but without their username present, could work...

                  Aside from deleting our comments and topics, a mechanic that allows a user to "disown" a piece of their content, effectively leaving it in-place but without their username present, could work pretty well.

                  One of the first concerns with this idea would just be random losers making controversial comments and then disowning them so that they can troll everyone, but it's not outside of the realm of possibility to keep a record of who originally made that topic or comment so Deimos can handle them.

                  Another tactic could be that the Malice... or other tags carry a heavier weight with disowned comments, so that the community can be extra effective at moderating disowned content.

                  There's probably another pitfall to this idea I am not thinking of, but I'm going to write this down as another subject to make a suggestion about.

                  5 votes
                  1. Deimos Link Parent
                    I expect that when (if?) I implement that, it would keep the record of who owned the comment for 30 days and then wipe it out (similar to most other private/sensitive data). If it hasn't been an...

                    I expect that when (if?) I implement that, it would keep the record of who owned the comment for 30 days and then wipe it out (similar to most other private/sensitive data). If it hasn't been an issue in the 30 days since they disowned it, it's very unlikely to matter past that point.

                    5 votes
                  2. [2]
                    Algernon_Asimov Link Parent
                    That's not what Amarok was talking about. He's talking about a mechanism where you can choose not to be notified of replies to your comments - like the 'disable inbox replies' option on Reddit.

                    That's not what Amarok was talking about. He's talking about a mechanism where you can choose not to be notified of replies to your comments - like the 'disable inbox replies' option on Reddit.

                    3 votes
                    1. Amarok Link Parent
                      Yeah, but something more thread-specific, so that for the mental health of a user, if a thread is annoying them and they want to walk away from it, they can activate this option somehow, and that...

                      Yeah, but something more thread-specific, so that for the mental health of a user, if a thread is annoying them and they want to walk away from it, they can activate this option somehow, and that thread and anything in it will no longer trigger any such notifications that nag/remind them in the future. It's kinda-sorta related to the concept of a 'hide' mechanic, and it could possibly hide the thread from them so they'll never even see it if it's bumped onto their front page in the future.

                      I think it's worth a daily discussion sometime. Features that safeguard the mental health of the users of the website seems to me like a largely unexplored space, for-profit websites have zero interest in developing that sort of thing. I'd like to see Tildes take a mental-health-aware approach to the userbase going forward. Perhaps we could find simpler, natural ways to diffuse so many of the bad situations that arise in other discussion forums.

                      2 votes
        2. [5]
          dubteedub Link Parent
          I have found that you play devils advocate very effectively and it helps me to narrow down my beliefs and forces me to take stronger consideration of my arguments. I think its helpful to the...

          I have found that you play devils advocate very effectively and it helps me to narrow down my beliefs and forces me to take stronger consideration of my arguments. I think its helpful to the discussion, but can certainly see how it could get on some folks nerves. I respect your thoughts a lot, know you mean well, and glad you are here algernon.

          7 votes
          1. [4]
            frickindeal Link Parent
            They (sorry, not sure of gender) are definitely a user who doesn't mind being what my mother would call contrary. Always an opposing viewpoint, which can often be nice, but isn't always what the...

            They (sorry, not sure of gender) are definitely a user who doesn't mind being what my mother would call contrary. Always an opposing viewpoint, which can often be nice, but isn't always what the reader wants when it's in reply to a strongly-held conviction.

            9 votes
            1. [2]
              Algernon_Asimov Link Parent
              I'm a "he". And, yes, I'm a contrarian. But keep in mind that, if I agree with someone, I'm likely to just vote on their comment and move on, rather than leave a noise-type comment saying "I agree...

              I'm a "he".

              And, yes, I'm a contrarian. But keep in mind that, if I agree with someone, I'm likely to just vote on their comment and move on, rather than leave a noise-type comment saying "I agree with you." People therefore see only the times I disagree with someone; my agreement is invisible. I suffer from confirmation bias.

              But it's not that which is drawing most of the malice tags. I sometimes forget to be diplomatic online, because I'm a direct, honest, and blunt person in real life. That's what I have to improve on.

              (cc: @dubteedub)

              8 votes
              1. frickindeal Link Parent
                Well stated, and that clarifies some of the comments I was referring to. I don't think being direct and honest is ever a bad thing, as long as our interpretation of people's words is charitable...

                Well stated, and that clarifies some of the comments I was referring to. I don't think being direct and honest is ever a bad thing, as long as our interpretation of people's words is charitable and we're not nitpicking minor points.

                4 votes
            2. Amarok Link Parent
              I enjoy a good argument. I think that's something of an acquired taste for most people. :)

              I enjoy a good argument. I think that's something of an acquired taste for most people. :)

              3 votes
  3. [20]
    Algernon_Asimov Link
    Some random observations, comments, and questions: Deimos wasn't exaggerating when he said traffic had been dropping off recently. Would it make more sense to present the users & groups in order...

    Some random observations, comments, and questions:

    • Deimos wasn't exaggerating when he said traffic had been dropping off recently.

    • Would it make more sense to present the users & groups in order of traffic/quantity of posts, rather than alphabetically?

    • It's plain where the Tildes community's interests lie: the busiest groups are ~news, ~tech, ~music, ~games, & ~comp (~misc doesn't count).

    • How the fuck have I made more posts in ~tildes than anyone else??? I've barely posted there!

    12 votes
    1. [2]
      Deimos Link Parent
      Yeah, traffic has consistently been drifting downwards for quite a while now. We definitely haven't hit that point of critical mass where the activity is high enough that it's self-sustaining....

      Yeah, traffic has consistently been drifting downwards for quite a while now. We definitely haven't hit that point of critical mass where the activity is high enough that it's self-sustaining.

      It's mostly that we haven't really had a large burst of interest/invites recently. On some level, that was a bit deliberate—I knew that I was going to be away for a pretty good chunk of December, so I didn't want to have the site in an unstable state during that. I expect that the public visibility will be the next big increase. Afterwards, I hope that it should also be easier to attract new people, since right now the complete inability to see anything about the site without getting an invite first is a pretty big barrier.

      15 votes
      1. Amarok Link Parent
        Invite links, and some form of anonymous commenting / user application area where people here can ask for invites and get voted in by us all somehow. Let's get the waiting room off of reddit and...

        Invite links, and some form of anonymous commenting / user application area where people here can ask for invites and get voted in by us all somehow. Let's get the waiting room off of reddit and moved over here where everyone can pitch in handing out invites. Maybe that can even be a first-pass take at the anon posting system we've talked about for threads that are open for public comment. Start it out as an invite system, see what works, refine it.

        6 votes
    2. [3]
      Bauke (edited ) Link Parent
      To answer some questions: I thought it would make the most sense alphabetically because I include both topics and votes, so deciding which order wasn't the most straightforward but I think...
      • Exemplary

      To answer some questions:

      Would it make more sense to present the users & groups in order of traffic/quantity of posts, rather than alphabetically?

      I thought it would make the most sense alphabetically because I include both topics and votes, so deciding which order wasn't the most straightforward but I think alphabetical is fine. I could regenerate the statistics and make them sort by topics though.

      Edit: regenerated with users/groups sorted by topics https://gitlab.com/snippets/1798579

      How the fuck have I made more posts in ~tildes than anyone else??? I've barely posted there!

      This is actually funny because I double-checked earlier since I thought I would have more than you (with the Issue Log and all) but it turned out that I had (at the time) 14 topics while you have 15. Now including this current topic, that puts me up to 15 too so we're tied, hahaha.

      9 votes
      1. [2]
        Algernon_Asimov Link Parent
        Thanks! Well... yes. If I'd thought about it, I would have assumed the same. Challenge accepted? :)

        regenerated with users/groups sorted by topics

        Thanks!

        I thought I would have more than you (with the Issue Log and all)

        Well... yes. If I'd thought about it, I would have assumed the same.

        Now including this current topic, that puts me up to 15 too so we're tied, hahaha.

        Challenge accepted? :)

        4 votes
        1. Amarok Link Parent
          Be careful. I got into a submission race with happybadger on listentothis that lasted like four years. I think he's still 80 ahead of me with both of us having over 5k submissions. :P

          Be careful. I got into a submission race with happybadger on listentothis that lasted like four years. I think he's still 80 ahead of me with both of us having over 5k submissions. :P

          4 votes
    3. [4]
      clerical_terrors Link Parent
      If you go through some of the older threads you will find a lot of usernames who do not seem to have commented on the site for quite a few months, traffic is lagging everywhere. and on that note:...

      Deimos wasn't exaggerating when he said traffic had been dropping off recently.

      If you go through some of the older threads you will find a lot of usernames who do not seem to have commented on the site for quite a few months, traffic is lagging everywhere. and on that note:

      It's plain where the Tildes community's interests lie: the busiest groups are ~news, ~tech, ~music, ~games, & ~comp (~misc doesn't count).

      I believe this has a lot to do with where the initial arrivals have come from: mostly Reddit, Gitlab, and other tech oriented sites. And I imagine the invitation system mostly biases in favor of this as well. Not to mention that the very low population means other groups are fairly barren and thus new arrivals who are more interested in them are more likely to be discouraged by a lack of content.

      13 votes
      1. [2]
        Algernon_Asimov Link Parent
        Oh, I know why this site is tech-heavy. It's just that, as a non-tech-type myself, it can sometimes feel a little overwhelming and isolating. It's one reason I'm eager for Deimos to open up the...

        I believe this has a lot to do with where the initial arrivals have come from: mostly Reddit, Gitlab, and other tech oriented sites.

        Oh, I know why this site is tech-heavy. It's just that, as a non-tech-type myself, it can sometimes feel a little overwhelming and isolating. It's one reason I'm eager for Deimos to open up the site a bit more, so we can get other people in here to increase the variety of contributors and content. (If I wasn't pushing myself to keep finding content for ~humanities, it would be in the bottom 5 groups for activity.)

        9 votes
        1. patience_limited Link Parent
          I was honestly surprised that ~tech was still my top community - my interests and coverage have been fairly broad. I've tried to avoid topics that are already getting heavy coverage elsewhere, and...

          I was honestly surprised that ~tech was still my top community - my interests and coverage have been fairly broad. I've tried to avoid topics that are already getting heavy coverage elsewhere, and my day-to-day work has gotten less technical and more project-oriented, so there's less territory where I might add value to the ~comp and ~tech discussion. Tildes is one of the few ~hobbies I can make time for right now, and mostly involves the reading I would do anyway.

          7 votes
      2. dubteedub Link Parent
        I know i make a point to try and post more in ~tv, ~movies, ~lgbt, and ~enviro since those groups can be pretty empty a lot. This seens like a good time to revisit the sort preference. I liked the...

        I know i make a point to try and post more in ~tv, ~movies, ~lgbt, and ~enviro since those groups can be pretty empty a lot.

        This seens like a good time to revisit the sort preference. I liked the three day sort originally, but now it leaves lots if grouos looking like theres nk content / intersst there.

        6 votes
    4. [10]
      dubteedub (edited ) Link Parent
      I know i dropped off quite a bit between Thanksgiving and Christmas to spend more time with my family, but in posting much more again now. I assume other folks probably had a similar slump too....

      I know i dropped off quite a bit between Thanksgiving and Christmas to spend more time with my family, but in posting much more again now. I assume other folks probably had a similar slump too.

      Also, i thought i might be a bit closer to you in overall number of posts, but you still got me beat by a couple hundred. Damn, gonna need to step it up.

      5 votes
      1. [9]
        frickindeal Link Parent
        It's funny how I saw threads when I first got here talking about how these totals, when they inevitably happened from people scraping the site and despite the fact there was no easy way to see...

        It's funny how I saw threads when I first got here talking about how these totals, when they inevitably happened from people scraping the site and despite the fact there was no easy way to see your total upvotes, would just lead to competition and reddit-style "karma farming" on Tildes, and here I see it in action already. I'm not suggesting it'll have a net negative effect here, but a lot of people seem surprised by their "stats." Interesting.

        4 votes
        1. [2]
          Algernon_Asimov Link Parent
          Do you really believe people are actually competing to make more posts / score more votes than other people? I thought @dubteedub's joke about stepping it up was just that: a joke. I didn't take...

          would just lead to competition and reddit-style "karma farming" on Tildes, and here I see it in action already.

          Do you really believe people are actually competing to make more posts / score more votes than other people? I thought @dubteedub's joke about stepping it up was just that: a joke. I didn't take it seriously.

          However, if @dubteedub does makes good on their joke to "step it up" to keep up with me, I think that would be a good thing because it would mean more content for Tildes.

          As the most prolific contributor here (apart from Deimos himself), I can assure you that I'm not competing for anything. I'm just trying, for better or worse, to keep content flowing on this website so it doesn't die for lack of activity. In a way, it makes me feel a bit embarrassed and awkward when I come near the top of these statistics (this is the second time I've seen statistics like this). I'm not trying to stand out. I'm just trying to contribute. There's part of me that wishes people like @Bauke would not scrape and display this data, because I don't want the attention.

          6 votes
          1. frickindeal Link Parent
            That's more to my point. There's more than one comment here referencing a desire or need to improve "stats." I find it interesting in an analytical way. @dubteedub's comment is likely...

            There's part of me that wishes people like @Bauke would not scrape and display this data, because I don't want the attention.

            That's more to my point. There's more than one comment here referencing a desire or need to improve "stats." I find it interesting in an analytical way. @dubteedub's comment is likely tongue-in-cheek, but there is this:

            but you still got me beat by a couple hundred

            Stat-comparing, no? It may be light-hearted, but it's there. I'm not saying it's negative; it's obviously very natural, since I see it occur again and again across the internet. Be it likes, upvotes, views, visits, whatever stat, people as a whole will inevitably compete to have good stats. Some may abstain or just not care, but the trend is to compete, and it's fascinating.

            This quote is yours:

            If you've made the Top 10 contributors

            Certainly has an air of competition when you refer to "top" anything. Again, no judgement, just interest from a sociological viewpoint.

            4 votes
        2. [6]
          dubteedub Link Parent
          I was being more tongue in cheek with the "gonna have to step it up." Algernon had previously estimated he was the most active Tilderino after Deimos and in my estimation I just thought I was a...

          I was being more tongue in cheek with the "gonna have to step it up."

          Algernon had previously estimated he was the most active Tilderino after Deimos and in my estimation I just thought I was a bit closer to him.

          2 votes
          1. [5]
            Algernon_Asimov Link Parent
            It wasn't an estimate. It was hard data. Someone else (I want to say crius, but I won't @-tag them just in case I'm wrong) did similar data scraping a few months back, but they left the data...

            Algernon had previously estimated he was the most active Tilderino after Deimos

            It wasn't an estimate. It was hard data. Someone else (I want to say crius, but I won't @-tag them just in case I'm wrong) did similar data scraping a few months back, but they left the data anonymous. I saw some anomalies in the data, and asked them some questions privately. In that private conversation, they revealed that I was the second-most active Tilder after Deimos.

            2 votes
            1. [4]
              dubteedub Link Parent
              That is fair. I thought you had said something like "I'm pretty sure I am the second most prolific poster on Tildes behind Deimos," but you may have been more definitive. I assume being the one...

              That is fair. I thought you had said something like "I'm pretty sure I am the second most prolific poster on Tildes behind Deimos," but you may have been more definitive.

              I assume being the one really active Australian makes your data pop out a bit more than those of us in North America.

              1 vote
              1. [3]
                Algernon_Asimov Link Parent
                I may have been being coy because I didn't want to run around saying "I'm the second most prolific poster". It sounds like boasting. How would that work? The data isn't grouped or categorised by...

                I thought you had said something like "I'm pretty sure I am the second most prolific poster on Tildes behind Deimos," but you may have been more definitive.

                I may have been being coy because I didn't want to run around saying "I'm the second most prolific poster". It sounds like boasting.

                I assume being the one really active Australian makes your data pop out a bit more than those of us in North America.

                How would that work? The data isn't grouped or categorised by country.

                It's nothing to do with anything clever or subtle. It's just the simple fact that I post a lot.

                3 votes
                1. [2]
                  dubteedub Link Parent
                  If post data were displayed by time of day or something I could imagine you would be able to pick out your posts pretty easily since most of them come in when the Americans are sleeping.

                  If post data were displayed by time of day or something I could imagine you would be able to pick out your posts pretty easily since most of them come in when the Americans are sleeping.

                  1. Algernon_Asimov Link Parent
                    But they're not, so the point is moot. I'm confused.

                    If post data were displayed by time of day

                    But they're not, so the point is moot.

                    I'm confused.

  4. [4]
    patience_limited Link
    Thanks, Bauke, for your efforts in both getting this off the ground, and the helpful analytics. As to contributions, I'm doing the best I can on a limited budget of time, but committed to the...

    Thanks, Bauke, for your efforts in both getting this off the ground, and the helpful analytics.

    As to contributions, I'm doing the best I can on a limited budget of time, but committed to the Tildes concept and aiming for both quality and uniqueness of material.

    11 votes
    1. [3]
      Algernon_Asimov Link Parent
      If you've made the Top 10 contributors, I don't think you need to apologise for not doing enough!

      As to contributions, I'm doing the best I can on a limited budget of time

      If you've made the Top 10 contributors, I don't think you need to apologise for not doing enough!

      6 votes
      1. [2]
        patience_limited Link Parent
        Well, it's not precisely top 10, just more than 100 topics. The aim has been at least one a day, but some days are better than others. Mainly, I'll toss up interesting bits from general reading.

        Well, it's not precisely top 10, just more than 100 topics. The aim has been at least one a day, but some days are better than others. Mainly, I'll toss up interesting bits from general reading.

        7 votes
        1. Algernon_Asimov Link Parent
          Fine: Top 16! :P It's still nothing for you to apologise for.

          Well, it's not precisely top 10

          Fine: Top 16! :P

          It's still nothing for you to apologise for.

          5 votes
  5. Bishop Link
    i don't really have anything constructive to add - just wanted to say this is really cool lmao. also hell yes ~creative we really out here 💜 𝕏 🖤

    i don't really have anything constructive to add - just wanted to say this is really cool lmao.

    also hell yes ~creative we really out here 💜 𝕏 🖤

    9 votes
  6. [2]
    Bauke Link
    Here are 2 more graphs looking at: Average topics and votes for each day per hour and average topics and votes per day. I've posted the data for these on GitLab here. Album with both graphs.
    • Exemplary

    Here are 2 more graphs looking at: Average topics and votes for each day per hour and average topics and votes per day. I've posted the data for these on GitLab here.

    Album with both graphs.

    6 votes
    1. Algernon_Asimov Link Parent
      Surprise, surprise - Tildes' busiest time is during American weekdays! Who'd'a thunk it? :P

      Average topics and votes for each day per hour

      Surprise, surprise - Tildes' busiest time is during American weekdays! Who'd'a thunk it? :P

      2 votes
  7. [5]
    Gaywallet Link
    I find it very interesting that the number of comments and votes are so close to each other. We often talk about the 90/10 rule in tech where only 10% of people contribute to the conversation. The...

    I find it very interesting that the number of comments and votes are so close to each other. We often talk about the 90/10 rule in tech where only 10% of people contribute to the conversation. The ratio of votes to comments isn't quite indicative of views to comments, but if you compare votes to comments on Reddit, for example, it's probably close to 90/10 or perhaps even less (wrt participation).

    8 votes
    1. [2]
      cadadr Link Parent
      Well we're the initial bunch that's came seeking a proper general purpose discussion forum online, so I'd say that's expected at this point. That'd change depending on how we grow in the future,...

      Well we're the initial bunch that's came seeking a proper general purpose discussion forum online, so I'd say that's expected at this point. That'd change depending on how we grow in the future, tho. One reason I support keeping Tildes browsable but invite-only.

      13 votes
      1. frickindeal Link Parent
        I do a ton of voting here, and a lot less commenting, and I believe it's for exactly that reason: I'm a lot more willing to contribute here, even if it's just by voting, because I want this place...

        I do a ton of voting here, and a lot less commenting, and I believe it's for exactly that reason: I'm a lot more willing to contribute here, even if it's just by voting, because I want this place to succeed and see the value.

        11 votes
    2. [2]
      Deimos Link Parent
      I think you also have to take into account that you need an invite to do anything on Tildes, and getting that invite almost certainly involved requesting it somewhere (reddit, email, etc.). The...

      I think you also have to take into account that you need an invite to do anything on Tildes, and getting that invite almost certainly involved requesting it somewhere (reddit, email, etc.). The action of requesting the invite is probably already above the threshold of what most of "the 90" would bother doing, so we're already starting from a group that's more willing to get involved.

      11 votes
      1. Algernon_Asimov Link Parent
        Actually... I had my invite just handed to me, without asking! I mentioned to a co-mod that I'd just learned about this new website called Tildes, which I thought he might be interested in. He...

        Actually... I had my invite just handed to me, without asking!

        I mentioned to a co-mod that I'd just learned about this new website called Tildes, which I thought he might be interested in. He told me he was already on it, and sent me an invite.

        So you can all blame him!

        5 votes