• Most votes
  • Most comments
  • Newest
  • Activity
  • Showing only topics in ~tildes with the tag "data". Back to normal view / Search all groups
    1. TLDR: Album of all 3 graphs Edit 3: Added some more graphs in my top level comment here. Hi all, over the past few days I've slowly been working on a scraper for Tildes with the goal of getting...

      TLDR: Album of all 3 graphs

      Edit 3: Added some more graphs in my top level comment here.

      Hi all, over the past few days I've slowly been working on a scraper for Tildes with the goal of getting some statistics and as an exercise in learning Puppeteer. I may or may not put the source for it online later but it's surprisingly easy to do (so maybe you can learn too, wink wink nudge nudge). Also, I'm not sure whether I want to publish the full data file as I want to respect the privacy of the people, I don't know. You let me know.

      On to the specifics of what I'm scraping then, currently the only things are the topics in the topic listing, which give me:

      export interface ITopic {
        author: string;   // Author of the topic
        comments: number; // Amount of comments the topic has
        date: Date;       // Date the topic was posted
        group: string;    // Group the topic was posted in
        id: string;       // The base 36 ID of the topic
        link: string;     // A link to the topic, primarily for easier Markdown formatting
        tags: string[];   // Tags of the topic, currently not doing anything with them
        title: string;    // Title of the topic, also not using this for anything
        type: TopicTypes; // The type of topic, ie: Link or Text
        votes: number;    // Amount of votes the topic has
      }
      

      And now that the boring stuff is out of the way, actual data:

      Topics

      Line graph of topics per week and line graph of topics per month

      Year Week # of Topics - Year Month # of Topics
      2018 17 119 - 2018 4 42
      2018 18 143 - 2018 5 1306
      2018 19 188 - 2018 6 2045
      2018 20 451 - 2018 7 1440
      2018 21 440 - 2018 8 2016
      2018 22 831 - 2018 9 1405
      2018 23 429 - 2018 10 1370
      2018 24 403 - 2018 11 940
      2018 25 298 - 2018 12 728
      2018 26 281 - 2019 1 334
      2018 27 241 -
      2018 28 336 -
      2018 29 406 -
      2018 30 418 -
      2018 31 506 -
      2018 32 484 -
      2018 33 479 -
      2018 34 350 -
      2018 35 345 -
      2018 36 345 -
      2018 37 354 -
      2018 38 305 -
      2018 39 306 -
      2018 40 289 -
      2018 41 326 -
      2018 42 286 -
      2018 43 324 -
      2018 44 231 -
      2018 45 207 -
      2018 46 243 -
      2018 47 180 -
      2018 48 173 -
      2018 49 199 -
      2018 50 159 -
      2018 51 150 -
      2018 52 152 -
      2019 1 143 -
      2019 2 106 -

      Users

      This table only lists users with more than 100 topics, otherwise it would be far too big. All votes listed are only votes received on topics.

      User # of Topics # of Votes Most Topics In Group (Topics) Most Votes In Group (Votes)
      Algernon_Asimov 728 5863 news (288) news (1828)
      Catt 280 2829 news (40) talk (440)
      Deimos 884 15837 tech (225) tildes.official (6385)
      EightRoundsRapid 472 3068 news (146) news (977)
      EscReality 128 1066 food (41) food (328)
      Neverland 165 1911 news (79) news (922)
      Whom 119 1179 music (87) music (864)
      asteroid 169 1316 tech (50) tech (365)
      boredop 117 862 music (52) music (321)
      cfabbro 275 1725 music (71) music (326)
      dubteedub 513 6263 news (151) news (2120)
      ducks 107 1411 misc (23) tech (285)
      jmillikin 134 1585 news (30) news (351)
      patience_limited 124 1350 tech (19) tech (236)
      rkcr 127 1406 misc (29) tech (323)
      spit-evil-olive-tips 386 3926 news (66) news (735)

      Groups

      The top users' numbers here are how many topics/votes they have in that specific group. And of course Deimos is listed at the bottom with the highest number of topics/votes.

      Graph of this table, excluding top users.

      Group # of Topics # of Votes # of Comments Top User (Topics) Top User (Votes)
      anime 116 974 877 Cleb (19) Cleb (193)
      books 239 2203 2012 Catt (19) cadadr (211)
      comp 671 8890 6150 Deimos (41) Deimos (626)
      creative 259 2586 1294 Bishop (49) Bishop (364)
      enviro 252 2757 1136 dubteedub (42) dubteedub (523)
      food 239 2096 1502 spit-evil-olive-tips (45) spit-evil-olive-tips (368)
      games 891 10914 9038 Deimos (193) Deimos (2230)
      health 174 1635 1062 Catt (26) Catt (238)
      hobbies 85 1100 1403 spit-evil-olive-tips (3) RedditRefugee (48)
      humanities 254 2291 1206 Algernon_Asimov (97) Algernon_Asimov (796)
      lgbt 173 1919 1124 Algernon_Asimov (61) Algernon_Asimov (535)
      life 245 3291 2324 Catt (27) Catt (319)
      misc 884 9286 5332 EightRoundsRapid (69) dubteedub (855)
      movies 263 2331 1722 dubteedub (42) dubteedub (381)
      music 1295 7676 5029 EightRoundsRapid (134) Whom (864)
      news 1700 18115 8817 Algernon_Asimov (288) dubteedub (2120)
      science 519 4952 1551 Algernon_Asimov (66) Algernon_Asimov (556)
      sports 250 1602 722 boredop (24) dubteedub (155)
      talk 600 11406 18618 Catt (37) Catt (440)
      tech 1363 19199 12104 Deimos (225) Deimos (3462)
      tildes 730 11513 14360 Algernon_Asimov (15) Kat (331)
      tildes.official 132 6385 6814 Deimos (132) Deimos (6385)
      tv 292 3200 2733 dubteedub (40) dubteedub (367)
      Totals: 11626 136321 106930 Deimos (884) Deimos (15837)

      I don't want to draw any conclusions or make guesses so I'll leave that up to the reader, I just wanted to share some of the data.

      Edit: minor typos here and there.

      Edit 2: some plans for the future (mainly for myself):

      • Get user's highest voted topic (total and in specific group)
      • Get user's highest commented on topic (total and in specific group)
      • Do stuff with tags like which tags are used a lot etc.
      • Figure out when topics are posted in the week and in the day.
      • Do stuff with title length like average, shortest and longest, etc.
      • Scrape text topics' word counts and get average, shortest/longest, etc.
      41 votes
    2. Intro Hello everyone. Due to @Kat’s ever-failing health, I will be analyzing the data instead of her this time around. If you have no idea what this is about, see the demographic survey that was...

      Intro

      Hello everyone.

      Due to @Kat’s ever-failing health, I will be analyzing the data instead of her this time around. If you have no idea what this is about, see the demographic survey that was posted on the day of Tildes’ half-year birthday. She’s done this before, so let’s see what's new.

      The original survey was answered by 404 people, while the half year survey was answered by 293. Though the total number of replies was lower, the completion rate was actually higher: 293 responses from 422 unique visitors, or 69.4%, up from the first year’s 404/599=67.4%. The decrease in answers is most likely attributed to the change of the default sort from “Activity, all time” to “Activity, 3 days”: the response rate held fairly consistent for the first three days, then plummeted after the third as the topic stopped being able to gain any publicity. Though response rates on the original were not high after the first three days, there was a steady trickle up until the survey stopped accepting responses.

      While the numbers are relatively big (for a community of this size), do take anything found with a healthy dose of scepticism. Even though the original dataset she shared with me does not contain any identifiable information (all I can see are randomly-generated user strings) the specifics of that data will not be posted, as was mentioned during the original survey. This is because I am unable to be certain I can sufficiently anonymize it. Typeform has created a summary of the data on a per-question basis with substantially more datapoints than this thread, which you can find https://themeerkat.typeform.com/report/H2TtYg/rVf75AqbKaPncy6y.

      Explanation

      I will compare the statistics with a similarish reference set based on the six most common territories, all of which are above one percentage of the survey answers. That means when I compare on the general populace, I will base it on numbers from USA, Canada, UK, Australia, the Netherlands, and France.

      This means it will be weighted like this:

      USA CAN UK AUS NL FR
      55 22 20 10 8 6
      45.45% 18.18% 16.53% 8.26% 6.61% 4.96%

      I’ll clean up my data sheet and post it in the comments later. You all are absolutely encouraged to fix it because it will most likely contain errors.

      The interesting stuff

      What has changed in the first half year?

      Age

      This time around an age range was used instead of an exact numerical input, but if we were to assume that everyone is aged in the middle of their age range (so 20 for 18-22 year olds, for instance), the average age of a user would be 26.84 years, or 26 years, 9 months, and 4 days old (roughly). So we’ve grown a bit younger than last year, on average.

      Gender and identity

      Gender distribution seems to be roughly the same. We see a small decrease in percentage of heterosexuals, divided roughly evenly on the remaining categories. We also see a significant increase in the amount of transgender users, but since the amount reported is small, that could also just be statistical noise. The percentage of polyamorous people has remained exactly the same. For pronouns, there are only three users who prefer it/its, and zero who prefer any neopronoun set: every “Other” was offering commentary on the question rather than answering it. Similarly, almost all of the “Others” for orientation were expressing that they didn’t understand the specifics of the options given.

      All in all, little has changed.

      Territorial

      In both surveys, three options dominated: the USA, the UK, and Canada. On that end, little has changed, though it seems that all of the Swedes disappeared, with zero answering the half year survey as compared to eight for the first one. Wonder what they’ve been busy with.

      Native language

      Unsurprisingly, about everyone speaks English. What is more surprising is the lack of native multilinguals: fewer than 6% of Americans who natively speak English also natively speak a second language. For comparison, that’s 10% for Australians, 21% for Canadians, and 13% for the UK. This represents an overall decrease in geographic diversity, with users coming from 36 different countries as compared to 42 the first time.

      Religion

      Compared to the world at-large, we sure are a god-denying folk. A whopping 52% of us consider ourselves atheists, whereas the sample data puts it at 12.1%, so we’re far from the norm of our fellow citizens.

      We got a few interesting answers in the “other” section of the religion part of the survey. We got a few interesting ones I had never heard of before, like “Discordian”. But generally speaking, around half of them were either “none” or one of the actual options. Two stood out to me though.

      To the one Chinese user who filled it in as “The heck is chinese traditional”: I have no idea either.

      To the one Australian user who wrote “Left-hand path Heathen”, you be yourself, mate.

      Politics

      The average has barely moved in the last half year—we’re still slanted very much to the left. Unlike the first survey, there was no freeform input this time around, so the specifics are hard to discuss.

      Computers

      We have seen a drastic fall in the percentage of Windows users. It was at 60%, and is down to 43%. Nearly all of this has gone to Linux, which is now at 38%. That’s quite large, especially compared to the reference data, which has Linux use among web users at 1.23%. It’s like a herd of penguins in here.

      Mobile phones

      Compared to half a year ago, not many of us have switched mobile OS. Compared to the calculated data, we like Android slightly more than average. 62% vs 72%.

      Not much interesting in the “other” section, though I will give a salute to the one American user still holding out on Windows 10 Mobile.

      Work

      We have a pretty even distribution with three exceptions. “Computer software”, “Never employed”, and “IT”. Nearly 3/4 who answered “Never employed” are currently students.

      Among the students, we only have one student that proudly smokes and has no interest in quitting. The campaigns seem to be working.

      Tildes usage

      If we look at the users who visit Tildes multiple times per day, we see a few interesting trends. Nearly all of them use Android, and nearly all of them are employed. Beyond that it all seems surprisingly… average.

      Overall, people rated Tildes as a platform as-it-stands a 5.7/7 (0.81), and their optimism for the future of the site at a 5.4/7 (0.77). The most important reason they use the site (of the options given) is “Minimal, fast design” at a 4.6/5 (0.92), with “Privacy-consciousness and lack of trackers” right on its heels. 20.8% of users have ever contributed money to Tildes (surprisingly high, compared to most donation campaigns), with about half as many making a recurring donation.

      Despite @Kat’s insidious attempt to influence the data, “waves” as a demonym only received 5.5% of the vote. The leader for that, overwhelmingly, is “no demonym at all”, with a combined 49% of the votes and 18.5% of respondents strongly preferring the site not to have a demonym. Second place, the generic “users”, only has 15.8% in comparison. The first Tildes-specific demonym present is Tilders/~​rs, with 13.4%.

      Most notably, about ⅔ of users would prefer Tildes to be remain invite-only long-term.

      Freeform questions

      The survey had three freeform questions: “What do you like most about Tildes, thus far?”, “What do you like least about Tildes, thus far?”, and “What is the most pressing missing feature/‘pain point’ for you about Tildes in its current state?” All the comments fill over 30 pages, so it seems like we really have a lot to say. You can download and look at all of the raw answers here, if you’d like. They’ve been shuffled to ensure privacy.

      Likes

      A large majority of the comments boil down to “a quality of discussion where disagreement is discussed in a respectful and level-headed way”. A very significant amount also point out the lack of “low effort content and trolls” as a good thing. A significant amount also mention the simple and quick-loading interface. We also have one user who believes he can find a twerk team on Tildes.

      So on this, @Deimos can feel proud for what he has done. Though you know what really makes the site good? There is one comment that properly gets it: “The people, d’awwww.”. Yes, that includes you.

      Dislikes

      But not everything is perfect, though negatives about Tildes seem to be a lot less unanimous than the positives. There are a few that repeat a bit more often than others: the biggest one is “left centrism in discussions” or “echo chambers”, though in a close second, as with any political discussion, is its exact opposite with complaints about “too much discussion about left-centrism in discussions”—notably, though, in the question “Do you feel as though Tildes has a good mix of political opinions, for your personal preferences?”, the leading answer was “Yes” with 63%. A small amount of users also think we have too many software developers.

      Beyond that, the main complaint that stands out is “lack of users and content”, which I am sure will improve in time.

      Missing feature/pain point

      This too is very varied. A lot of the comments are actually about features that have been introduced since the survey was done, like bookmarking. Honestly, it’s not that many complaints compared to just likes and dislikes.

      The “majority” seem to be on a lack of tag autocompletion, USA-centrism, and the lack of a mobile app.

      There was one more section: “If you would like to offer any long-form commentary, criticism, or feedback regarding Tildes, you may do so here.” Due to its nature, I’ll let you read through them yourself in the raw data, if you’re interested.

      Closing words

      First of all, to everyone who took the time to answer: thank you! I hope this post and the survey has brought some fun to everyone. If there’s an interest, I am sure that Kat, myself, or someone else will make another one at the one year anniversary. We already got some feedback in the previous thread, but we’re always open for more.

      I will do some additional data comparisons on request. I might be a bit occupied this weekend, though, so that will come when it comes.

      46 votes