Geez, Bauke, just when I think you're starting to slow down on Tildes projects a little, you turn around and launch yet another one. This looks great, thanks as always for working on things like...
Geez, Bauke, just when I think you're starting to slow down on Tildes projects a little, you turn around and launch yet another one.
This looks great, thanks as always for working on things like this. Maybe I can help pull out some historical data for topics/comments when you get that set up, so you can have the past filled in too.
Unique users posting every day/week/month would be interesting to see how big the active user base actually is. It seems like only a small fraction post anything?
If you have any suggestions or anything else about the site, feel free to comment or message me them.
Unique users posting every day/week/month would be interesting to see how big the active user base actually is. It seems like only a small fraction post anything?
After looking at these datasets again I've realized these datasets aren't really compatible. Your comment/the data you scraped shows people that have either commented or posted here in the last...
After looking at these datasets again I've realized these datasets aren't really compatible. Your comment/the data you scraped shows people that have either commented or posted here in the last month, shown together and my comment/the data @Bauke posted shows people that have commented or posted in any given week , shown separately. So unless he has scraped his data in such a way that he can make a dataset comparable to yours or vice versa, this isn't that useful.
This kind of projects is what put Tildes ahead to others Reddit like platforms. Great work!. Interesting Anime has the most subscribers but has little activity.
This kind of projects is what put Tildes ahead to others Reddit like platforms. Great work!.
Interesting Anime has the most subscribers but has little activity.
It's sorted alphabetically by group. I made the same mistake when I read it the other day. It surprised me so much I spent a bit of time reading it carefully.
It's sorted alphabetically by group. I made the same mistake when I read it the other day. It surprised me so much I spent a bit of time reading it carefully.
Has any progress been made on all those other charts? If not, why? The "over time" data you scraped here was scraped with a week per data point. Could you scrape that data with a day per data...
Once those are done, I'll be moving on to topics and comments, and probably recreate all the statistics I had available in the last time I did this.
The stats in question
Topics over Time
Comments over Time
Posts over Time
Posts Available, Deleted & Removed
Topic Types
Posts per Group
Most Frequent Tags
Unique Post Authors
Posts over Time by Group
Posts during the Week
Posts during the Day per Hour
Time between subsequent Comments
Has any progress been made on all those other charts? If not, why?
If you have any suggestions or anything else about the site, feel free to comment or message me them.
The "over time" data you scraped here was scraped with a week per data point. Could you scrape that data with a day per data point and plug that data in those charts so we can see the history of the site from day 1 of the site instead of whenever your bot begins taking snaphots of it?
Pedantic but it would be neat to specify at what time of the day exactly these snaphots of the site are taken.
Oh okay. I was wondering why the site only had 29 days of data but this post is quite a bit older than that. Now I know :p You could compromise and have an all-time chart and a few more recent...
I did get started on the group subscriber charts a few months ago but got distracted by some other stuff and haven't been back to it yet.
There are also some other problems that have become apparent now that it has been running for a while, which I want to get fixed first. (For example the numbers on the charts, they've become unreadable now that there are so many. Or the fact that the site errors out on generation when /financials isn't updated.
I did plan on starting again on it this coming week, so there will likely be some changes soon. And I also wanted to open-source the site as well, since I've previously never done that with my scraping stuff, but I figure since I want this to run for a while it's better if it's out there.
Oh okay. I was wondering why the site only had 29 days of data but this post is quite a bit older than that. Now I know :p
The data there was actually scraped completely individually, I just decided to calculate the statistics to be weekly, so technically I could put it in whatever time period I want, however you'd need either a gigantic chart to display that properly or a live adjustable chart, both of which I have little desire to attempt.
You could compromise and have an all-time chart and a few more recent charts for us to view separately (although you might need to do some extra coding to not show multiple charts but instead allow us to click to show us different amounts of data like "last week, last month, last year, all time, etc.), or do you not feel like doing that?
Also, the data that's on the site right now (financials and group subscriptions), I hadn't scraped before, so those numbers are unfortunately lost to time.
Isn't that because donation and user-joining history isn't kept track of long-term? Since you can always see when a comment was posted, what tags it has and who posted them (unless their account is deleted and their comments are kept up via 'unknown username' IIRC) so anyone could go back and scrape all that data again unlike user-joining and donation history, right? Or have you been scraping these comments since day 1 or something?
if you hover over the underlined date from the Last generated on ... bit in the footer, it shows you exactly (down to the millisecond) when the site and latest snapshot was generated.
At this stage, is it worth making this distinction? @Deimos, have you given any thought to incorporating this, or any other Tildes-related, project under the official banner?
At this stage, is it worth making this distinction?
@Deimos, have you given any thought to incorporating this, or any other Tildes-related, project under the official banner?
I don't really know what that would mean. I'd like to have some official stats available on the site itself, but I'm not going to like... let people upload projects to the Tildes domain with no...
People building related projects on their own is great, it means they can use whatever tools/tech/etc. that they're interested in. Trying to make it "official" means they'll probably have to deal with my restrictions on what I want to have on the Tildes server/domain, which isn't always something they'd want to do.
I'm not proposing. That, and the second paragraph's presented obstacles, is averted when you give a badge of approval to a tool you deem valuable that isn't of Tildes itself. @Bauke sure as shit...
but I'm not going to like... let people upload projects to the Tildes domain with no oversight
I'm not proposing.
That, and the second paragraph's presented obstacles, is averted when you give a badge of approval to a tool you deem valuable that isn't of Tildes itself. @Bauke sure as shit deserves one, given any single Tildes-related project.
"I'm commander Deimos, and this is my favorite unofficial Tildes project on the Internet"
Geez, Bauke, just when I think you're starting to slow down on Tildes projects a little, you turn around and launch yet another one.
This looks great, thanks as always for working on things like this. Maybe I can help pull out some historical data for topics/comments when you get that set up, so you can have the past filled in too.
Unique users posting every day/week/month would be interesting to see how big the active user base actually is. It seems like only a small fraction post anything?
According to this chart he made some time ago, and this statistic Deimos scraped a bit longer ago, ~200 posters and 1000 users.
That's definitely low. There have been 538 different users that have posted at least one topic or comment in the last 30 days.
I'd like to know the median data. Should be better represtation
After looking at these datasets again I've realized these datasets aren't really compatible. Your comment/the data you scraped shows people that have either commented or posted here in the last month, shown together and my comment/the data @Bauke posted shows people that have commented or posted in any given week , shown separately. So unless he has scraped his data in such a way that he can make a dataset comparable to yours or vice versa, this isn't that useful.
Very cool @Bauke, you never cease to impress me! How often are you scraping?
Anyone who uses SVG charts in their design gets my instant respect. There's literally no excuse to use anything else.
Very cool. I look forward to seeing how this goes.
Nice but I have to say I can't really read this, the colors fuck me up.
This kind of projects is what put Tildes ahead to others Reddit like platforms. Great work!.
Interesting Anime has the most subscribers but has little activity.
It doesn't?
You are right my mistake :)
It's sorted alphabetically by group. I made the same mistake when I read it the other day. It surprised me so much I spent a bit of time reading it carefully.
Oddly enough it renders as a square on my phone.
Has any progress been made on all those other charts? If not, why?
The "over time" data you scraped here was scraped with a week per data point. Could you scrape that data with a day per data point and plug that data in those charts so we can see the history of the site from day 1 of the site instead of whenever your bot begins taking snaphots of it?
Pedantic but it would be neat to specify at what time of the day exactly these snaphots of the site are taken.
Oh okay. I was wondering why the site only had 29 days of data but this post is quite a bit older than that. Now I know :p
You could compromise and have an all-time chart and a few more recent charts for us to view separately (although you might need to do some extra coding to not show multiple charts but instead allow us to click to show us different amounts of data like "last week, last month, last year, all time, etc.), or do you not feel like doing that?
Isn't that because donation and user-joining history isn't kept track of long-term? Since you can always see when a comment was posted, what tags it has and who posted them (unless their account is deleted and their comments are kept up via 'unknown username' IIRC) so anyone could go back and scrape all that data again unlike user-joining and donation history, right? Or have you been scraping these comments since day 1 or something?
Also pedantic but you can't hover on mobile :P
Maybe allow mobile users to tap on it for info?
Why is it hosted on your domain and not on Tildes'?
At this stage, is it worth making this distinction?
@Deimos, have you given any thought to incorporating this, or any other Tildes-related, project under the official banner?
I don't really know what that would mean. I'd like to have some official stats available on the site itself, but I'm not going to like... let people upload projects to the Tildes domain with no oversight.
People building related projects on their own is great, it means they can use whatever tools/tech/etc. that they're interested in. Trying to make it "official" means they'll probably have to deal with my restrictions on what I want to have on the Tildes server/domain, which isn't always something they'd want to do.
I'm not proposing.
That, and the second paragraph's presented obstacles, is averted when you give a badge of approval to a tool you deem valuable that isn't of Tildes itself. @Bauke sure as shit deserves one, given any single Tildes-related project.
"I'm commander Deimos, and this is my favorite unofficial Tildes project on the Internet"
Technically I don't think it's affiliated with Tildes in anyway.