Nope, you're right. I've got search started in dev, but still wanted to do a bit more work on it (and like @rkcr says, I didn't think it would be needed for a little while, anyway). Anything you...
Nope, you're right. I've got search started in dev, but still wanted to do a bit more work on it (and like @rkcr says, I didn't think it would be needed for a little while, anyway).
Anything you think is "essential" with it initially? Just basic title/text search, or was there something else that you were trying to look up?
When you have time, can you (briefly) shed some light on why exactly reddit's search function is such a complete failure? How have you differentiated the build-out for the ~ search function to...
When you have time, can you (briefly) shed some light on why exactly reddit's search function is such a complete failure? How have you differentiated the build-out for the ~ search function to keep it from devolving into spaghetti code? Just better initial planning?
Well one of the reasons I can think of off the top of my head doesn't even involve reddit Inc. The users of reddit generally use more clickbait-esque titles (often by necessity since it's the only...
Well one of the reasons I can think of off the top of my head doesn't even involve reddit Inc. The users of reddit generally use more clickbait-esque titles (often by necessity since it's the only way to get attention in the fast moving new queues) instead of descriptive ones which makes finding the content again later impossible unless you remember the exact title.
E.g. "Someone finally did it!" for a video of a motorcycle ramping off a truck car-hauling trailer. So if you search for "biker ramp truck" or any variation thereof you won't find it.
Using descriptive, accurate titles and tags along with every user making an effort to spend time in the new queues so we don't devolve into relying on clickbait, is something we are going to have to foster in ~ if we want search to work effectively here.
I see. Totally agree with the tag-based approach - forcing users to comply with a standard title formatting for a wide variety of types of content is virtually impossible. Just spitballing here,...
I see. Totally agree with the tag-based approach - forcing users to comply with a standard title formatting for a wide variety of types of content is virtually impossible. Just spitballing here, but what if ~ also stored the original title of a video, article, etc. and made that searchable? Doesn't necessarily have to appear on the post's page (since Youtube titles for instance can be pretty unsightly), and it would be relevant/accessible only when searching.
That is basically the idea behind the ~ metadata collection. @deimos is planning on using all sorts of metadata scapers and subject specific database APIs that should help categorize/contextualize...
That is basically the idea behind the ~ metadata collection. @deimos is planning on using all sorts of metadata scapers and subject specific database APIs that should help categorize/contextualize the submission which will help narrow down search results even if the submission title is legitimately hard to search for or the tags are too broad/generic.
p.s. Yeah I agree that coming up with set-in-stone standards for titles, especially across a wide variety of subjects and groups, is in most cases virtually impossible. But coming up with some general rules isn't hard and neither is enforcement of them. E.g. Don't use clickbait titles, try to be as descriptive and accurate as you can. Don't editorialize headlines from news organization. Etc.
Exactly. When reddit got started, the web was nothing like it is today. There was no useful metadata. Now, just about everything on the entire internet comes with a mountain of metadata attached....
Exactly. When reddit got started, the web was nothing like it is today. There was no useful metadata.
Now, just about everything on the entire internet comes with a mountain of metadata attached. News articles are already tagged when they are published. Youtube's system tags every video. Free public databases exist for all major topics (tvdb, imdb, musicbrainz, arxiv, etc etc).
When a link is submitted, the best way to handle things is for the site itself to attempt to grab all of the relevant metadata and make some kind of determination about the type of content and the topic area. Import all of the metadata at the other end of the link first, then use that to figure it out. Once that's done, a second round of data lookups can be performed against any relevant free information sources.
All of this information can be presented in the sidebar for any given submission. It can also prime the search engine and give it real power.
The ~communities and the tagging system live side by side - communities are for people to share culture and information, while the tags are for the system's tools and better content management. We've seen from reddit that communities are kinda bad at content management, and we've seen from other networks like twitter that tags alone aren't going to get you a functional community. Combine the two and you can overcome the weaknesses of both methods to build something better.
It's never going to be a 'perfect' marriage of systems, but it'll certainly be better than anything else out there right now... and we can refine it over time.
I think there are quite a few reasons why reddit's search has always been so bad. The primary one is—like so many other things on reddit—they get a passable version of the feature working, and...
I think there are quite a few reasons why reddit's search has always been so bad. The primary one is—like so many other things on reddit—they get a passable version of the feature working, and then just ignore it for years. There are tons of things that could have been done to make search better, but nobody worked on it at all for a very long time.
Other than that, just some quick things that come to mind that made it not work well:
like other people mentioned, bad titles that don't give any info about what the post is actually about
not taking into account any of the extra info available unless you specifically search those fields - flair, domain, subreddit, etc.
default sorting that doesn't necessarily reflect what a lot of people are trying to use search for - sorting by "relevance, from all time" often isn't what people want, they're often trying to figure out if something already got submitted today or similar. This isn't easy to solve, but I also don't know if that's the best default.
comment data wasn't indexed or included in any way - this could have made up for some of the other deficiencies, since comments are probably mentioning the topic even if the post itself didn't
Nope, you're right. I've got search started in dev, but still wanted to do a bit more work on it (and like @rkcr says, I didn't think it would be needed for a little while, anyway).
Anything you think is "essential" with it initially? Just basic title/text search, or was there something else that you were trying to look up?
When you have time, can you (briefly) shed some light on why exactly reddit's search function is such a complete failure? How have you differentiated the build-out for the ~ search function to keep it from devolving into spaghetti code? Just better initial planning?
Well one of the reasons I can think of off the top of my head doesn't even involve reddit Inc. The users of reddit generally use more clickbait-esque titles (often by necessity since it's the only way to get attention in the fast moving new queues) instead of descriptive ones which makes finding the content again later impossible unless you remember the exact title.
E.g. "Someone finally did it!" for a video of a motorcycle ramping off a truck car-hauling trailer. So if you search for "biker ramp truck" or any variation thereof you won't find it.
Using descriptive, accurate titles and tags along with every user making an effort to spend time in the new queues so we don't devolve into relying on clickbait, is something we are going to have to foster in ~ if we want search to work effectively here.
I see. Totally agree with the tag-based approach - forcing users to comply with a standard title formatting for a wide variety of types of content is virtually impossible. Just spitballing here, but what if ~ also stored the original title of a video, article, etc. and made that searchable? Doesn't necessarily have to appear on the post's page (since Youtube titles for instance can be pretty unsightly), and it would be relevant/accessible only when searching.
That is basically the idea behind the ~ metadata collection. @deimos is planning on using all sorts of metadata scapers and subject specific database APIs that should help categorize/contextualize the submission which will help narrow down search results even if the submission title is legitimately hard to search for or the tags are too broad/generic.
p.s. Yeah I agree that coming up with set-in-stone standards for titles, especially across a wide variety of subjects and groups, is in most cases virtually impossible. But coming up with some general rules isn't hard and neither is enforcement of them. E.g. Don't use clickbait titles, try to be as descriptive and accurate as you can. Don't editorialize headlines from news organization. Etc.
Exactly. When reddit got started, the web was nothing like it is today. There was no useful metadata.
Now, just about everything on the entire internet comes with a mountain of metadata attached. News articles are already tagged when they are published. Youtube's system tags every video. Free public databases exist for all major topics (tvdb, imdb, musicbrainz, arxiv, etc etc).
When a link is submitted, the best way to handle things is for the site itself to attempt to grab all of the relevant metadata and make some kind of determination about the type of content and the topic area. Import all of the metadata at the other end of the link first, then use that to figure it out. Once that's done, a second round of data lookups can be performed against any relevant free information sources.
All of this information can be presented in the sidebar for any given submission. It can also prime the search engine and give it real power.
The ~communities and the tagging system live side by side - communities are for people to share culture and information, while the tags are for the system's tools and better content management. We've seen from reddit that communities are kinda bad at content management, and we've seen from other networks like twitter that tags alone aren't going to get you a functional community. Combine the two and you can overcome the weaknesses of both methods to build something better.
It's never going to be a 'perfect' marriage of systems, but it'll certainly be better than anything else out there right now... and we can refine it over time.
I think there are quite a few reasons why reddit's search has always been so bad. The primary one is—like so many other things on reddit—they get a passable version of the feature working, and then just ignore it for years. There are tons of things that could have been done to make search better, but nobody worked on it at all for a very long time.
Other than that, just some quick things that come to mind that made it not work well:
Basic title/text is likely fine. I only noticed it was missing because I went to check that something wasn't posted before I made a post this morning.
There's so little content at the moment that ctrl+f functions well enough as search. :P
No need to search, bby. I'm right here for you.
<3