Discussions of progress or collapse often get mired in the question of significant discoveries and inventions. After wrestling with several organisational cencepts for various catalogues, and running into the Ever Growing List dilemma, I hit on what I call BOTI, or Best of the Interval (day, week, month, year, decade, century, etc.). It's similar to the tickler file 43 folder perpetual filing system of GTD. For technical types, a round-robin database or circular buffer.

(As with my bullet journal experiments, the effort is uneven but recoverable, which is its core strength.)

By setting up a cascade of buffers --- day of month, (optionally week or weekdays), month of year, year of decade, decade of century, century of millennium, millennium of 10kyr, a progressively larger scale record (roughly order-of-magnitude based), with a resolution of day but a maximum retention of (here) 10,000 years but only 83 record bins. How much you choose to put in each bin is up to you, but the idea is that only to most significant information is carried forward. Yes, some information is lost but total data storage requirements are known once the bin size and count are established.

Another problem BOTI addresses is finite attention. If you limit yourself to a finite set of items per year, say ten to one hundred (about what a moderately motivated individual could be aware of), BOTI is a form of noise-filtering. Items which seemed urgent or captivating in the moment often fade in significance with time, and often overlooked element rise in significance with time and context. 'Let it settle with time" is a good cure to FOMO.

There's the question of revisiting context. I'd argue that significance might be substantially revised years, decades, possibly centuries after a discovery or inventiion. So an end-of-period purge of all but the top items isn't what we're looking for. Gut a gradual forgetting / pruning seems the general idea.

Back to science and technology: It's hard to assess significance in the moment, and day-to-day reports of science and technology advances are noisy. I've been looking for possible sources to use and am finding little that's satisfactory. I'd like suggestions.

• Many newspapers and magazines run annual "best of" features. These typically include books, but not science (or at least not regularly). Some of the books are science- or technonolgy-related, though.
• There are the Nobel prizes, notably in physics, chemistry, and medicine, with lists at Wikipedia (linked). The Fields Medal in maths. Other fields have their awards, of which lists might prove useful...
• I'm having trouble finding something like a yearbook of science or technology, though some titles match, e.g., McGraw-Hill yearbook of science and technology. On closer look, this might answer my question, at least for yearbooks.
• Wikipedia has some promising but either inconsistent or untidily organised pages or collections, including the List of years in science, Timeline of historic inventions, Timeline of scientific discoveries, Timeline of scientific thought, among numerous other timelines. Compilations are useful but aren't themselves rankings. See also "never ending list" above.

There is a goal here: trends over time. I've a few senses of directions of research and progress, possibly also of biases in awards. Looking at, for example, Nobels in physics, chemistry, and medicine from, say, 1901--1960 vs. 1961--2020, there seems to be a marked shift, though categorising that might be difficult. The breakpoint isn't necessarily 1960 either --- 1950 or 1940 might be argued for.

There is the question of how to measure significance of scientific discoveries or technological inventions. I'm not going to get into that though several standard measures (e.g., counting patents issued) strike me as highly problematic, despite being common in research. Discussion might be interesting.

Mostly, though, I'm looking for data sources.

Stop me if you've heard this one before:

• You get an idea for something you'd like to learn more about. (Maybe you have a question, maybe you want to explore a new hobby, or maybe you want to make a more informed decision.)
• You type something into a search engine.
• You click a result, only to realize that what you're reading is poorly written. It seems rushed, surface-level, and ill-informed. "This doesn't answer my question at all!" you think to yourself.
• You go back, and try another one, and another one, only to give up and put the idea back in your head.

I don't think these webpages are written to be useful in the first place. They seem to be written to attract attention to the website for other reasons (ad revenue, affiliate links, to draw attention to a product or service). Regardless of why it's happening, though, I want to find a better way to search.

The sort of content I'm looking for is written by someone who really cares about the topic. I want to learn from dorks and nerds and passionate people. Once I stumbled across this blog about extra virgin olive oil. The website isn't pretty, and it goes way more in depth than I'll ever need, but I trust the author, and there are some really interesting nuggets of insight on these pages. (e.g. "Another myth debunked: Heating EVOO makes it ‘toxic’")

Do you have any tips or tricks to more reliably find these sorts of sources (whether online or in-person)?