milkywayflyinginsect's recent activity

  1. Comment on Why normal people aren’t using AI agents in ~tech

    milkywayflyinginsect
    Link Parent
    I don't use Reddit, but I came across this comment of a guy in ChatGPT subreddit that burned through most of his 20$ monthly limits in a single day because his agent worked for 8 hours. Made me...

    I don't use Reddit, but I came across this comment of a guy in ChatGPT subreddit that burned through most of his 20$ monthly limits in a single day because his agent worked for 8 hours.

    Made me cringe so hard. I'm guessing the subscription limits are pretty high, compared to what you would get by paying directly for the API.

    1 vote

  2. Comment on What we lost when we quit using crappy old web forums in ~tech

    milkywayflyinginsect
    Link Parent
    Don't feel bad. I'm a 2000s "kid" and even though not long ago I considered myself young, more and more things are reminding me the opposite these days. I still feel 18 inside :D The first forums...

    Don't feel bad. I'm a 2000s "kid" and even though not long ago I considered myself young, more and more things are reminding me the opposite these days. I still feel 18 inside :D

    The first forums I came across were gaming related (counter-strike, runescape ...) at around 2013-2014. I can't say for all of them, but phpBB rings a strong bell for me so at least some of them must have been that.

    When did Usenet stop being a thing, by the way? Late 90's like the article mentioned? And how did it differ from modern forums?

    5 votes

  3. Comment on What we lost when we quit using crappy old web forums in ~tech

    milkywayflyinginsect
    Link
    Was interesting to find out about USENET, an early forum-like system which, by my understanding, predated the modern world wide web and used it's own custom protocols. Also, I somehow missed the...

    Was interesting to find out about USENET, an early forum-like system which, by my understanding, predated the modern world wide web and used it's own custom protocols.

    Also, I somehow missed the fact that "Discourse" is open-source. Now it makes sense why so many open-source communities use it... and it's pretty good and has A LOT of features.

    14 votes

  5. Power consumption of LLM's

    ~tech Ask
    I haven't been closely following the releases of new models and the research papers that sometimes go along with them. So I was wondering: is power consumption ever seriously talked about by...

    I haven't been closely following the releases of new models and the research papers that sometimes go along with them. So I was wondering: is power consumption ever seriously talked about by OpenAI, Google, Anthropic, and others? Do we get some specific numbers of how much power their models actually consume to produce 100 tokens? The cost of training some of these models? Or we're not yet at that stage yet and nobody cares and at best we can just get really rough estimates based on "trust me bro" tweets by their respective CEO's?

    Some time ago, I came across this post. GPT-OSS (2OB, 120B) are meager, yet energy efficient models and I was surprised that the power consumption is still larger than what I had estimated before.

    For 120B to generate a 1000 tokens (which is really not a lot) it would take up around 83 Wh of electricity. For context my PC consumes ~100Wh when idling. So it's almost equivalent as leaving my PC on for an hour. Considering that proprietary models are TRILLIONS of parameters and probably not as energy efficient the true power consumption of these models is concerning. Of course, these big data centers do a lot of things to maximize the efficiency that this "test" fails to do. But even if you half the energy consumption it's still significant given their size and their pervasiveness in handling everyday, trivial tasks.

    I haven't come across similar posts or studies for newer open source models, so if somebody has, please share.

    Also, I don't seem to fully understand how does context size fit into all of this. The larger the context size the more power it would take to produce those 100 tokens?

    26 votes

  12. Comment on Long-term experiences with Google search alternatives? in ~tech

    milkywayflyinginsect
    Link
    I try to point this out anywhere I see Kagi mentioned. Kagi is currently using Yandex as one of it's index providers thus supporting Russia. Around 2% of their expenses go every month to Yandex....

    I try to point this out anywhere I see Kagi mentioned. Kagi is currently using Yandex as one of it's index providers thus supporting Russia. Around 2% of their expenses go every month to Yandex.

    You can check the whole discussion on their discord or here :
    https://kagifeedback.org/d/5445-reconsider-yandex-integration-due-to-the-geopolitical-status-quo/19

    For me it's completely unacceptable, I urge everyone else to not use their product as well. As an alternative I use Ecosia / Qwant / Duckduckgo and Perplexity for my AI needs.

    3 votes