xk3's recent activity

  1. Comment on The internet used to be ✨fun✨ in ~tech

    xk3
    Link Parent
    Recently, I started reading these: 1/0 (Tailsteak, https://www.undefined.net/1/0/; the first 200 strips or so are the best) Absurd Notions (Kevin Pease, http://www.absurdnotions.org/page1.html)...

    Recently, I started reading these:

    Freefall also seems pretty good but I'm still in the 1999s with that one and there are new strips released even this month ! Amazing commitment

    I plan on keeping this list up to date: https://github.com/chapmanjacobd/journal/blob/main/lists/comics.list

    1 vote
  2. Comment on Chrome/Firefox Plugin to locally scrape data from multiple URLs in ~tech

    xk3
    (edited )
    Link Parent
    yeah HTML is not very strict about how information is stored because it is mostly a visual document format. Parsing unstructured data intelligently is difficult. Loading infinite pages is...

    yeah HTML is not very strict about how information is stored because it is mostly a visual document format. Parsing unstructured data intelligently is difficult.

    Loading infinite pages is comparatively easy to implement and works pretty much universally across all websites. I've never encountered a website where my auto-scrolling code didn't work.

    I spent some time over the past couple days to look into this more and I was able to get data out but because HTML is so unstructured the data is not very interesting:

    lb siteadd output.db --scroll --extract-html $URL -vvv
    

    -vvv will show the browser window so you can see what is going on. You can remove it to hide the automated browser.

    In output.db (this is just the first five rows of two tables from the first page):

    body_div_main_div_div_div_div_div (46 rows)

    title a_href span_text a_title a_text text span i_text
    Deutschkurse bei der Universität München e.V. https://adresse.dastelefonbuch.de/M%C3%BCnchen/1-Universit%C3%A4ten-Deutschkurse-bei-der-Universit%C3%A4t-M%C3%BCnchen-e-V-M%C3%BCnchen-Agnesstr.html Deutschkurse bei der Universität München e.V.
    # Agnesstr. 27, 80798 München, Schwabing-West ,, , Schwabing-West
    https://www.dastelefonbuch.de/service/free-call/Deutschkurse%20bei%20der%20Universit%C3%A4t%20M%C3%BCnchen%20e.V./089%202%2044%2010%2049-0/1126/0099?freeCallCmd=3038393234343130343930404034383564396166633863666235653635386635623766363865363334616339613333346630643639&service=pre ... Gratis anrufen Gratis anrufen 49-0 {"text": "089 2 44 10", "span": {"text": "..."}}
    http://www.dkfa.de www.dkfa.de Homepage
    Branche: Universitäten, Sprachschulen

    body_div_main_div_div_div_div_div_a_span (30 rows)

    text
    Agnesstr. 27
    80798
    München
    Ettensberger Str. 1
    87544

    So my tool is probably not useful for the German phonebook website but it might work okay on another site. It depends how they structure their HTML.

    But maybe if you know SQL my tool will make HTML parsing more accessible to you and you might be able to make use of this data after some additional transformations...

  3. Comment on Chrome/Firefox Plugin to locally scrape data from multiple URLs in ~tech

    xk3
    (edited )
    Link
    The website you linked to loads additional pages as HTML. I wrote something with seleniumwire that can do this if the data loads with JSON. It is a bit experimental and the results might be spread...

    within a website such as

    The website you linked to loads additional pages as HTML. I wrote something with seleniumwire that can do this if the data loads with JSON. It is a bit experimental and the results might be spread across a lot of different tables but it works.

    pip install xklb
    lb siteadd output.db --scroll $URL
    

    For HTML you might be able to use xidel to extract the content that you want.

    For kicks, I tried adding the same functionality for HTML, but I get a lot of this:

    xml.parsers.expat.ExpatError: not well-formed (invalid token): line 19, column 24

    which I'm not very surprised by. HTML is difficult for programs to parse. I'm sure it might work for some websites but certainly not the German phonebook site.

    1 vote
  4. Comment on Does something like a charity fund for FOSS exist? If not, do you think it could be a good idea? in ~tech

    xk3
    Link Parent
    For sure. I think they didn't have enough momentum and kind of fizzled out? Perhaps delayed and flawed by perfectionism. Every.org seems to have taken the opposite approach but now it is a...

    For sure. I think they didn't have enough momentum and kind of fizzled out? Perhaps delayed and flawed by perfectionism. Every.org seems to have taken the opposite approach but now it is a platform with actual active users. It would be interesting if they could experiment with monetary instruments like cooperative donating, etc.

    I like the work that https://numfocus.org/ is doing. I think there is some hope for collective style of non-profit administration. But a lot of volunteers and non-profits don't have the time, prestige, or the portfolio to onboard with a successful collective. I write open-source software and I really enjoy doing it. It's a lot more fun than applying for jobs and hearing nothing back so I'll probably just keep doing it regardless of donations and the number of lentils in my cupboard.

  5. Comment on Does something like a charity fund for FOSS exist? If not, do you think it could be a good idea? in ~tech

    xk3
    Link
    This centralized curation is a tricky problem. Although there have been many interesting systems like Snowdrift.coop, most of these projects are either not very successful or the money flows are...

    between some curated projects (most influential/important/promising/underfunded/etc)

    This centralized curation is a tricky problem. Although there have been many interesting systems like Snowdrift.coop, most of these projects are either not very successful or the money flows are biased towards a specific group of people.

    however, I can't afford to donate to all of them

    Then don't. A few large direct donations will do more good than many small donations even while it might feel like you are doing more good.

    Donate to the projects that you like or use the most. If everyone did that then we would enjoy the benefits of decentralized curation.

    4 votes
  6. Comment on What programming/technical projects have you been working on? in ~comp

    xk3
    Link
    Over the past couple days I did some research into space-efficient modern codecs and I settled on AV1 -c:v libsvtav1 -preset 8 -crf 44 for video AVIF 2400px for images Opus 96k for audio Pretty...

    Over the past couple days I did some research into space-efficient modern codecs and I settled on

    • AV1 -c:v libsvtav1 -preset 8 -crf 44 for video
    • AVIF 2400px for images
    • Opus 96k for audio

    Pretty neat to see my phone camera folder go from 20GB to 2GB. Of course the quality difference is noticeable when comparing against the originals (except for Opus... I can't really tell) but for images and video the difference is small enough for me.

    5 votes
  7. Comment on In-flight canoodling: is it ever acceptable to spoon at 40,000ft? in ~travel

    xk3
    Link Parent
    I think most people don't really care about the exact altitude of the plane because it is a figure of speech. I think the word for it is metonymy? If the article said "at or above 30,000 feet" or...

    I think most people don't really care about the exact altitude of the plane because it is a figure of speech. I think the word for it is metonymy?

    If the article said "at or above 30,000 feet" or "between 33,000 and 42,000 feet" instead of 40,000 feet it would seem oddly specific and it might not make people think of air travel as immediately

    14 votes
  8. Comment on My movie schedule in ~movies

    xk3
    Link
    Great list! Hope you're able to stick to it. If you have a list you can programmatically make an ICS file which you can import to Google Calendar pretty easily. I do a similar thing but for places...

    Great list! Hope you're able to stick to it.

    If you have a list you can programmatically make an ICS file which you can import to Google Calendar pretty easily. I do a similar thing but for places around the world:

    https://github.com/chapmanjacobd/everydayvirtualvacation/blob/main/cal.py

    edit: ah I see your linked post. It would be cool if you could share the ICS file

    2 votes
  9. Comment on Unraveling Havana Syndrome: New evidence links the GRU's assassination Unit 29155 to mysterious attacks on US officials and their families in ~science

    xk3
    Link Parent
    If the atom bomb was not public knowledge it would seem like a more unbelievable conspiracy theory. I think we would have roughly the same level of deterrence and effective countermeasures if that...

    If the atom bomb was not public knowledge it would seem like a more unbelievable conspiracy theory. I think we would have roughly the same level of deterrence and effective countermeasures if that weapon was still a secret shared only among government agencies.

    It's very possible that a sonic weapon of some kind exists. It's not outside the realm of possibility that it existed before we figured out how to split the atom. There are also good reasons for denying knowledge of any weapon for both US and Russian governments.

    It's also very possible that it doesn't exist but that what started as a psychological break has turned certain departments across the globe into a prototyping frenzy--or that they have thought about it but because guns and poison darts exist they have not put much forethought into actually developing it.

    But war is best fought at the psychological and morale levels--perhaps this is all propaganda perpetuated by specific or non-specific countries for various purposes.

    1 vote
  10. Comment on What did you do this week (and weekend)? in ~talk

    xk3
    Link
    This week I went to China and visited Shenzhen and Guangzhou. There's this moderately popular chain fish restaurant called Taier (太二) and they have this really delicious pickled "sauerkraut" fish...

    This week I went to China and visited Shenzhen and Guangzhou. There's this moderately popular chain fish restaurant called Taier (太二) and they have this really delicious pickled "sauerkraut" fish dish. I also ordered a cute plush fish keychain (4CNY; about $0.5 USD) with my meal and it came with two fairly substantial bags of cat food (maybe 1kg total). Apparently, they give out free cat food to anyone that walks by the restaurant and shows them the keychain--although I tried to do that at the location in Macau and they didn't know what I was talking about so it seems like it is only a mainland China thing.

    2 votes
  11. Comment on Unraveling Havana Syndrome: New evidence links the GRU's assassination Unit 29155 to mysterious attacks on US officials and their families in ~science

    xk3
    Link Parent
    I don't think it would be that expensive to develop something like this. For all we know this could have been prototyped and developed prior to 1990s

    I don't think it would be that expensive to develop something like this. For all we know this could have been prototyped and developed prior to 1990s

  12. Comment on Why do some people posting ChatGPT answer to the discussion/debate/question? in ~tech

    xk3
    Link
    I am an optimist. I think people that do this are well-intentioned. People feel lonely and want to socialize. They want to make friends. But not everyone is an expert on every topic and so they...

    I am an optimist. I think people that do this are well-intentioned. People feel lonely and want to socialize. They want to make friends. But not everyone is an expert on every topic and so they ask an "AI" to help them communicate with another human. It's better that they preface something with "I asked ChatGPT..." but I think eventually people will stop prefacing just like they have stopped prefacing "I asked Google..." even though they are totally different tools people are using them in a similar way with about the same amount of media literacy as before

  13. Comment on What programming/technical projects have you been working on? in ~comp

    xk3
    Link Parent
    I think that's reasonable. Depending on how deep you want to go with this you can get some UX ideas from https://kittensgame.com/web/

    every n seconds automatically mine 0-20 of each base resource & randomly craft as many products as possible

    I think that's reasonable. Depending on how deep you want to go with this you can get some UX ideas from https://kittensgame.com/web/

    3 votes
  14. Comment on What programming/technical projects have you been working on? in ~comp

    xk3
    Link Parent
    Cool this is an incremental game/demo. You might consider adding a button to turn it into an auto-idle game

    Cool this is an incremental game/demo. You might consider adding a button to turn it into an auto-idle game

    3 votes
  15. Comment on Fun programming challenge: figure out which sets of passports grant visa-free access to the whole world in ~comp

    xk3
    Link Parent
    Also, it would be interesting to see in comparison how many days it would take / how much money you'd need to spend to get a set of visas rather than another passport. Of course getting a visa and...

    Also, it would be interesting to see in comparison how many days it would take / how much money you'd need to spend to get a set of visas rather than another passport. Of course getting a visa and being able to use it to enter a country is not a "given" but neither is getting a passport a "given" for many countries

  16. Comment on 100,000 years and counting – how do we tell future generations about highly radioactive nuclear waste repositories? in ~enviro

    xk3
    Link Parent
    It's an unfortunate story but I think radiation in general is not very scary... It's just an invisible and very portable fire that you can't see or feel directly but can kill you. It exists...

    It's an unfortunate story but I think radiation in general is not very scary... It's just an invisible and very portable fire that you can't see or feel directly but can kill you. It exists everywhere at low levels but some rare places burn hot. Perhaps we live in hell surrounded by small invisible flames--but probably not. But if you live somewhere a long time it might be good to check for radon and radiation in building materials.

    Thankfully the list of global incidents is pretty short--though that could simply be because it's not the first thing people think to check

    1 vote
  17. Comment on Seems like all socials are being scraped for AI and personal/aggregate data. Is Tildes? in ~tildes

    xk3
    (edited )
    Link
    One thing I noticed when backing up my personal data via scraping Tildes is that if you don't have a cookie you can only scrape the first page of my user. I think Tildes is pretty small so any...
    • Exemplary

    One thing I noticed when backing up my personal data via scraping Tildes is that if you don't have a cookie you can only scrape the first page of my user. I think Tildes is pretty small so any custom script to scrape data is probably not worth it to companies

    8 votes
  18. Comment on 100,000 years and counting – how do we tell future generations about highly radioactive nuclear waste repositories? in ~enviro

    xk3
    (edited )
    Link Parent
    It makes me think about the Goiânia incident... I think any indication that there is something buried is worse than leaving the land as boring as possible. I think the location is important: bury...

    People before and after us weren't and won't be stupid

    It makes me think about the Goiânia incident...

    I think any indication that there is something buried is worse than leaving the land as boring as possible. I think the location is important: bury it tens of meters deep and far from bodies of water so that future human settlements are not likely to go there. There are places in the world where fewer animals live simply because they are so far from water--those are the ideal places to put this stuff

    10 votes
  19. Comment on 100,000 years and counting – how do we tell future generations about highly radioactive nuclear waste repositories? in ~enviro

    xk3
    Link Parent
    But "good" colors and "bad" colors are very culture dependent. Not that we shouldn't make a mood ring cat... but if the sign is interpreted as its opposite then we have more people digging up...

    But "good" colors and "bad" colors are very culture dependent. Not that we shouldn't make a mood ring cat... but if the sign is interpreted as its opposite then we have more people digging up radioactive waste than not bringing any attention to the place.

    3 votes
  20. Comment on Any friendly entrepreneurship communities that aren't rotten with the whole "grindset," hustle culture stuff? in ~life

    xk3
    Link
    I've found that "Maker" spaces and other innovation-focused organizations often have healthy entrepreneur and leadership/mentorship groups co-located or nearby

    I've found that "Maker" spaces and other innovation-focused organizations often have healthy entrepreneur and leadership/mentorship groups co-located or nearby

    8 votes