38
votes
Fintech founder charged with fraud after ‘AI’ shopping app found to be powered by humans in the Philippines
Link information
This data is scraped automatically and may be incorrect.
- Authors
- Charles Rollet
- Published
- Apr 10 2025
- Word count
- 305 words
Yes, but openly and transparently
But it's also what Amazon's "Just Walk Out" grocery shopping turned out to be, kind of. The stores claimed to be AI-powered, that cameras monitored your purchases and charged you based on what you walked out of the store with. It turned out, Amazon had about 1,000 employees in India monitoring the monitoring. There's now a joke that Amazon believes AI stands for Actual Indians.
On one hand, it makes sense, especially for new tech, to aggressively audit for system accuracy. But with the discovery that they were "auditing" 70% of purchases, it strikes me as another example of Amazon over-promising and under-delivering.
In the end, they opted to shutter the program, developing the Dash Cart system instead, which scans your items as they go into your cart. In the end, this makes a lot more sense, monitoring purchases at the cart's-eye view rather than the store's-eye view. No news yet on how many Indians they're using to run it, though.
It's the approach they should have taken to begin with. A lot of tech companies put way too much effort into using computers to solve a human problem as seamlessly as possible, to the point that they never think about how tweaking the parameters of the problem can make it 90% easier for a computer to solve. Having the scanning done in the cart instead of a complex network of overhead cameras and sensors may not have the same magical "wow" factor, but it's just as easy and convenient for the end user.
A story I heard of a robotics company trying to make a towel folding robot comes to mind. They struggled with reliability until someone had the idea to just color the corners of the towels, vastly simplifying the complexity of the computer vision half of the problem.
It's much easier to fix a policy problem with a policy solution than a policy problem with a technological solution.
Am I understanding you correctly that the resulting technology "developed" from attempting to have a fully transaction-less process, is "Dash Cart" which is actually an extremely old technology, aka Scan and Go? These have been a thing since at least the early 2000's in normal grocery stores. I'm also curious, does Dash Cart use dedicated scanners or can the user use their phones? That would be the only level of "innovation", imo, but really that is not innovation it's really translation of an already existing idea/product.
Haven't the faintest; I live in rural northern Wisconsin. I can't even get high speed internet, much less a smart grocery store.
But from what I'm reading online, the customer doesn't need to do the scanning. You put something in your cart, and the cart knows what you put in the cart, via various signals. Someone in the developed world will need to chime in to confirm my inferences though.
I used one around the time they launched and it had cameras that used ML to detect what was being put into it. It was incredibly unreliable. I came back about a year later and they had changed it so that you had to actually scan the package. So they were basically just those self checkout kiosks on wheels - more reliable but still very finicky. I’m sure they have changed them in the years since.
A few years ago UniQlo had (and still has) scan-less shopping, albeit not as you’re wandering around the store. You pick up whatever clothes you want, and you dump them all into a tub build into the checkouts, and it immediately registers exactly what items you’ve put in.
I believe every item of clothing has an embedded NFC or something, and the tub probably has a bunch of readers around the surface.
It still feels magical to never have to scan any barcodes, just unload armfuls of clothing into the magic bucket and scroll through the list on the screen to confirm and then scoop them out to put in your shopping bag and go. Makes my occasional bulk t-shirts trip a lot faster than scanning each tag one at a time
Edit: I just looked it up, it’s RFID rather than NFC, however I have no idea what the distinction is between these technologies.
Okay you described what I witnessed in Chile and tried to explain to other people, with no success lol.
I’m still wondering how this works, because it was so accurate ! Clothes. Food. Etc. Your explanation makes more sense than whatever my brain came up with, but I would still love to see a video or otherwise learn how these work.
Nah it's Artificial² Intelligence, they didn't read the fine print.
It was my understanding that most of these AI firms are taking the "fake it till you make it" approach with AI algorithms right? Because with LLM generative AI we still don't have general intelligence, and we're all still wrestling with hallucinations, so ALL fintech and all tech claiming AI are faking it right? I guess it's only a problem when they run out of runway funding and investors begin to actually ask questions?
I mean, even setting aside that there's tons of AI from well before LLMs and GenAI that's still useful for various applications, this simply isn't true even of places jumping on the current GenAI hype train. Most of the tasks various tech companies (including most fintech) are trying to apply generative AI to don't strictly require general intelligence, and how to avoid hallucinations harming the output and having downstream effects is something that's very task-dependent and something that can be worked to avoid and mitigate.
Plus, not all tasks that someone might want AI for can even easily be outsourced like this -- many can, but whether it's possible or cost-effective also varies a lot based on the specific task. It's often not even cheaper when it is possible, even when underpaying workers in countries with lower wages.
I'm not surprised there are companies trying this tactic to benefit from AI hype, but tech companies using GenAI irresponsibly is a bigger risk in general than them lying about using AI like this company did, tbqh.
Ooooh thanks for chiming in, definitely not well informed on the subject
It's simply the Silicon Valley way.
As I read this, it would seem that A²I is preferable. I wonder if people cost less than machines for this work. If so, then people would be preferable from a business perspective. I am doubtless not fully informed, but this doesn't seem like the kind of dangerous work or drudgery for which machines would obviously be preferred.
It just seems that we should not view paying machines more as inherently preferable to paying people less for the same task.
The only way paying humans is cheaper for tasks like this is when you're woefully underpaying them. I'm not sure that should be celebrated.
Am I the only one who felt weird after reading “powered by humans in the Philippines”? lol I don’t know what it is about this phraseology that irks me.
No, you're not, it's definitely a dehumanizing Matrix-esque way of thinking about people
Weve finally achieved Artificial Artificial General Intelligence 👍
👋 Mission Accomplished 👍