Do you think you'd use a hardware-based file sharing solution?
All major operating systems have their own file-sharing protocols (AirDrop, Nearby Share, etc.) which are incompatible with each other. There do exist apps for "cross-platform file sharing", however, they require installation on both the sender and the receiver devices, which makes it a bad solution for quickly sending files to other people.
I was thinking about making a file sharing solution that acts as a bridge between those different protocols and only requires action from one of the parties. However, there's a huge issue with AirDrop - it uses a custom networking protocol called AWDL. There are open implementations like OWL, but they still require low-level wifi access, so it can't be run on Android without kernel modifications.
This means that the only way this can be implemented is by using a physical device, eg. a Raspberry Pi Zero W or a similar microcomputer that runs a custom firmware. An obvious problem is that it requires a lot of effort - you need to buy a Raspberry, flash it with the firmware and carry it around with you, just to be able to send or receive files to other devices without using third-party websites.
So I'm personally not sure if that's worth making. But maybe you'll still be interested? Is this a big enough problem for you to want to use a hardware solution?
I'd suggest asking some folks how they currently solve sending files when airdrop isn't an option. I expect the answer to be messengers or email. Less elegant and requires internet, but I think that'll almost always beat carrying an extra thing around and powering it.
It sounds like a really cool hack, though, and if you want it for yourself you should build it!
Yep, Telegram saved messages for me. Going between Linux/MacOS/iPhone/Android on my devices.
My file sharing solution is a Discord private message channel with just myself and an alt account. As a college student, I do homework on my iPad, and then submit it on my desktop or laptop since Gradescope is a pain on mobile. It's essentially instant file transmission that only needs an internet connection, and since all my devices have Discord, I can access it from anywhere. This comes in handy if I need to quickly show a friend a homework problem on my phone, for example. As a bonus, everything is stored in reverse chronological order, with the newest files at the bottom.
This also works well for sharing stuff with friends, because they all have Discord accounts.
I prefer this over something like Google Drive because organizing stuff into a folder when uploading and then finding that folder to download from is much more time-consuming. Don't want to be spamming my email inbox with random homework files either.
I don’t want to be that “have you tried…” asshole when you’ve clearly got a system that works for you, but I’m genuinely curious since it’s so different to my way of doing things: wouldn’t dropping everything into the root of GDrive/iCloud/Dropbox/whatever without folders do the same job, but with file search and sync as well?
Not OP, but do the same with quite a few reasons. First and foremost, because I very much minimize what programs are running on my devices. But Discord is generally always running on my computer regardless since I'm often using it to communicate. So there's no extra programs and not really any delay in opening (except on my phone).
I don't really enjoy Cloud based storage because my internet is painfully slow to the point where even a small 5mb upload can cause my internet to hang for a solid minute plus. So having my devices constantly syncing can interfere with other things I might be doing over the internet. With Discord, I just upload whatever I need transferred, wait for it to finish, and go back to whatever I was doing never worrying about it again.
Another big reason is it doubles as a notepad. I can see the files and associated notes without any opening anything or clicking on anything. It's just there. I imagine that would also be helpful when using Ctrl+F to find relevant files, but I don't store nearly enough on it for that to be relevant.
Really though, most of what I use it for is a notepad with embeds. The file storage just happens to be there as a bonus that occasionally gets used when I need to transfer something. It's no good for something like a lists or documents, but if I have some text or a link I want to save, I just throw it in Discord with no delay since it's already open and just have it there.
I did also just start using as an "image sharing" service earlier this year too, since you can direct link images uploaded through Discord. Unfortunately, a lot of others must have also started doing this because they recently announced that won't work outside of Discord by the end of the year any longer.
All that said, I'm sure there's a better alternative, but I barely use it and it gets the job done. So I really can't be bothered to find something better.
Thanks! That "notepad with embeds" description totally made it click in my mind - makes a lot more sense to me now.
Wuju made a lot of good points. Sending links/copy pastes between devices is a big thing that can't be done in a cloud drive. I often find myself sending a link to a website that would be a pain to navigate on my phone so I can open it on desktop. Also good for short notes/reminders.
About file search: for 99% of the stuff that I send myself in Discord, it's only a one-time thing. I rarely need to open a homework assignment 2 weeks after I submitted it, for example. If I do, I can pin it in the channel but I haven't yet needed to for anything.
As for sync: all my devices have Discord, so it's essentially synced. If I don't have an internet connection, a cloud service wouldn't work either.
I did a quick speed test comparing uploading a pic from my phone to my GDrive root and downloading it onto my PC to uploading that same pic from my phone to Discord and Discord download to PC. Note that I use the browser version of GDrive, not the desktop. To keep things fair, I started with Discord closed and no GDrive tab open (for my typical use case though, Discord is open all the time but not GDrive).
Results: 25.5s for Discord, 28.6s for GDrive. Probably would have saved 3-5s if Discord was already open.
So, it's around the same time with Discord a bit faster. Probably wouldn't beat Airdrop between 2 Apple devices, but I think this is the fastest way I can transfer files between my devices.
The only issue is with >25MB files since that's Discord's limit. But in my use case, those are very very rare and I can just resort to GDrive.
You don't even need the alt account, you can create a server with no invites & never add anyone else to it. Then you can have as many files as you want, and store notes, screenshots, use it for filesharing, etc. (This is what I do.)
Yes, I use Telegram myself for sharing files
It is a cool hack which could be interesting to make, but knowing myself, I won't have enough motivation to make something that other people aren't going to use
I've been using KDE Connect to send from and to iOS and Linux/Windows.
I use WhatsApp or Discord to send to friends or when KDE Connect isn't cooperating on PC.
By having nothing but USB-C, I tend to use rsync to send to and from my MBA and a Linux PC.
I've seen mentions of Localsend which seems like it could be much more convenient than using rsync or KDE connect, though sending to other people would still depend on WhatsApp, Discord or a cloud storage.
Do you have any tips for getting started with rsync? My use case for it would be – I think – a manual backup of the two most important directories on my system, but after hacking up a basic script a while back, I still seemed to be missing something as it didn’t work.
Add a -v —dry-run to your call and it will show you exactly what it would do outside of a dry run!
My typical use is:
Where:
Note that rsync treats
dir/,diranddir/*differently, and it’s not always intuitive what it’s doing. My rule of thumb is to make sure you use the same ending for both your source and destination, and when in doubt use dry-runPast that
man rsyncis your friendI suggest reading the manual in “man rsync” like arghdos said.
I use rsync basically by the power of reading the manual and random posts if something doesn’t work or if I need something more specific.
I think you need an ssh server installed if you’re copying from a Windows PC on network and unlike most(?) Linux distros, It isn’t installed by default.
Trial and error can get you there, but some things like keeping the folder structure can be tricky. Once you have a working syntax though you can reuse it with minor modifications depending on what you need.
Check out Borg. I think it's a better solution for backups as it keeps multiple snapshots, encrypts and dedupes.
Right now the main thing I use for this is snapdrop or one of its many instances and forks such as the now leading https://pairdrop.net project.
Useful for one-off transfers of data between my phone and PC or between phones. No apps needed and the data transfer is encrypted and point-to-point thanks to WebRTC.
It's open source too so you can inspect how things work under the hood or host your own instance.
Super cool, but it's being hugged to death at the moment. Here's the git repo:
https://github.com/schlagmichdoch/pairdrop
I haven't measured the performances of paidrop compared to their native counterparts but it has worked super well for my use cases. The existence of pairdrop kinda negates the need for a hardware file transfer device IMHO.
It negates the need in practice, but unfortunately not for all use cases on a philosophical level: As far as I can tell, we still have to go sending device → server → recipient device even if both parties are within an arm’s reach in the same room, iff there is no shared network connection around (e.g. being outdoors, sharing pictures after a common activity, which would be my most common use case).
But that is high-level criticism for sure. In any case it is a super neat project, and anything that boosts cross-platform compatibility these days is welcome. And if devices are in fact in the same network, the above concerns don’t even apply. Also like you said, I found it to seem more than speedy enough from my quick tests just now.
I don't own any Apple devices, but "Send file over Bluetooth" has been pretty reliable between Linux/Windows/Android. Just gotta pair them.
I also use KDE connect if it's installed.
Bluetooth file transfers are painfully slow though. Newer versions might be a bit faster but I found it really annoying when it would take over a minute for a single photo to come across.
That could well be true....I never really tried anything larger than a few MB. I don't recall a photo being a problem but even with new versions I'm guessing a short video would be quite annoying.
Yeah I think the biggest appeal of OP's device would be the throughput.
I'm asking more about sending files to other people, not to your own devices
Yea, I did this too. We'd pair our phones together then just hit "Send File" and "Accept File" respectively.
Admittedly we were only exchanging a few ebooks.
Does such a solution need to be wireless?
There is this thumb drive with both a lightning and USB-C connector. I guess one downside is having to do the transfer twice, but if you're going to carry something around, this plus a USB-A adapter for older PCs is probably the simplest option.
If you were going to make a device of your own, I imagine a wired device would be simpler to write code for and have the advantage that you could draw power from one of the connectors. If you made it mirror each devices file system to the other device, that would be quite an achievement.
I recall seeing a gadget like this (but not at all portable) at a Tmobile store that the rep used to sync files from my old device to my new one, but that was years ago. That would probably be the place where something like this would be most used -- lots of transient devices of different makes and models and a good reason to connect them together.
It’s kinda nuts that this is even a problem in 2023, that the demise of open standards has gotten so bad that there’s no easy way of getting a text document between two popular mobile operating systems despite them being capable of 1gbps+ wireless connections.
I really wish legislation to address this sort of thing had any sort of traction, because by this point it’s abundantly clear that companies are always going to favor walled gardens to pad their bottom lines and enforce lock in at the expense of everyone else’s well being.
I think it’s also that the internet has abstracted away a good chunk of that desire for local interop. It’s frustrating that we need to rely on pinging something to and from a server just to get it between two devices in arms reach, but it’s also not a massive pain point day to day just emailing/messaging whatever file needs to be shared. The UX is about the same as it would be for purely local sharing in 95% of cases, even if the underlying approach is significantly less efficient.
Then again, just messaging it in the US is it’s own problem, which loops back to precisely your point about walled gardens, and that’s only fixed outside the US because the walled garden everyone else uses happens to work on both platforms…
Ugh, not Google's RCS bullshit again.
One thing that video didn't mention is that Google's Messages app also uses SMS to contact iPhones as well as every phone that doesn't have Messages or another app from a partner that licenses google's proprietary RCS implementation. It also misses out that there is absolutely nothing stopping Google from making an iPhone Messages app to solve that problem. And they're also forgetting to mention that the RCS implementation is a proprietary system that Google owns and operates, and that encryption is a proprietary extension they developed and not actually part of the standard.
I'm fine with criticizing Apple for not opening up iMessage, but Google is just the pot calling the kettle black. There are many messaging apps you can use that work perfectly fine on both iPhone and Android which have most or all of those advanced features. It appears that the English speaking world outside of the US are perfectly fine using WhatsApp regardless of what phone they choose.
Can you elaborate? iMessage is a proprietary and requires being an authenticated Apple account holder, does it not? It is not like SMS/MMS/RCS in that way.
I think they meant Google making an RCS Messages app for iPhone, not an iMessage Messages app for Android.
I’m not entirely sure what you want me to elaborate on. Both Messages and iMessage is proprietary. RCS as it is implemented is also proprietary. Google is just trying to pretend that it isn’t to force Apple to comply with their demands.
That’s interesting, it’s pushed me to do a bit more reading on something that I thought was just a general GSMA standard. If I’m understanding correctly, it’s designed such that each mobile network should be running their own, interoperable RCS servers - but in practice very few of them are, so Google has decided to route everything through their own servers instead? If I’m understanding that correctly, and particularly if Google are making that choice unilaterally rather than integrating with the carrier-run servers that do exist, then that’s way different to what I understood the situation to be.
As I understand it they do still allow third party clients, though? If that’s something I’m wrong about or something they could rescind in future then it’s hardly much comfort, but seems like a potentially significant improvement. Not significant enough to sway me if it isn’t an actual shared standard like I thought, but still something.
For what it’s worth I’m in a WhatsApp market anyway. It’s a significant practical improvement: hardware choice isn’t dictated by one software function that has no reason to be hardware specific, but it’s still a total proprietary lock in, which pisses me off in practice (very poor multi-device and backup support, until recently) and in principle. Even Signal’s not doing a great job on that front, and they’re open source.
Google publishes no public API for one to send an RCS message the same way they had for SMS/MMS. The only working alternatives I am aware of for Google’s Messages app is Samsung’s messaging app and some carrier created messaging apps that license Google’s tech. I’d love to tell you exactly how their messaging platform works but it’s completely proprietary so it’s effectively a black box.
I have skimmed through the RCS Universal Profile and it is really short on actual implementation details. From what I have read there were many different RCS implementations at one point but it doesn’t actually seem like any of them had been interoperable. It’s the reason why most people have not even heard of RCS before Google started working on it. Now every US carrier is using Google’s software to run their RCS services on the back end.
It's not like open standards died. They're still around. It's just that they're not as convenient as the proprietary stuff so nobody uses them. There's nothing stopping you from transfering your files via FTP that didn't stop you in the past.
It's that pesky network effect thing again. I don't want to use Facebook, but if that's the only place that I can get updates from local organizations, I don't have much a choice.
FTP was easy back in the day when browsers supported it and a very high percentage of users had a client for more advance use. It was just as easy as "Here's the link". But now it's "Well go install this software that used to be fairly ubiquitous but is now arcane to anyone born after 1997".
I'm not saying browsers were wrong to remove FTP. But man, wouldn't it be nice if we could have educated everyone just a weeee bit more so that we could just use SSHFS and SFTP on all our devices?
To be fair, FTP was an awful standard and people only used it because it was the standard and had a name that was brilliant marketing-wise. In other words, its usage was based on expedience, just like modern proprietary alternatives.
Too often, even the FOSS world will prioritize "but it's the standard!" and 'network-effect pragmatism' over architectural quality.
That article feels really unfair. The FTP RFC was written before the web was invented and the internet as a whole was limited and largely limited to universities and research institutions. It defaults to ASCII as it would have been the overwhelmingly common thing to transmit because binary files were often not worth sharing. IIRC it was written with the assumption you may not even be using it over TCP/IP.
The reason why I brought it up was expediency, like you mentioned, and ubiquity. In reality there are many more options to choose from like NFS or SMB.
In the end, meeting standards has value. I agree FTP belongs in the dustbin of history though. SFTP is just so damn good though, and it's a shame that people associate the two together.
Very, very tangential, but I always find SFTP surprisingly slow to deal with folders containing a largeish number of files. For the amount I use it it’s never been enough of an issue to actually dig further into, but since you mention it: is that per-file overhead in the protocol or have I messed up the config somewhere?
Those proprietary things are only proprietary for the purpose of lock-in. Why isn't airdrop an open protocol?
Actually there are a lot of things stopping me. My phone, and more importantly, no one I’d want to share files with natively support FTP.
There’s nothing stopping phone manufacturers from allowing their file transfer protocols to be implemented by competitors, except that they think they can make more money by purposely making their customers lives less convenient.
The last time I checked there are FTP clients for every phone platform. Android has a wide selection of server apps, and I assume iOS has them as well but aren't in a place I can look right now.
Yeah, but they don’t natively support it. I can’t just tell someone to grab a file from my phone via ftp. I’d have to configure and install an FTP server app, and they’d have to install and configure an ftp client.
There’s nothing stopping apple and google from enabling that functionally, or some other modern file sharing protocol natively, they don’t for business reasons.
That’s what someone means when they say a protocol dies. Not that all human knowledge of that protocol is gone, but it’s not in widespread use.
Open protocols used to be the default which were used everywhere. Email was sent by smtp and retrieved by pop, files were shared with ftp, messages were exchanged with irc or xmpp. Voice communication happened with SIP and RTP. Those things are mostly dead and replaced by proprietary protocols that only work within the vendors walled garden now.
People exchange messages with slack or ms teams. They voice chat using discord. They share files with airdrop. It doesn’t have to be this way; companies could collaborate and create modern open standards, they just figure they can make more money by locking their clients into their ecosystem via network effect and then crush their competition, which leads us all worse off.
Having companies compete to make better discord clients, or better slack backend software, or better implementations of airdrop on their devices sounds like a much better world than the one we live in now.
This would be a good solution for transferring files between your own devices, but not when you have to send group photos from your Android phone to your friends iPhone
Maybe I am missing something. When you were talking about hacking low level wireless protocols on a raspberry pi, I assume that means the end result is something that runs locally. So your android device and your friend's Iphone are in the same room, and you connect both devices to the bridge wirelessly, then are able to share files between them.
Couldn't you plug the thumbdrive I mentioned into your android phone, copy files onto the drive, hand the drive to your friend, and they plug it in to their Iphone and copy the files off of it?
No, the point is that only one person needs to do something, eg. you connect your Android phone to the bridge and send the files to the bridge via Nearby Share, then the bridge sends them to your friend via AirDrop, the friend doesn't have to connect to the bridge
Reading this response, I really think this is a solution in search of a problem. It's a very cool solution! But I don't think the amount of work involved is justified by the gain (saving party B from having to plug in a flash drive).
Your answer to both this response and the Bluetooth response confuse me. Explain why it wouldn’t work? You drop files on the drive, give friend the drive, copy files from drive. Why would that only work if both mobile devices are yours?
I have something like this.
https://www.amazon.com/Plugable-Transfer-Compatible-Computer-Software/dp/B005OTPVMY
It lets you plug two computers together and transfer files without establishing a network. It behaves like a USB drive when you plug it in so you can install the software that it takes to work.
Honest question, Why can't you just use a flash drive? They even make adapters for them.
Do iOS devices recognize external storage? It has been a long time, but last time I had one, it wasn't an option. And based on the AirDrop references, this seems like an important target for OP.
Yes, they do, as of iOS 13 from 2019.
Thanks! Should've just looked this one up myself, my follow-up would've been what filesystems are supported.
Apple
An exFAT drive could do the trick as a cross platform storage/file transfer device, though not without its own limitations. I wouldn't consider it viable for my use cases personally, though I have little need for an on the go file sharing solution that's compatible with Apple devices.
No NTFS means if you need to share large files between Windows you're SOL.
exFAT supports file sizes up to 128 petabytes, what files are you moving around that are bigger than that?
I think I was thinking of vFAT, my bad.
Honestly I don't know, I am not that familiar with apple either. I assumed it would be basic functionality but I wouldn't be surprised to see apple purposely cripple or remove it.
As I responded somewhere else in this thread:
I think you would just be adding to the problem, instead of solving it.
https://imgs.xkcd.com/comics/standards_2x.png
I believe this is one of those things that it has to be Apple, Google and Microsoft to get together and figure it out. Except I doubt Apple would agree to get together in the first place
I think it is both cheaper and easier just to get both parties to install a cross-platform app than for one party to buy and carry around a dedicated piece of hardware.
Back in the day when internet/LAN transfer speeds were still super slow for most we had what was called a sneaker net, just walk the files on floppy disks/zip disks/cds over to where you need it. I will still do that from time to time with an external SSD.
I have zero experience with stuff like this, but don't power users setup FTP servers for cross-device wireless file sharing?
For a general user I agree it's overly complicated, and that there needs to be a simpler solution.
Even proprietary protocols like AirDrop can be hamstrung.
I don't have any problem with sharing files between my devices, what I care about is sending stuff to other people
Does Wi-Fi Direct meet most of your requirements for a software based solution? It’s based on a standard and is very cross platform, except for Apple devices. Maybe an iOS-compatible widi dongle would make for a good hardware product?
I use localsend on devices that I send files regularly be them mine or my family's etc (on the same network) otherwise I use signal/whatsapp to friends normally.
Don't know if I'd carry additional hardwear for this purpose, cool idea though.
https://github.com/localsend/localsend
Which is fine for "quick sharing", especially given Signal is free, but if I’m moving their primary copy of the family photo to my parents’ devices, I’d like for that to not be a compressed file with visibly decreased image quality…
You can send whole files in WhatsApp, no compression. I havent been able to figure out how to do it in Signal however if you set your media settings to 'High' I find the quality loss acceptable for non print usage the majority of the time.
For more than a couple files, local send or portable SSD.
I agree with the SSD – now that the iPhones have also finally adopted USB-C, it should be better possible between truly all modern devices to use an external disk.
But what do you mean with local send? As far as I could tell, that matter (cross-platform/OS local file sharing) is precisely the matter of the post in the first place…
And to be honest, I don’t blame Signal in case they’re not supporting lossless file sharing. It’s a nonprofit and frankly not what they’re made for, I’m happy with “only” being able to use it as an excellent messenger and being able to send a few pictures here and there, even compressed.
Yeha I'm so glad to see it with USB C, we're getting there!
I was referring to this project local send.
However it's not really applicable to this thread as it's more about sharing on the fly, getting people to download an app defeats the point.
For my own devices at home and work though local send is awesome.
Yeha I love signal, can't complain about it at all.
Ah I see now, yeah I definitely misunderstood that…
Nonetheless thanks for the link and mentioning that project! I will definitely install it on my devices and see if I can get it to work for file sharing from phone to Raspberry Pi (for actual computers, UNIX secure send on the command line is of course an option… but not on the iPhone as fad as I’m aware, whether it be my SE or the most recent one)
I though Airdrop was just a wrapper around Bonjour, and Bonjour was just a wrapper around multicast-dns?