10
votes
File sharing over a network
Me and my friend arrive at an arbitrary place, we have access to a network from there. Now, we want to share a file and the network connection is all we have. The challenge: make the file go from my device to my friends device in a pure p2p setting. If you know, for sure, that incoming connections are allowed this is very simple but here i want to explore which solutions exist that do not assume this.
Assumptions:
- Same network altough possibly different access points (one might be wired and the other wireless)
- We have no prior knowledge about the network, incoming traffic might be blocked (outgoing isn't for sure)
- No extra machines can aid in the transaction (no hole punching etc)
- Should work reliably for any kind of device that you have free -- as in freedom -- control over. that is PCs, android phones/tablets and macs. most of Apple's other hardware can be excluded because they don't allow for anything anyway.
- hard mode: We are both digitally illiterate
Goal:
- Send a file, p2p, from one party to another.
Me (MSc cs) and my friend (PhD cs) tried to do this last week. And it appears to be among the hardest problems in CS. I would like to discuss this and hear which solutions you might have for this problem.
Edits:
- this is not an assignment
- Added some specifics to the assumption set
- we're looking for practical solutions here.
- more specs
Syncthing can use UPnP hole-punching, but it doesn't require it by any means.
If both devices are on the same LAN (Ethernet broadcast domain, technically) they'll find each other automatically, and if at least one has the port opened in their local firewall, it'll Just Work.
And its on f-droid! Nice.
And yet this is a problem that Apple has solved: AirDrop.
Only works from Apple, to Apple
I thought that was implied by the word 'Apple.' :P
I don't know about you, but to me that falls under the "doesn't work" category
It's actually quite simple (with Linux)...
Destination box
Source box:
Provided you are on the same network (your gateway comment confuses this, as a gateway is specifically something that joins separate networks) and open the port of the destination firewall, you're golden.
We've played with this too, but here you assume that incoming traffic is allowed. For all you know it might be blocked, like at the university where all incoming packets are dropped.
Why I mentioned about requiring firewall to allow ports. If you don't have control over the hardware you're using to do this transfer, then I believe you are attempting to bypass computer security in a way that would not be legal, or at the very least acceptable.
This scenario should be legal in any case. The sending party consents with sending the data and the receiving party consents with receiving. The only role of the network hardware is to pass the packets containing the data from me, to my friend.
The problem statement is in my eyes extremely simple: send bits from one machine directly to another, the content is irrelevant, both parties consent. I claim that this impossible in almost all cases where you are a digitally illiterate person that resides in a posibbly restricted network. Because there are always third parties that control the network and their sole goal is to make your life miserable.
This is just a manifestation of the restriction of digital freedom, because isp/company IT dep. policies assume that almost all people are stupid: They buy IOT devices that are shit, don't have their pc security in order and download any .exe file there is to get. What is a solution? Put them in restricted networks, behind a NAT, with a draconic firewall. Make everything that the internet was designed to do impossible, without centralised control. My argument is that this is erring on the wrong side. Instead of disallowing anything that could be used for bad, they should allow everything that could be used for good. Mission critical systems should instead be air-gapped, or be part of a private intranet. As a rule of thumb: If you can get out, you should be able to get back in.
It's annoying sure, but I think you have it a little backwards. The vast majority of times the biggest threat to a BYOD network is the devices that people bring on to it. Unpatched, loaded to the gills with malware and ready to cause a shitload of problems. You'd be a fool to admin a large network where you can't control what is connecting to it and what is being sent over it. For every competent user there are hoards of people that possess no understanding on any level about networks or computers. If you never been in a position where it was your pay cheque on the line to keep a large network healthy and functioning I suggest you not attribute to malice what can easily be explained by defence against your peers incompetence.
EDIT: Just to make something abundantly clear: I am assuming this is something like a university network, where people are going to flip their shit if they don't have near 27/4 connectivity to university systems etc to submit work. There is no practical way to isolate this, and fundamentally allowing students to share files p2p isn't a service they want to support. If you had a good reason for doing it that was related to the purpose of the network then you could just ask. You'd probably get what you want.
The game changes entirely when it's an ISP. I pay top dollar to have access to a very technically competent ISP whose values align with mine (I want a /48 and zero filtering please and thank you).
I agree with large sections of the last two comments.
On GyroTech's, assuming the packets are being dropped at the network level rather than at your hardware, they don't want you sending the data. If you want to use a hack to get around that, I won't help with it.
On yours, the internet should be completely free. In practice, it's free to an extent. However, you're on a private network. The owners of that network shouldn't be forced to allow any of your traffic through. It costs them resources to move your data, and if your data is a movie (as you hinted at in another comment) that could potentially open them up to legal troubles which are best avoided by making their network a means of using the internet rather than a means of connecting to other users on it.
As you can use the internet, there are still means of using this network but they would require bouncing the file outwards if peer-to-peer connections are disabled. If you're not sending all that much data, consider encrypting it and putting it on pastebin in base64 or a cloud storage service. (Google Drive, Dropbox, etc)
Edit: Never mind that last paragraph, you don't want to put the file anywhere on the internet. If you can't do peer-to-peer connections on this network, I don't see a solution...
That's fine, i don't need help poking holes because i'm perfectly capable of doing so via my vpn, vps, ssh tunnels or wireguard tunnels. The point is to have p2p capability without these. Amish p2p, just plain.
As mentioned, if the network administrator has locked-down access to explicitly allowed datastreams than you are out of luck.
From a netadmins perspective, this is a good thing, as your datastreams could well easily be the next big worm infesting all devices on the network as likely as some 'innocent' p2p data.
Does the owner/admin of the network consent with transmitting this data between these parties?
It shouldn't matter. If network access is provided to both of us, the contents of our packets are private and of no concern to the network administrator.
With respect, this is completely and utterly wrong. If you, as a private home-owner, lend me, a friend, a set of keys to your place, does not allow me to now do whatever I want in your private residence. I cannot set up a shop, I cannot invite people over and throw a party, I cannot set up a home theatre and start showing films.
The network, and usage policy, are the private systems of the owner. If said owner doesn't want any traffic other than that which is strictly permitted, they are completely within their rights to drop it.
I agree, there's a weird emphasis here on using a network that isn't under your own control.
If you have a situation where public/corporate networks are unavailable (not your company) or blocked (school), and you have two devices (belonging to you) in the same room, the solution is to use either bluetooth or ad-hoc WiFi (or a cable) to create your own network, and use that.
I'm actually surprised that this is the only response mentioning bluetooth. Two devices with bluetooth capabilities should be able to connect without an available (or cooperating) network, and you should be able to transfer a file between them as a result. I hadn't considered an ad-hoc wifi network, but setting that up with a network-accessible directory should work, too.
Now I'm curious if you could connect to an ad-hoc wifi network and SSH into the same machine that the ad-hoc network originates from. It sounds like it should be possible, but I really want to verify that at some point. It could really end up being handy if so :)
I think you'd likely have to fiddle around with the IP address assignments and which address sshd listens on, depending on what the defaults are for your distro, but it certainly should be possible.
Bluetooth is probably the simplest option, but the bandwidth is lower, while ad-hoc/WiFi Direct is (in my experience) a little more fiddly to set up, but will run as fast as the hardware allows.
They are in their right to deny traffic, but not based on the content of the packet. Having said that, your key-lending analogy really sucks for this particular situation.
That's not at all true. Most public/corporate/educational networks have acceptable use policies that you agree to when you connect to them, or as a condition of your employment/study. Those network operators are full well within their rights to restrict you from transferring adult content, hate speech, illegal content, or literally anything they want to.
There may be technical controls that prevent them from being able to determine the type of content being transferred, but if they don't like the sources and destinations, or the amount of traffic they're sending, they are fully 100% within their rights to kick you off their network.
On virtually all of them, attempting to circumvent those controls is grounds for removal from the network as well, or worse.
That's not a rule. Net Neutrality fights are related to ISPs but not to private networks.
My network, my rules... or my publicly traded company might have rules based around Sarbanes Oxley or my medical school has them based around HIPAA requirements... or I need to make sure my limited network doesn't get flooded because Billy wants to watch Netflix at lunch or Susie wants to stream her Spotify and now the network is flooded and barcode reporting in Oracle is now delayed for shipment orders going out and the truck has to wait an extra 20 minutes... or my Guest network in my home can't torrent so that my IP doesn't get tracked from my wife's friend's daughter not turning qbittorrent off... or that legal hold for the Sampson Audit needs to stay in place untouched and isn't allowed to be transferred off the server to a USB drive...
More and more I question what you're asking for here - particularly what your expectations are of what you want to do and what is ethical to do in this situation.
You've already said it, no more need be done. It's their network, they can make completely arbitrary rules of traffic denial if they want.
Edit to add: As an addendum, Tildes prides itself on being a place of civil discourse. If you feel my analogy it not apt, you would do better to actually refute it rather than simply saying it 'sucks'.
If you're on the same network, how is the traffic being blocked? If you're on different networks with a firewall or routers in between you, the answer to this question will be entirely dependent on how that gear is configured.
Is there NAT involved? Which ports are being blocked inbound? Is there SPI? In the most locked down networks, this will be literally impossible. There is no way for your outbound connection to make its way to your friend's IP if he's behind NAT without help from the network administrators.
If you are truly on the same network (subnet), GyroTech's suggestion will work swimmingly. Otherwise, there are just way too many variables to consider, and no one will be able to make a useful suggestion without more information.
We are on the same subnet, but it's possible that i have a wifi connection into the network and my friend is connected with an ethernet cable. So there is at least one extra device. NAT is a whole different beast, which can be defeated by ssh tunnels if you don't have network control.
If you're on the same subnet, the suggestion above will work. Firewalls are placed at network boundaries, and unless your network does extremely restrictive 802.1x enforced dACLs, there won't be any restrictions within the same subnet. Even if there are dACLs, you can just use an allowed port.
You won't be able to get around NAT via SSH though. You can make outbound connections, but without having 1 to 1 NAT rules in place, or some kind of port forwarding, you won't be able to hit your friend's internal IP.
What your looking for is nitroshare. It's simple, fast, cross platform, and there's no fuss.
Do you really need to use that network? Why not just create a WiFi hotspot without internet access and share it over LAN? It meets all the other requirements (including the hard mode if you download an app) and solves your problem of sharing data locally in any situation even better, as it doesn’t rely on any existing connection.
Of course other methods are much easier than this. But the point is to use the network
Are basic SMB sharing connections not available? It sounds like you are admins of your own machines if you're looking at software solutions and using VPN and SSH already. If you're on PC and\or Mac, simply \IPADDRESS\Share Folder\ should work (with a couple settings changes built into the OS). Unless the network you are on is specifically setup so you can't see other machines or share directly - and that means it's a private network and trying to get around what someone else has put in place for their own security and stability.
This thread really makes me think you're at a school on Eduroam (which definitely segregates devices) and trying to get around it for Plex sharing or something similar. Just a guess that could be completely off; just the impression I get.
We are on eduroam but we do not want to setup plex. Because everyone keeps guessing about specifics: The instance is sharing a huge chunk of researchdata over eduroam without some intermediary server.
Gotcha.
Eduroam is not designed to handle or support what you're trying to do... It's primary function is easy internet access for students, not local network functions. Think "free public WiFi". Anything you try to do to work around the restrictions put in place would ultimately be a violation of the security and terms of use. You're best option will likely be using one of those Dropbox sync style services.
Or using the local network that's in place for faculty and staff for local network use. I'm guessing you're trying to use Eduroam as a geographicly spanning local network though otherwise you'd have already switched to this option if your sharing was on the same campus.
I don't really know what you're getting at. The sounds like a school assignment. I mean why can't you just put it in a torrent file, upload it, and have them download it? I just think this is coming off real school assignment like because why wouldn't you just email it or use a cloud to share it.
Torrent hast undesired side effects, while email and cloud are usually volume limited. Also, there's gotta be a way to not involve the internet here.
On Linux? Open a ssh server on one end, connect. (You can transmit your local ip to one another, right?) Then use SFTP to grab/drop whatever files you need. I don't remember the exact commands but I remember moving files through a ssh Tunnel.
On Windows this would probably be more work but could work the same way maybe.
ssh tunneling works in general. Although, we've tried this at the university and they drop all incoming packets.
Actually, that's my whole point. There are somewhat solutions out there so I'm not really understanding the question.
Windows has SFTP servers available as well. Openssh recently fixed the issue where you couldn't lock people down but even now you have to use the config file and write it line by line. Chroot jails were broken up until recently. A real popular Windows alternative for an SFTP server is Bitvise. But torrenting would be faster.
Because it seems to be a serious challenge. I don't want to share a movie by mailing it, neither do i want to put it on a server of someone i don't know. We've tried torrents of course. But the problem is that there needs to be a way for the DHT to be updated, or you need to run your own tracker.
This should be something that is easy, just like chat. but it turns out that it's not because almost everyone resides in a severely restricted network. Tbh i don't care what kind of data we send, it's meant to be an instantiation of setting up a pure p2p connection without using someone elses computer.
If you're in the same room, the solution is to stop trying to use a network you can't control (the owner of the infrastructure will always win), and create your own network via ad-hoc WiFi or similar.
I've used my phone in hotspot mode as a means of local filesharing either directly from the phone or bouncing laptops through the phone. I don't recommend trying to fight infrastructure you don't own, especially not if it's a school or business whose good will is important to you.
Well, you have sFTP. OpenSSH has a Windows alternative, but for Windows I prefer bitvise. You can set up a VPN at home and just connect to it and have access to your files too. But if uploading and downloading are a huge part of it, torrenting will be the faster method.
On Linux, you could upload it and then wget it. That or nc it.
It is easy if you're truly on the same network. It sounds like you and your friend actually reside in different, connected networks (internetworks). In this scenario, the same exact technical barriers that prevent easily P2P file sharing over the internet (mainly firewalls and NAT) apply to you and your friend, and the same ways around them need to be applied. (Using only outbound connections to a 3rd party, or if no NAT, HTTP tunneling or some other way of disguising your traffic as allowed traffic). That would, of course, be circumventing the controls on your network, and may not end well for you.
To make it even harder you're not specifying the kind of device. It sounds like you're talking about PCs but what about mobile-to-PC, PC-to-mobile or mobile-to-mobile?
I'll specify it. Should work reliabliy regardless of the device.
https://www.justbeamit.com/
simplest p2p service i've seen and totally free
Have you tried file.pizza?