“The network data includes data from over 550 collection points worldwide, to include collection points in Europe, the Middle East, North/South America, Africa and Asia, and is updated with at least 100 billion new records each day,” a description of the Augury platform in a U.S. government procurement record reviewed by Motherboard reads. It adds that Augury provides access to “petabytes” of current and historical data.
Team Cymru says on its website that its solution provides “access to a super majority of all activity on the internet.”
“Augury is the visibility into 93% of internet traffic,” another website describing the tool reads.
The Augury platform makes a wide array of different types of internet data available to its users, according to online procurement records. These types of data include packet capture data (PCAP) related to email, remote desktop, and file sharing protocols. PCAP generally refers to a full capture of data, and encompasses very detailed information about network activity. PCAP data includes the request sent from one server to another, and the response from that server too.
Screenshots of an apparent Augury panel obtained by Motherboard show results containing cookies, URLs visited, and email data.
I have a lot of questions about this Augury tool. I'd guess the vast, vast majority of the netflow data it's hoovering up is TLS-encrypted. If they're really adding 100 billion records a day to a...
I have a lot of questions about this Augury tool. I'd guess the vast, vast majority of the netflow data it's hoovering up is TLS-encrypted. If they're really adding 100 billion records a day to a store that's already petabytes in size, how much of that is completely incomprehensible noise? It sounds like they're also correlating that with ISP data, if I'm reading it right. Maybe there's a piece of the puzzle I'm missing but users should be able to nullify that bit by using third-party DNS (even better with DoH) and a good VPN. Let me know if I'm overlooking some crucial revelation here.
I guess my point is, the internet has made great strides at hardening itself against surveillance since we first learned about Room 641A, PRISM, XKeyscore, etc. Those were truly alarming at the time. Today, with robust encryption nearly universal, I have a lot more confidence in the ability of users (savvy ones, at least) to stay private online. Again, maybe I misread something and these new tools blow the doors off that assumption. This arms race has been going on for decades and new innovations are liable to emerge from either side.
Fully agreed. A lot of the data collection seems like marketing fluff + the same old tricks that have been in use a long time. The netflow stuff is pretty scary though, at least in theory, as it...
Fully agreed. A lot of the data collection seems like marketing fluff + the same old tricks that have been in use a long time.
The netflow stuff is pretty scary though, at least in theory, as it isn’t easily defeated by VPN or third-party DNS. If I’m understanding correctly, the premise is that if the FBI or other agency finds the physical servers of a DNM or torrent site (e.g. WhatCD) and want to honeypot it without touching the server itself, they can hook into the networking infra in front of the server, and then cross reference the number/size/timing of packets coming to/from the server against the ISP netflow data to identify operators and users of the site. This defeats standard privacy tools pretty easily - only way to definitively counter this sort of traffic analysis attack is to mask your encrypted traffic by mixing in a lot of empty noise.
I have a lot of questions about this Augury tool. I'd guess the vast, vast majority of the netflow data it's hoovering up is TLS-encrypted. If they're really adding 100 billion records a day to a store that's already petabytes in size, how much of that is completely incomprehensible noise? It sounds like they're also correlating that with ISP data, if I'm reading it right. Maybe there's a piece of the puzzle I'm missing but users should be able to nullify that bit by using third-party DNS (even better with DoH) and a good VPN. Let me know if I'm overlooking some crucial revelation here.
I guess my point is, the internet has made great strides at hardening itself against surveillance since we first learned about Room 641A, PRISM, XKeyscore, etc. Those were truly alarming at the time. Today, with robust encryption nearly universal, I have a lot more confidence in the ability of users (savvy ones, at least) to stay private online. Again, maybe I misread something and these new tools blow the doors off that assumption. This arms race has been going on for decades and new innovations are liable to emerge from either side.
Fully agreed. A lot of the data collection seems like marketing fluff + the same old tricks that have been in use a long time.
The netflow stuff is pretty scary though, at least in theory, as it isn’t easily defeated by VPN or third-party DNS. If I’m understanding correctly, the premise is that if the FBI or other agency finds the physical servers of a DNM or torrent site (e.g. WhatCD) and want to honeypot it without touching the server itself, they can hook into the networking infra in front of the server, and then cross reference the number/size/timing of packets coming to/from the server against the ISP netflow data to identify operators and users of the site. This defeats standard privacy tools pretty easily - only way to definitively counter this sort of traffic analysis attack is to mask your encrypted traffic by mixing in a lot of empty noise.