First, what's metadata?
It's "Data about Data", an inexact, mutable and undefinable concept, best described as "it varies on what we have, who wants to know and what the law says it is".
In the world of Librarians, there's a whole industry around it, starting with the Dublin Core Metadata Set. This extends into
In the world of your PC, every file has extensive metadata stored on it:
who created it and when, who can read and write it,
In the world of (public) Telecommunications Carriers, its the mostly implicit data stored in logs for traffic engineering, performance and billing purposes. Implicit data is things like the time a packet arrived or a connection started and finished.
Are the Username, IP headers and URL's "metadata" or not?
They appear in the system logs, along with implicit data, but don't appear in the user content transferred. You have to work on those logs, sorting all the entries, to collect all entries for a User into a coherent set, to recreate their browsing history. Does the inferred and the browsing history recreated from the system logs constitute their browsing history? That's a tortuous legal argument, and one that has to be pinned down exactly in the legislation.
A friend pointed out to me yesterday that serious Criminals and real terrorists won't be caught by any of the data collection proposed under the legislation.
To make the point:
High strength methods for anonymous leakers don't just exist but are in common use, Strongbox (once "Dead Drop") is used by the likes of Conde Casts' New Yorker and I'm told the Guardian. The most prominent recent leaker, Edward Snowden, entrusted his communications to a version of this. With the combined resources of the NSA and GCHQ, if there was a way to break Snowden's anonymity and back-track what he did, they'd have done it.
The "Strongbox" software is free and readily available to anyone who wants to conduct military-grade untraceable communications. It's designed to be simple enough to setup and use that non-technical journalists can do so.
Spelling that out: serious criminals and real terrorists, who are known for their sophisticated use of leading-edge Internet technologies, know about this system, have the knowledge, ability and motivation to use it, and we can safely assume, because we can never know otherwise, are using it, or a modified version.
For the less paranoid criminal and terrorist, or simply for lower-level, everyday communications, there are multiple, simple systems for them to efficiently and untraceably exchange files and messages:
- PGP, or the GNU version, GPG, is a strong public-key encryption system that's very widely used and freely available. There are no public admissions that it's been compromised.
- 'Tor", The Onion Ring network, is used by Strongbox. It provides a shared, but slower, method to anonymise traffic to and from websites. Just using it renders raw logs useless, though techniques are published to identify sources and presumably tested by the NSA.
- Multiple versions of "dead drops" are available. From using the Draft folder of an email account with a shared password (messages are never sent, but deleted once read by recipients) to more sophisticated built-for-purpose services.
- In the last year, the "Blackphone" and other high-privacy services for email and other uses have become available commercially. Before that, remember the 2013 UK riots were enabled specifically by the secure messaging service of Blackberry. The UK Police, and you'd think GCHQ, were unable to read the messages or obtain warrants to read the plain text from the supplier systems.
- But the really simple, obvious and hard to defeat solution, for those determined to evade detection, is high-strength public-key encryption provided by VPN's, Virtual Private Networks. These can be purpose-built software or repurposed public software usually based on SSL (https, port 443) or SSH (stunned, sftp, etc, port 22).
- If internet users route all their traffic via a VPN, all the logs show is from XXX to YYY, very long duration, many packets sent.
- If the monitoring authorities are able to co-operate and legally share data across jurisdictions, and log the "plain text" end of the VPN services, they cannot directly match any of the inbound traffic with any outbound streams, because of the large number of simultaneous, legitimate streams. Real-time packet matching with very high-resolution timing could be used with "bandwidth modulation" techniques to correlate, for a single user, their VPN output stream with their PC. This can be easily defeated with something like Tor over VPN's.
If you're making real money from your unlawful activities or are undertaking military-grade operations, be they terrorist or APT-style strategic intelligence gathering, you will be completely missed by the proposed "metadata" collections, provided by warrant or not.
So who will be caught up in the "Anti-terrorist" net of these meta-data?
Only people who aren't trying to hide.
The most obvious group is those engaged "Copyright Theft" by downloading bit-torrents of their favourite shows, like "Game of Thrones".
People who download content for their personal use, cannot be said to be "pirates" or actually breaking Copyright, because they are NOT producing unauthorised copies. Australia understood decades ago that with new technologies, people will circumvent onerous copy restrictions, so for blank cassette and VHS tapes, a levy was imposed.
The High Court decided that strictly personal use of copied material was "fair use" under the amended Copyright Act, not "piracy". People who make and sell copies of content, or play them in public, for paying audiences or not, do break the Australian Copyright Act. These people are pirates, they are deliberately profiting off other people's labour and creative endeavours.
These "anti-terrorist" data collection laws, that focus on long-term collection and retention of IP numbers and associated 'ports', are only of use in identifying and prosecuting those engaged in bit-torrent downloads. This is why they don't want URL's or email metadata, because they don't apply to bit-torrent.
The people who refuse to pay the outrageous Foxtel tax for "Game of Thrones", because Netflix sold Foxtel the Australian monopoly rights and locals here are refused connection by Netflix.
The "store two years of data" is designed to be used retro-actively.
Once the first prosecutions are made against "Pirates", ordinary householders downloading strictly for personal use, people will stop naively using bit-torrent. Only it will be too late, all their prior activity will have been logged and the AFP will, under the "Criminal Act" provisions, pursue them vigorously.
Expect the first prosecutions around a year after the legislation is enacted and given assent.