Confirmed: Govt Intercepts can't identify source from large Telcos mobile internet

From long-running posts on problems with Carrrier-Grade NAT (CGNAT) by Geoff Huston of APNIC, I'd previously posited that the current call for "metadata retention" is driven by Intelligence Agencies needing to identify sources. This SMH article points to a recent 9-page PDF Consultation Draft, extracts below. Stilgerihan in ZDnet points to both the SMH piece and a 16-pg 2010 Draft, heavily redacted when first released under FOI.
Links: Brandis' follies, and metadata catches downloaders not serious crims.

CGNAT is only used by the largest Telcos. Twenty years ago, Optus used NAT for its Cable (HFC) internet and Huston writes about mobile Internet now using private 10-Network addresses internally.

The impact of new requirements on the central routers / NAT devices of large Telcos is important. These handle ingress/egress of all packets to their clients and will throttle all traffic if overloaded. Purchasing new routing & switching equipment, along with the necessary network redesign and testing (plus inevitable major outages) won't be covered by the Government and is a serious impost.

If a Telco has 1M devices in service, with 2,000 simultaneous sessions through a small number (16-254) of outbound IP numbers, they will cycle quickly through the 20-24 bits of available IP+port combinations. They will now have a requirement to log all those translations.

Huston estimated 30,000 separate translations per device, per day, i.e, 1250/device/hour, or 1.25 billion log entries per hour (347,000/sec), sustained and 3-10 times that in peak hours. A carrier-grade device's switching fabric will switch NAT at those rates, but I guess that logging has to be carried out by the "control plane", the CPU on the router backplane. My guess is 32bytes per compressed log entry (~100Mbps), Huston suggested 512bytes and 1.5Gbps. This results in very large archival datasets to be stored, but much worse, to be accessed as an indexed database. That's a very, very expensive system just to retrieve the TIA data when requested, for nothing.

I know the size of servers needed to handle 500k-5M transactions per second (~$100,000+) and suspect that even CISCO would struggle in putting that much compute power in a router. This sort of activity must eventually end up handled by the switching fabric, leaving the control plane available for what its designed for: routing and control. The global market for these super-switches is small (1-5,000 units), making their forced development very expensive for customers, even by Carrier standards.

It's the triple whammy of all super-fast devices: bleeding edge technology (the most expensive kind to design, build and test), limited production runs to amortise development costs over and very high Gross Margins to allow the vendor to stay in business and design the next one.

Not only is Brandis very publicly announcing they have, or have had, problems identifying the source of Internet access, he's now attempting to shift the costs onto very large onto Telcos. This is data kept & accessed strictly for Government purposes, it should be entirely funded by the Government and this considerable burden not be imposed on Carriers.

The cost and complexity is high enough to prevent ISP/RSP's and mobile operators increasing their customer base to where CGNAT is required. I think that comes under the Constitution's Restriction of Trade provisions.

The serious organised crime and terrorists Brandis et al say they are targeting won't be affected at all by these measures:

1. The small number of master hackers selling their services & knowledge already know this blind-spot,  and are selling now exactly how to utilise it.
2. The next step for serious criminals and terrorists is to use VPN's to be completely opaque to intelligence gathering. Again, nothing the black hats don't know and aren't already selling.

Extracts:

1(b) This requirement intends to capture both present and past identifiers allocated to an account or service by the service provider (such as an IMSI, IP or email address, or other network identifier). [An account number]

Note: Category 2(a) does not apply to or require the retention of destination web address identifiers, such as destination IP addresses or URLs. This exception is intended to ensure that providers of retail and wholesale internet access services are not required to engage in session logging. However, operators of such services remain obliged to retain network address allocation records (including Network Address Translation records) under category 1(b).

4(a) and 4(b). These requirements intend to accurately capture the link between a communication or connection and the time at which it occurred. [Means all ISP/RSP's must have atomic clocks, or tier 0 clocks fed by GPS units.]

6(a) This requirement intends to capture the identifier(s) of the equipment from which a communication is sent or is attempted to be sent. Examples of such identifiers include the unique IMSI of the party originating the communication, the unique IMEI of the mobile device used to originate the communication, or the MAC address of the network interface used to originate the communication. [i.e. for WiFi services]

7 (a). This requirement intends to capture the physical and logical location of the device or equipment used to send or receive a communication.
Note: Location information contained in the content of communications, such as assisted GPS information passing over a service or network, is not telecommunications data and is not included in this data set.