This is a horror story. Lucky for you, it comes with a happy ending.
I gave a lightning talk on CGN logging at NANOG 54 in San Diego which started with those very words. The abstract lays out the high points:
Per-connection logging is one of the major hurdles when deploying a CGN system in your network. This talk focuses on just how bad it is and on a possible solution that can drastically limit, or even eliminate, CGN logging while still providing traceability for abuse response. Chris will first present data on CGN logging and log volumes from research and lab testing conducted over the past two years at CableLabs and elsewhere. He will then explain a proposed solution: Deterministic CGN. This solution is documented in draft-donley-behave-deterministic-cgn “Deterministic Address Mapping to Reduce Logging in Carrier Grade NAT Deployments.”
Hopefully the slides help fill in the details (if not, feel free to shoot me a question, or an invite to come speak):
I also presented on CGN technology in much more breadth at the 2011 SCTE Cable-Tec Expo in Atlanta, as part of a session on IPv6 Readiness & Transition. If you were at the Expo, look for the paper (titled “The Experience Gap: Coping with the Looming IPv4 Address Shortage“) in your meeting materials!
Some CGN system can send log data in binary format (netflow, custom flow set’s) that reduce required space.
The other thing is that some operators want to log all connections + cgn logs. I.e. classic netflow + CGN log
@Yuri
rather using the good old netflow, one could use NSEL which combines netflow and CGN Log. this is basically incompressible. tried it twice, got it down by 1.5%, so by nothing. but it only contains original addrs, ports, xlated addrs and ports, create/delete, vrf and origin. altogether it’s just 36 bytes per entry, so 72 bytes per flow.
in some countries LEAs require full flow logs, which includes dst ip address and port too, so using preallocated port ranges won’t save the day on the storage side, but greatly reduces the amount of data you have to hand over to the authorities (compared to the good old next-free-port scheme)
Hello,
I am looking for your “The Experience Gap:Coping with the Looming IPv4 Address Shortage” presentation. Could you inform me where can i find it ? Kind Regards.
Hi Burak,
So, good question, that talk was given at SCTE’s Cable-Tec Expo and they don’t publish the papers nor the presentations publicly. Attendees get a CD with all the materials and I think you can buy them after the fact, but there is no where to just find it. :/ If you were at the Expo or work for a CableLabs member, shoot me an email and we can figure it out.
However, I did give a similar presentation at this year’s North American IPv6 Summit which I posted here: http://chrisgrundemann.com/index.php/2012/cgn-observations-recommendations/ and the PDF is available here: http://www.rmv6tf.org/2012-IPv6-Summit-Presentations/CGN_Observations_Recomendations-NAv6S_2012.pdf.
Hope that helps! =)