CGNAT Logs and the Shape of Pain
There is a particular kind of pain that only shows up once an ISP crosses a certain invisible line. It does not come from outages, fiber cuts, or angry customers. It comes from math.
That pain is CGNAT logging.
This post exists because I was talking about processing CGNAT logs with my son and realized how often the industry talks about the tooling without ever talking about the physics. People ask why disk fills up. They ask why OpenSearch cannot just handle it. They ask why it feels so expensive for what sounds like “just logs.”
Those questions all miss the same point.
CGNAT logging is not a software problem. It is a scale problem.
Let us walk through it.
What CGNAT Logging Really Is
CGNAT logging is often described as “IP logging.” That description is not quite correct.
CGNAT logging is session accounting taken to its logical extreme. RADIUS has done this for decades at the subscriber level. CGNAT does it at the flow level, millions of times per hour.
Every time a subscriber device opens a flow, a record must exist somewhere that answers a very specific future question:
“At this exact moment in time, who was using this public IP and port?”
To answer that, a single CGNAT record usually includes:
• subscriber identifier
• internal source IP
• internal source port
• public source IP
• public source port
• destination IP
• destination port
• protocol
• timestamps
That is one record per flow.
Not per customer.
Not per device.
Not per minute.
Per flow.
Modern devices create flows constantly, even when they appear idle. Phones chatter. TVs poll. Browsers preload. Cloud services whisper to each other all day long.
CGNAT does not care why the flow exists. It only cares that it existed.
The Math Nobody Wants to Run
Let us stay conservative.
Assume a modest rural provider:
• 20,000 subscribers
• 500 flows per subscriber per hour
That is:
10 million flow records per hour
240 million flow records per day
Even if you compress aggressively and store only 200 bytes per record after serialization, you are looking at roughly:
48 GB per day
1.4 TB per month
17 TB per year
That number does not include:
• replicas
• indexing overhead
• retention buffers
• growth
If you run replicas, and you should if you care about losing subpoena data, you can double or triple that number very quickly.
This is where the pain begins to feel personal.
Why “Just Use OpenSearch” Sounds Right and Fails Anyway
OpenSearch is excellent at what it does.
It is designed for:
• fast search
• aggregation
• dashboards
• time window queries
It is not designed to be a long term compliance warehouse.
Every document stored in OpenSearch pays a tax:
• raw document storage
• inverted index structures
• doc values
• segment merges
• JVM memory pressure
That tax is worth it when you need fast answers.
It is brutal when you just need to retain data.
CGNAT logs are overwhelmingly retention data. Most of them are never queried. They exist so that the few that matter can be found later.
Using OpenSearch as the primary long term store means you are paying a high performance indexing tax on data that almost never earns it.
OpenSearch is not broken when it eats disk. It is doing exactly what it was designed to do.
Why Disk Is Always the First Thing Management Refuses to Buy
Routers are tangible. Fiber is visible. Towers look impressive.
Logs are invisible until they are suddenly very visible.
CGNAT storage does not improve customer experience. It does not increase speed. It does not reduce outages. It only prevents legal and regulatory pain later.
That makes it a hard sell until the first subpoena arrives.
At that moment, storage stops being optional and becomes urgent.
Every ISP learns this lesson eventually.
Some learn it calmly. Some learn it while panicking.
The Correct Mental Model
The mistake is thinking in terms of tools instead of tiers.
CGNAT logging needs layers.
Hot data
Short retention
Indexed
Searchable
This is where OpenSearch belongs.
Warm data
Compressed
Partitioned by time
Minimal indexing
Cheap disks
Cold data
Object storage
Rarely accessed
Compliance only
OpenSearch should never be the warm or cold layer. If it is, you are paying premium prices for data that does not need premium handling.
Why This Hurts Rural Providers More
The logging requirement does not scale with revenue.
A rural ISP must log the same level of detail as a national carrier, but spreads that cost across far fewer customers.
The math is identical. The margins are not.
CGNAT logging punishes small and medium providers simply for existing in the modern internet.
The Uncomfortable Truth
If you run CGNAT, you are in the data retention business whether you want to be or not.
There are only three real levers:
• reduce retention
• reduce detail
• buy storage
No search engine, database, or dashboard changes that reality.
The Takeaway
When someone asks:
“Why cannot OpenSearch just handle CGNAT logs?”
The honest answer is:
“Because CGNAT logs are not a search problem. They are a physics and economics problem.”
In the next post, we will explore a different question.
If this is unavoidable, is there a smarter way to do it?
That is where things get interesting.
Part 2 -> CGNAT Logs: An Idea