Stateful NAT with Asymmetric Routing

In IOS 12.3, Cisco introduced support for asymmetric routing with SNAT, or Stateful NAT.  The SNAT feature allows multiple routers to share NAT tables, generally with the expectation of an HSRP standby router being able to take over the NAT translations from the primary.  Prior to 12.3, all NAT traffic had to pass through one router - the primary while it's up, or the secondary while it's down.  The idea was to make sure the NAT sessions didn't timeout on the primary router from it only seeing parts of a TCP session. With the outside asymmetric feature, this requirement is removed, and the "backup" router can share state with the primary router, keeping the NAT sessions alive.

While it's technically optional now, SNAT is embedded with HSRP.  I've had a couple production scenarios where asymmetric HSRP is necessary.  Normally the scenario looks something like this:

In this example, EDGE1 and EDGE2 are the HSRP pair.  Both EDGE1 and EDGE2 are peered to different ISPs with eBGP.  They're also peered to each other via iBGP.  Why HSRP here?  This is a perfectly good hot/hot scenario.  In the average route/switch diagram, the expectation would be to have hosts or other routers behind EDGE1 and EDGE2.  In the real world, unless you're an ISP or an enterprise with a large public presence, the average large business puts an HA firewall pair in hot/cold directly behind the Internet edge.  If these were routers instead, you could use iBGP to pass the full Internet tables downstream, and have load sharing in that fashion.  If these were hosts instead, you could use GLBP and get egress load sharing on to the EDGE routers by having each host have an alternating ARP for the default gateway.


As the firewall pair is usually configured hot/cold, and most firewalls don't speak BGP, we only have one device ARPing for a gateway.  In this scenario, EDGE1 would be the active HSRP router, and EDGE2 the standby.  EDGE1 receives 100% of the egress traffic, as the firewall only knows about the one IP from HSRP.  If EDGE2 has a better path to a destination, EDGE1 will know about it from the iBGP peering between them, and pass the traffic (via the switches behind them) to EDGE2, and EDGE2 will egress the traffic.  Another option is to use OER/PFR (http://brbccie.blogspot.com/2012/10/oerpfr-configuration.html) on EDGE1 and EDGE2, and get a better load share than just relying on the BGP tables.  Neither of these are necessarily an ideal setup, but it is effective and easy to implement, and is very common for a medium/large business.  More exotic configurations are certainly possible but are behind the scope of this document.

Inbound load sharing, on the other hand, is very easily accomplished in this scenario.  EDGE2 will continue to receive traffic from the Internet, as it advertises this organization's IP ranges to ISP B.  The BGP tables will ensure some level of inbound load sharing by default.  This means that EDGE2 will be receiving traffic destined for the firewalls, even as it is the "standby" device.

So what's this have to do with SNAT?  Absolutely nothing, yet.  I merely wanted to introduce the idea of why you'd want HSRP + BGP in an easy-to-understand scenario.  Now let's take that same scenario and add NAT to the mix.  Let's say we have a service that is low-security and doesn't play well through the firewalls.  We don't want to assign each device that runs it a public IP, however.  This could lead to something similar to a DMZ, but being routed through the routers directly instead of through the firewall.

This will be our working diagram:

We won't delve into the firewalls or firewall-backed devices at all, they're only there to explain why we'd build a scenario out like this.  Edge 1 and Edge 2 will be NATing into the layer 2 leading to "NATTEDHOST".  Edge 1 will be the primary HSRP router, Edge 2 will be the secondary.

I'm not going to explain all the IP ranges in this section, suffice to say they're all public address space and all advertised into BGP.  1.1.1.1 is the IP on INETHOST that we'll be pinging and telnetting to in order to verify SNAT configuration.



This section was hard to diagram, so I blew the area up some.  The concept is not that difficult, however.  The edge routers have interfaces towards NATTEDHOST and FIREWALL PAIR.  The interfaces facing FIREWALL PAIR are the public IPs we introduce into BGP: 95.1.1.0/24.  The interfaces facing NATTEDHOST are the private IPs, 192.168.0.0/24.  The NAT is being performed on both EDGE1 and EDGE2.  HSRP is responsible for the gateways of both 95.1.1.0/24 (95.1.1.1) and 192.168.0.0/24 (192.168.0.1).  EDGE1 is the active router.

Let's look at the relevant config, but first without SNAT configured.

EDGE1:

interface FastEthernet0/0
 ip address 37.0.0.2 255.255.255.252
 ip nat outside
 ip virtual-reassembly
 speed 100
 full-duplex
!
interface FastEthernet0/1
 description TO_FIREWALL
 ip address 95.1.1.2 255.255.255.0
 ip virtual-reassembly
 speed 100
 full-duplex
 standby 2 ip 95.1.1.1
 standby 2 priority 200
 standby 2 preempt
!
interface FastEthernet1/0
 ip address 192.168.0.2 255.255.255.0
 ip nat inside
 ip virtual-reassembly
 speed 100
 full-duplex
 standby 1 ip 192.168.0.1
 standby 1 priority 200
 standby 1 preempt
!
router bgp 9999
 no synchronization
 bgp log-neighbor-changes
 network 95.1.1.0 mask 255.255.255.0
 neighbor 37.0.0.1 remote-as 2000
 neighbor 95.1.1.3 remote-as 9999
 no auto-summary
!
ip nat pool NAT-POOL 95.1.1.250 95.1.1.250 prefix-length 24
ip nat inside source list PRIVATE-RANGE pool NAT-POOL overload
!
ip access-list standard PRIVATE-RANGE
 permit 192.168.0.0 0.0.0.255

EDGE2:

interface FastEthernet0/0
 ip address 38.0.0.2 255.255.255.252
 ip nat outside
 ip virtual-reassembly
 speed 100
 full-duplex
!
interface FastEthernet0/1
 ip address 95.1.1.3 255.255.255.0
 speed 100
 full-duplex
 standby 2 ip 95.1.1.1
 standby 2 priority 50
 standby 2 preempt
!
interface FastEthernet1/0
 ip address 192.168.0.3 255.255.255.0
 ip nat inside
 ip virtual-reassembly
 speed 100
 full-duplex
 standby 1 ip 192.168.0.1
 standby 1 priority 50
 standby 1 preempt
!
router bgp 9999
 no synchronization
 bgp log-neighbor-changes
 network 95.1.1.0 mask 255.255.255.0
 neighbor 38.0.0.1 remote-as 3000
 neighbor 95.1.1.2 remote-as 9999
 no auto-summary
!
ip nat pool NAT-POOL 95.1.1.250 95.1.1.250 prefix-length 24
ip nat inside source list PRIVATE-RANGE pool NAT-POOL overload
!
ip access-list standard PRIVATE-RANGE
 permit 192.168.0.0 0.0.0.255

Of course the obvious flaw in this design is that if BGP routed traffic can, under perfectly normal circumstances, be asynchronous.  So we could reach out from NATTEDHOST via EDGE1 towards INETHOST, and have INETHOST send traffic back via ISPB and EDGE2:

As EDGE2 doesn't know about the NAT translation that took place at EDGE1, it has no way of returning the traffic back to NATTEDHOST.

To be certain the flow occurs this way, I've put a static route on INETHOST to push traffic headed towards 95.1.1.250 (the PAT in place for NATTEDHOST) towards ISPB:

ip route 95.1.1.250 255.255.255.255 FastEthernet0/1 60.0.0.2

Now let's implement SNAT!

This feature is configured in such a way that it appears very scalable and hierarchical.  However, you're shoehorned into a specific configuration with it, so I suspect Cisco was just thinking ahead towards future features when they created this.  There's really not all that much to add:

EDGE 1:

interface FastEthernet1/0
 standby 1 name UNIQUE-SNAT-NAME

ip nat stateful id 1
 redundancy UNIQUE-SNAT-NAME
  mapping-id 10

ip nat inside source list PRIVATE-RANGE pool NAT-POOL mapping 10 overload

EDGE2:
interface FastEthernet1/0
 standby 1 name UNIQUE-SNAT-NAME

ip nat stateful id 2
 redundancy UNIQUE-SNAT-NAME
  mapping-id 10

ip nat inside source list PRIVATE-RANGE pool NAT-POOL mapping 10 overload

I normally pride myself in that my blogs tend to tear apart the nature and use of the commands, and to some extent I'll do that, but ....  as I explained above, there's hardly anything more to this.  You'll note that EDGE2 gets "ip nat stateful id 2" instead of "1" as EDGE 1 did.  Technically these can be any unique number.  The UNIQUE-SNAT-NAME can be any string of characters.  The mapping-id simply needs to correspond between the "ip nat inside" statement and the "ip nat stateful" section.

There are a few other options under the "ip nat stateful id X" command.  The Backup and Primary commands are used to implement SNAT without HSRP.  While beyond the scope of this document, they're pretty obvious, they're used to specify which routers to trade NAT tables with:

IP Stateful NAT configuration commands:
  backup      Configure backup's ip address
  exit        Exit from IP Stateful NAT primary mode
  no          Negate or set default values of a command
  primary     Configure primary's ip address
  redundancy  Define redundancy group name

You can also attempt to change the communication protocol, but it doesn't help much!

EDGE2(config-ipnat-snat)#redundancy UNIQUE-SNAT-NAME
EDGE2(config-ipnat-snat-red)#?

IP Stateful NAT Redundancy mode configuration commands:
  exit        Exit from IP Stateful NAT Redundancy config mode
  mapping-id  Configure mapping-id for this redundancy group
  no          Negate or set default values of a command
  protocol    Select transport protocol for this redundancy group

EDGE2(config-ipnat-snat-red)#protocol tcp
TCP is deprecated, switching to UDP protocol by default

Now let's look at the outcome of our configuration.

In order to fully prove this, I'd have to turn off CEF all the way around the "Internet" links, and debug ip packet.  This would take pages of blog and I'm not inclined to do it. So instead, I will show you the ping and telnet results, and the output from the SNAT "show" commands.  We would expect this traffic flow:



NATTEDHOST#ping 1.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 68/100/152 ms

NATTEDHOST#telnet 1.1.1.1
Trying 1.1.1.1 ... Open

Password required, but none set
[Connection to 1.1.1.1 closed by foreign host]

EDGE2#sh ip snat peer 192.168.0.2
Show NAT Entries created by peer: 192.168.0.2
Pro Inside global        Inside local                 Outside local      Outside global
icmp 95.1.1.250:0      192.168.0.100:0          1.1.1.1:0             1.1.1.1:0
tcp 95.1.1.250:40935  192.168.0.100:40935 1.1.1.1:23           1.1.1.1:23

We see the success of the telnet and ping, and also we can see via "show ip snat peer 192.168.0.2" that the states are being shared between the routers.  Another useful command to see if SNAT is even working is:

EDGE2#show ip snat distributed

Stateful NAT Connected Peers

SNAT: Mode IP-REDUNDANCY :: STANDBY
    : State READY
    : Local Address 192.168.0.3
    : Local NAT id 2
    : Peer Address 192.168.0.2
    : Peer NAT id 1
    : Mapping List 10

debug ip snat detailed is also useful:

EDGE1#debug ip snat det
IP SNAT detailed debugging is on
EDGE1#
*Mar  1 05:01:46.858: SNAT (sndmsg): ADD new entry from router-id 1
*Mar  1 05:01:46.862: (SNAT): Got Id:1 for NAT Entry (1,4)
*Mar  1 05:01:46.862: SNAT (Sending): Add-Entry(1,4) Fl:2 M-Fl:8020 L:0 A-Type:0 A-Fl:0 id 1
*Mar  1 05:01:46.862: SNAT (Sending): Enqueued ADD Message of Router-Id 1 for Router-Id 2
*Mar  1 05:01:48.690: SNAT (Process): Received SYNC message of Router-Id 2
*Mar  1 05:01:49.758: SNAT (Sending): Enqueued SYNC Message of Router-Id 1 for Router-Id 2

Hope you enjoyed,

Jeff