Private VLANs - How they really work

You're probably already familiar with the basics of a private VLAN: it allows you to group hosts in a single subnet on Ethernet, but limit which hosts can talk to each other at layer 2.  A common design is to have the default gateway accessible to the entire subnet, but prevent the individual hosts from talking to each other (an isolated VLAN). Another common design is to break the VLAN up into various smaller groups that can talk to each other (community VLANs), but allow all hosts in all groups to talk to the default gateway. Instead of a default gateway, the promiscuous host might be a community server, such as a backup server.

Configuring them is not very tricky.  Our topology will consist of two 3560 switches, SW1 and SW2, trunked together on Fa0/13.  R1 will simulate our default gateway, on 192.168.0.1.

SW1:

vlan 124

  private-vlan primary

  private-vlan association 216,402

vlan 216

  private-vlan community

vlan 402

  private-vlan isolated

interface FastEthernet0/1

 switchport private-vlan mapping 124 216,402

 switchport mode private-vlan promiscuous

interface FastEthernet0/13  ! Trunk to SW2

 switchport trunk encapsulation dot1q

 switchport mode trunk

SW2:

vlan 124

  private-vlan primary

  private-vlan association 216,402

vlan 216

  private-vlan community

vlan 402

  private-vlan isolated

interface FastEthernet0/13 ! trunk to SW1

 switchport trunk encapsulation dot1q

 switchport mode trunk

interface FastEthernet0/2  ! Connects to R2, a community host

 switchport private-vlan host-association 124 216

 switchport mode private-vlan host

interface FastEthernet0/4 ! Connects to R4, a community host

 switchport private-vlan host-association 124 216

 switchport mode private-vlan host

interface FastEthernet0/6 ! Connects to R6, an isolated host

 switchport private-vlan host-association 124 402, an isolated host

 switchport mode private-vlan host

I'm not going to go over this config in extreme detail, because this information can be found very easily anywhere.  I just needed a baseline to show a few other interesting things.

R1 is the promiscuous default gateway, R6 is an isolated host, R2 and R4 are in a community VLAN.

First, let's make sure it works.

Can everyone reach R1?

R2#ping 192.168.0.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.0.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

R4#ping 192.168.0.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.0.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

R6#ping 192.168.0.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.0.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

OK, our promiscuous port works.

Can R2 ping R4?

R2#ping 192.168.0.4

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.0.4, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

Looks good, community VLAN works.

Can R6 reach anything besides R1?

R6#ping 192.168.0.2

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

R6#ping 192.168.0.4

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.0.4, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

Nope, it really is isolated.

So it works - fantastic.  But how does it work? Answering that question takes a bit of research.

This page documents the high-level workings:

http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configuration/guide/pvlans.html

Specifically,

"• Primary VLAN— The primary VLAN carries unidirectional traffic downstream from the promiscuous ports to the (isolated and community) host ports and to other promiscuous ports.

• Isolated VLAN — [edited for brevity] An isolated VLAN is a secondary VLAN that carries unidirectional traffic upstream from the hosts toward the promiscuous ports and the gateway.

• Community VLAN—A community VLAN is a secondary VLAN that carries upstream traffic from the community ports to the promiscuous port gateways and to other host ports in the same community. [edited for brevity] "

So, in a nutshell, the primary VLAN carries traffic from the promiscuous port to everyone else, unidirectionally.  An isolated VLAN carries traffic from every isolated host to the promiscuous port. And a community VLAN carries bidirectional traffic for members of the community, but only unidirectional traffic towards the promiscuous port.

OK, we got that, but still, how does it work?

It's all a clever manipulation of MAC address learning.

Referencing the pings I made above, let's look at the mac table on SW1 and SW2.

SW1#show mac address-table | i 124|216|402

 402    0013.c460.2be0    DYNAMIC pv  Fa0/1

 402    0019.2fb8.d552    BLOCKED     Fa0/13

 124    0013.c460.2be0    DYNAMIC     Fa0/1

 124    0019.2fb8.d552    DYNAMIC pv  Fa0/13

 124    0019.e880.09c0    DYNAMIC pv  Fa0/13

 216    0013.c460.2be0    DYNAMIC pv  Fa0/1

 216    0019.e880.09c0    DYNAMIC     Fa0/13

SW2#show mac address-table | i 124|216|402

 402    0013.c460.2be0    DYNAMIC pv  Fa0/13

 402    0014.1ceb.f60f    BLOCKED     Fa0/13

 402    0019.2fb8.d552    BLOCKED     Fa0/6

 124    0013.c460.2be0    DYNAMIC     Fa0/13

 124    0014.1ceb.f60f    DYNAMIC pv  Fa0/13

 124    0019.2fb8.d552    DYNAMIC pv  Fa0/6

 124    0019.e880.09c0    DYNAMIC pv  Fa0/2

 124    0024.c4eb.ed68    DYNAMIC pv  Fa0/4

 216    0013.c460.2be0    DYNAMIC pv  Fa0/13

 216    0019.e880.09c0    DYNAMIC     Fa0/2

 216    0024.c4eb.ed68    DYNAMIC     Fa0/4

There's a lot going on there, so let's break this down into more manageable chunks.

R6 is our simplest host - it's a single isolated host, so let's start there.

R6#show int fa0/0 | i bia

  Hardware is Gt96k FE, address is 0019.2fb8.d552 (bia 0019.2fb8.d552)

So we now know that 0019.2fb8.d552 is R6's MAC address on it's Fa0/0 port.

SW1#show mac address-table | i 0019.2fb8.d552

 402    0019.2fb8.d552    BLOCKED     Fa0/13

 124    0019.2fb8.d552    DYNAMIC pv  Fa0/13

SW2#show mac address-table | i 0019.2fb8.d552

 402    0019.2fb8.d552    BLOCKED     Fa0/6

 124    0019.2fb8.d552    DYNAMIC pv  Fa0/6

We see some new terms here in the CAM table, "BLOCKED" and "DYNAMIC pv".

Best I can tell without any documentation on these, this is what they mean:

BLOCKED = Do not forward traffic to this MAC (essentially the same as not learning it)

DYNAMIC pv = This is a receive-only MAC.

So putting that in context of R6, R6 is allowed to SEND on 402, and can RECEIVE on 124. It cannot receive on 402, because the MAC is "BLOCKED".  This is what enforces an isolated VLAN.  The MAC learning process takes place, but the CAM table flags them as unusable.  Therefore, any traffic sent towards that MAC on that VLAN would be discarded.

Let's look at our earlier definition of isolated VLAN from Cisco:

• Isolated VLAN — [edited for brevity] An isolated VLAN is a secondary VLAN that carries unidirectional traffic upstream from the hosts toward the promiscuous ports and the gateway.

This is true, based upon that R6 (and any other future isolated host) can't learn MACs on that VLAN, except...

R1#show int fa0/0 | i bia

  Hardware is AmdFE, address is 0013.c460.2be0 (bia 0013.c460.2be0)

SW1#show mac address-table | i 0013.c460.2be0

 402    0013.c460.2be0    DYNAMIC pv  Fa0/1

[edited for brevity]

R1's MAC can be learned on 402.  So R6 will be able to send frames out 402 towards R1.  This completes the concept that 402 is a "one way" VLAN from Isolated -> Promiscuous.

But what about R1 -> R6?

• Primary VLAN— The primary VLAN carries unidirectional traffic downstream from the promiscuous ports to the (isolated and community) host ports and to other promiscuous ports.

SW1#show mac address-table | i 124

 124    0013.c460.2be0    DYNAMIC     Fa0/1

 124    0019.2fb8.d552    DYNAMIC pv  Fa0/13

on Fa0/1, we see R1, and on Fa0/13, we see R6.  R6 has the "DYNAMIC pv" status, meaning it is a receive-only MAC.  Frames originating from 0019.2fb8.d552 would not be accepted, only frames towards 0019.2fb8.d552.  We'll be generating more traffic in a moment, and the switches will learn more on vlan 124.

Now let's look at the community hosts.

First, re-generating traffic for MAC learning:

R2#ping 192.168.0.4

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.0.4, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

R2#show int fa0/0 | i bia

  Hardware is AmdFE, address is 0019.e880.09c0 (bia 0019.e880.09c0)

R4#show int fa0/0 | i bia

  Hardware is Gt96k FE, address is 0024.c4eb.ed68 (bia 0024.c4eb.ed68)

There's our MACs for R2 and R4.

How's the table look?

SW1#show mac address-table | i 0019.e880.09c0 ! R2

 124    0019.e880.09c0    DYNAMIC pv  Fa0/13

 216    0019.e880.09c0    DYNAMIC     Fa0/13

SW1#show mac address-table | i 0024.c4eb.ed68 ! R4

 124    0024.c4eb.ed68    DYNAMIC pv  Fa0/13

 216    0024.c4eb.ed68    DYNAMIC     Fa0/13

SW1#show mac address-table | i 0013.c460.2be0 ! R1

 402    0013.c460.2be0    DYNAMIC pv  Fa0/1

 124    0013.c460.2be0    DYNAMIC     Fa0/1

 216    0013.c460.2be0    DYNAMIC pv  Fa0/1

You should notice immediately we don't have any "BLOCKED" status here.  That's because R2 and R4 can talk to each other. This port has no affiliation with the isolated VLAN in any fashion, so R2 and R4's MACs simply aren't learned on VLAN 402. We do have some "DYNAMIC pv".  The primary VLAN is still used to carry traffic from R1 (promiscuous) to R2 and R4.  Therefore, R2 and R4 must be learned on VLAN 124 (as "DYNAMIC pv"), but not able to speak, or they'd be able to reach hosts outside their community.  Also, R1 should be "DYNAMIC pv" on 216 (community VLAN), so that replies are forced over 124 (primary VLAN).

With that wrapped up, here's some other cool stuff I did while working on this!

- Pushing private VLANs through a switch that doesn't support them.  This is a bad, bad idea.  Before I labbed this out this granuarly, I thought, sure, why not?  If the switch doesn't know it's a private VLAN, the whole "DYNAMIC pv" and "BLOCKED" MAC learning concepts stop working, and all hell breaks loose.  Just don't do it!

- Try putting a regular, non-private, access port up on one of the primary or secondary VLANs.  It just doesn't work - you can send frames at the port and the switch just won't learn your MAC and ignores all the traffic.

- Not so exotic on my last bullet, but what about using an SVI as part of a private VLAN?  Well for starters, this is totally doable, but only as a promiscuous port. You'd be using the switch as a default gateway instead of a standalone router.  The config looks like this:

interface Vlan124

 ip address 192.168.0.249 255.255.255.0

 private-vlan mapping 216,402

R6#ping 192.168.0.249

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.0.249, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

Trying to make an SVI with vlan 216 or 402 does not work.

Another oddity:

SW1#show int vlan124 | i bia

  Hardware is EtherSVI, address is 0014.1ceb.f641 (bia 0014.1ceb.f641)

SW1#sh mac address-table | i 0014.1ceb.f641

SW1#

Well that's not something you see every day...

Happy Studying,

Jeff

[mini] OSPF Point-to-Multipoint .... Multicast?

I recently took a practice lab and got dinged for points on an OSPF area question. Without quoting the actual practice lab, the question was referencing a frame-relay link and said something akin to "use an OSPF area type that doesn't elect a DR and multicasts updates".

I've gotten in a (bad?) habit of just using point-to-multipoint with frame-relay. If you've studied OSPF at a design/granular level, you are probably aware that point-to-multipoint is relatively inefficient. However, it works in all sorts of crazy environments, so I just like using it in labs because of versatility.

So even though "point-to-point" would've worked in this environment, I used point-to-multipoint anyway.

And the answer guide insisted point-to-point was the only viable option.

I thought about this, and what the heck is the point of "point-to-multipoint non-broadcast" if "point-to-multipoint" doesn't multicast?  Also, I know point-to-multipoint auto-discovers neighbors, so how the heck is it not multicasting?

My lab is R1 -> Frame Relay Switch -> R2
DLCI is R1 102 -> R2 201

R1:
interface Serial0/0
 ip address 192.168.0.1 255.255.255.0
 encapsulation frame-relay
 ip ospf network point-to-multipoint
 no keepalive
 clock rate 2000000
 frame-relay map ip 192.168.0.2 102 broadcast
 no frame-relay inverse-arp

router ospf 1

 log-adjacency-changes

 network 0.0.0.0 255.255.255.255 area 0

R2:

interface Serial0/0

 ip address 192.168.0.2 255.255.255.0

 encapsulation frame-relay

 ip ospf network point-to-multipoint

 no keepalive

 clock rate 2000000

 frame-relay map ip 192.168.0.1 201 broadcast

 no frame-relay inverse-arp

 shutdown

router ospf 1

 log-adjacency-changes

 network 0.0.0.0 255.255.255.255 area 0

R2(config-if)#no shut

R2(config-if)#

*Mar  1 00:20:11.075: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.0.1 on Serial0/0 from LOADING to FULL, Loading Done

Clearly, we have automatic neighbor discovery.

Aha!  Multicast!  I knew it!

But wait... here comes an update:



So the truth comes out - Hello packets are multicast, hence the automatic neighbor discovery.  But updates are unicast, so the lab was correct in docking me.  Live and learn!

Jeff

[mini] VTY Rotary

I've always found it helps a great deal to have a use-case for a feature. There's thousands of features to learn and be at least somewhat familiar with when attempting the CCIE lab. Remembering them all is a real challenge, but knowing how to apply a feature and why you'd want to use it make it all that much easier to remember. One of those crazy features is "rotary" when used in conjunction with a VTY line.

I totally get what it does:

line vty 0 4
 password cisco
 login
 rotary 1

This config would allow you to telnet to this router on port 23, enter the password "cisco", and get privilege level 1.  With "rotary 1", you could also telnet to 3001 and have the same experience.  Basically, it would mimic port 23 on port 3001.  

R2#telnet 192.168.0.1 3001

Trying 192.168.0.1, 3001 ... Open

User Access Verification

Password:

R1>

If you used "rotary 2", you'd be able to telnet to 3002, etc.

That's the nuts and bolts of what rotary does.  I am immediately reminded of a quote from Despicable Me:

http://www.youtube.com/watch?v=aD4k148YcDU

"...because I was wondering, under what circumstances would we use this?"

I haven't exactly been dying to telnet to my equipment on alternative port numbers.

Now, I finally understand the use case.  It has to do with using different authentication methods on different lines.

For example:
line vty 0 4
 privilege level 1 ! default, but included for clarity
 password cisco
 login
line vty 5
 privilege level 15
 password secretpassword
 login

We see line 5 has a higher privilege level than lines 0-4.  So how do you hit line 5?  Well, I suppose you could telnet at the router 5 times and fill up the first four lines, then hit it again, but that's not very practical. Not to mention you may not know the password for 0-4, if you're an admin-type logging in to line 5.  Enter rotary:

line vty 5

 privilege level 15

 password secretpassword

 login

 rotary 1

Now when we telnet to port 23:

R2#telnet 192.168.0.1

Trying 192.168.0.1 ... Open

User Access Verification

Password: cisco

R1>

Now when we telnet to port 3001:

R2#telnet 192.168.0.1 3001

Trying 192.168.0.1, 3001 ... Open

User Access Verification

Password: secretpassword

R1#

Trying each line's respective password on the other's port number produces the expected failure.

That's a simple use case, let's take a more advanced one.

Let's say you're using lock & key / dynamic ACLs and need *local* auth on one line only. 

R1(config)#aaa new-model

R1(config)#aaa authentication login default group radius

R1(config)#aaa authentication login LOCKANDKEY local

R1(config)#username LOCK password ANDKEY

line vty 0 40

 login authentication default

line vty 41

 login authentication LOCKANDKEY

 rotary 1

 autocommand  access-enable host

The idea here is to use RADIUS for authentication of lines 0-40, and local auth for line 41, to allow your Lock & Key ACL to work.  

I didn't actually setup a lock & key ACL or a RADIUS server, but this can get the point across still:

Regular telnet just fails in our case because of the lack of RADIUS servers:

R2#telnet 192.168.0.1

Trying 192.168.0.1 ... Open

% Authentication failed

% Authentication failed

% Authentication failed

[Connection to 192.168.0.1 closed by foreign host]

However, telnetting to 3001:

R2#telnet 192.168.0.1 3001

Trying 192.168.0.1, 3001 ... Open

User Access Verification

Username: LOCK

Password: ANDKEY

% No input access group defined for FastEthernet0/0.

[Connection to 192.168.0.1 closed by foreign host]

The error message is because of the lack of a lock & key ACL, but the proof of concept is the same.

Cheers,

Jeff Kronlage