RSVP: QoS plus CAC

RSVP is another one of those topics that's easy on the outside, but hard to get to the bottom of.  We all know it uses the IntServ model, and that it's an end-to-end "QoS mechanism".  But beyond knowing how to turn it on, do you really know its inner workings in IOS?  To be specific, I will be covering how IOS handles RSVP, not so much the math and RFC-intentions.

RSVP is an end-to-end signaling QoS signaling protocol.  The idea is that the application requiring QoS will speak RSVP natively, signaling towards its target that it is about to send data requiring a certain QoS level.  The majority of the routers in between the source and the destination should be speaking RSVP, and reading this message (PATH) as it goes by.  When the destination receives this PATH message, it sends back a RESV message towards the source.  The RESV message makes the actual "QoS" reservation as it makes its way back up the path.  RSVP is inherently one-way; if the receiver in this example also needs to send with a reservation, it needs to send its own, separate PATH back and wait for a RESV.  In other words, in this original example, you'll get a reservation only on the egress interfaces the directions the PATH went out, or the ingress interfaces the directions the RESV came in.

Before I diagram this, let's look at an important topic.  This reservation doesn't mean a whole lot unless you pair it with some QoS mechanism.  RSVP, on IOS, by itself, does nothing other than something akin to CAC (call access control).  If you allow 600K to be reserved on every interface, and you're using g711ulaw (~88k per call after overhead), you could expect 6 calls to be permitted.  RSVP, if setup end-to-end, does a very good job of making sure interfaces aren't oversubscribed.  The idea is if that 7th call were to be placed, all calls might suffer.

That's another point - "if" it's setup end-to-end.  RSVP is a routed signaling protocol, so it can skip hops and still work.  You hope those hops either have plenty of bandwidth or react to DSCP -- you can use DiffServ in conjuction with RSVP.  More on this later.

This diagram shows the sender sending out the PATH message and RESV being reflected back by the receiver.  As I don't have any applications hanging around that speak RSVP (does anyone?), my endpoints will be routers.

I will cover five major topics, and some minor ones.  The five major ones are:
- RSVP for CAC
- RSVP for CAC + bandwidth queuing
- RSVP for CAC + bandwidth + PQ
- RSVP for CAC + DiffServ
- RSVP for CAC + WRED

RSVP FOR CAC
This is the default function of RSVP if you're not running WFQ on the interface.

Moving forward, assume all interfaces are appropriately IPed and EIGRP is configured on all interfaces.  All interfaces are reachable from all others. "SENDER" has an Lo0 address of 1.1.1.1, "RECEIVER" has a Lo0 address of 4.4.4.4; 2 has 2.2.2.2, 3 has 3.3.3.3.

The RSVP configuration is simple.

SENDER:
interface Serial0/0
 bandwidth 384
 no fair-queue
 ip rsvp bandwidth 200 100

R2:
interface Serial0/0
 bandwidth 384
 no fair-queue
 ip rsvp bandwidth 200 100

interface FastEthernet0/0
 ip rsvp bandwidth 200 100

R3:
interface Serial0/0
 bandwidth 384
 no fair-queue
 ip rsvp bandwidth 200 100

interface FastEthernet0/0
 ip rsvp bandwidth 200 100

RECEIVER:
interface Serial0/0
 bandwidth 384
 no fair-queue
 ip rsvp bandwidth 200 100

Let's take a look at a few elements before we create the RSVP reservation.
We're allowing 200K to be reserved of a 384k interface.  We either need to keep this 200k max reservation under the 75% limit imposed by IOS, or up the max limit. 

You'll notice I disabled WFQ (no fair-queue) on the serial interfaces.  WFQ is the default on slow serial interfaces.  Enabling WFQ would allow for a bandwidth reservation, so I've taken it out of the mix for the moment

One confusing element is that RSVP needs to be installed on both ingress and egress interfaces on R2 and R3.  Why, if it's only making a reservation from SENDER to RECEIVER?  Well, it turns out RSVP won't send a message on an interface that RSVP isn't enabled on.  So, in order for the RESV message to get from RECEIVER to SENDER, RSVP has to be enabled on the return interface.

Now we're going to create a PATH message from SENDER to RECEIVER, and a RESV from RECEIVER to SENDER.

SENDER:
ip rsvp sender-host 4.4.4.4 1.1.1.1 UDP 16392 16387 88 2

We'll send the PATH message towards 4.4.4.4 (RECEIVER) from 1.1.1.1 (SENDER), for udp 16387 -> 16392, requesting 88K (g711ulaw) with a burst value of 2k.

RECEIVER:
ip rsvp reservation-host 4.4.4.4 1.1.1.1 UDP 16392 16387 FF LOAD 88 2

We'll reply to the PATH mesage from 4.4.4.4 towards 1.1.1.1 for udp 16387 -> 16392, requesting a fixed reservation (FF - more on this later) and bandwidth (LOAD) of 88k with burst of 2k.

Now of course this method is really only any use for a lab, in production the application would need to make the reservation in a scenario where we're only using it for CAC.  I've read people suggest this can be used as a proxy reservation for a downstream host that needs a reservation but doesn't speak RSVP (the ip rsvp reservation command, as opposed to the ip rsvp reservation-host command, can make a proxy reservation for a downstream host).  Not sure how that would work - the resources are forever pinned up, not just when the flow is active...

Let's see it in action...

SENDER#sh ip rsvp install
RSVP: Serial0/0
BPS    To              From            Protoc DPort  Sport
88K    4.4.4.4         1.1.1.1         UDP    16392  16387

R2#sh ip rsvp install
RSVP: FastEthernet0/0
BPS    To              From            Protoc DPort  Sport
88K    4.4.4.4         1.1.1.1         UDP    16392  16387
RSVP: Serial0/0 has no installed reservations

R3#sh ip rsvp install
RSVP: FastEthernet0/0 has no installed reservations
RSVP: Serial0/0
BPS    To              From            Protoc DPort  Sport
88K    4.4.4.4         1.1.1.1         UDP    16392  16387

We see that every egress interface between SENDER and RECEIVER has made the reservation.  What should we see on RECEIVER?  Can you guess?

RECEIVER#sh ip rsvp install
RSVP: Serial0/0 has no installed reservations

Nothing; there's no egress interface on RECEIVER (aside from Lo0, but that's a virtual interface).

Let's look at one of the reservations a little deeper.

R2#sh ip rsvp install detail Fa0/0
RSVP: FastEthernet0/0 has the following installed reservations
RSVP Reservation. Destination is 4.4.4.4. Source is 1.1.1.1,
  Protocol is UDP, Destination port is 16392, Source port is 16387
  Traffic Control ID handle: 07000404
  Created: 00:51:18 UTC Fri Mar 1 2002
  Admitted flowspec:
    Reserved bandwidth: 88K bits/sec, Maximum burst: 2K bytes, Peak rate: 88K bits/sec   
    Min Policed Unit: 0 bytes, Max Pkt Size: 0 bytes
  Resource provider for this flow: None  Conversation supports 1 reservations [0x8000403]
  Data given reserved service: 0 packets (0 bytes)
  Data given best-effort service: 0 packets (0 bytes)
  Reserved traffic classified for 2061 seconds
  Long-term average bitrate (bits/sec): 0 reserved, 0 best-effort
  Policy: INSTALL. Policy source(s): Default

We've admitted the reservation, but not provided it any resources.  We have effective CAC.  If we were two request two more flows, the 2nd one would be denied by RSVP.

Another configuration option would've been to leave WFQ enabled but tell RSVP not to use it.

Let's modify R3:

interface Serial0/0
 fair-queue
 ip rsvp resource-provider none

R3#sh ip rsvp install detail | b Admitted
  Admitted flowspec:
    Reserved bandwidth: 88K bits/sec, Maximum burst: 2K bytes, Peak rate: 88K bits/sec
    Min Policed Unit: 0 bytes, Max Pkt Size: 0 bytes
  Resource provider for this flow: None
< output omitted>

Still just CAC - no queueing.

One last topic before we move on was the "FF" or "Fixed Filter" I referenced above:
ip rsvp reservation-host 4.4.4.4 1.1.1.1 UDP 16392 16387 FF LOAD 88 2

There are three different types of filters:
FF = Fixed Filter.  This means this reservation may only be used by the sender IP and Ports specified above.
SE = Shared Explicit.  A specific set of senders may make use of the reservation. Of note, I have not found a way to simulate this using just IOS.
WF = Wildcard Filter.  Any sender can make use of the reservation.  This method uses a sender of 0.0.0.0 and port of 0:

ip rsvp reservation-host 4.4.4.4 0.0.0.0 UDP 16392 0 WF LOAD 88 2

RSVP for CAC + bandwidth queuing

The next logical step is to have a queueing mechanism give us a bandwidth guarantee.  In IOS, this is accomplished with WFQ.  WFQ gives a low weight to RSVP flows, assuring they are likely to be in the queue first.  This is akin to a bandwidth reservation with CBWFQ.

SENDER:
Interface Serial0/0
 fair-queue

R2:
Interface FastEthernet0/0
 fair-queue

R3:
Interface Serial0/0
 fair-queue

For our scenario, WFQ is not required on the ingress interfaces, as the queuing only need be controlled on the egress interfaces.

Another interesting note, even though you only type in "fair-queue", if you do a show run you'll see something akin to:

fair-queue 64 128 7

The "7" is the number of RSVP queues allowed.  The maximum is 1000.  You can specify it manually, but I'm not sure exactly where IOS comes up with this number otherwise, as some of my labs have had 1000 queues, some have had a couple hundred, but it picks this number somehow (I'm guessing based on bandwidth) when you enable fair-queue.

In order for this to take effect, we need to "bounce" the reservation:

SENDER(config)#no ip rsvp sender-host 4.4.4.4 1.1.1.1 UDP 16392 16387 88 2

This will send a TEAR message to the participating RSVP routeres which will remove all the reservations in the path.

Then recreate it:

SENDER(config)#ip rsvp sender-host 4.4.4.4 1.1.1.1 UDP 16392 16387 88 2
SENDER#sh ip rsvp install detail

RSVP: Serial0/0 has the following installed reservations
<output omitted>
  Admitted flowspec:
    Reserved bandwidth: 88K bits/sec, Maximum burst: 2K bytes, Peak rate: 88K bits/sec
    Min Policed Unit: 0 bytes, Max Pkt Size: 0 bytes
  Resource provider for this flow:
    WFQ on hw idb Se0/0:  RESERVED queue 137.  Weight: 6, BW 88 kbps
<output omitted>

We now see we have a resource provider of WFQ... WFQ will now ensure that a minimum 88k of bandwidth is available for this flow.

This is the most flexible way to set this up.  If WFQ isn't enabled on an interface, you'll get CAC, but no queuing mechanism to enforce it.  The other devices with WFQ will implement the queueing mechanism.  If you want to force WFQ to be used (or otherwise reject the request), you would enter:

ip rsvp resource-provider wfq interface

RSVP for CAC + bandwidth + PQ
This part will be the weak spot in this article, because try as I may, I simply cannot get a priority queue to appear.  The idea is very simple: If traffic is small and of the right type, a PQ function will magically appear inside WFQ and all its traffic (up to the reservation size) will be serviced first.
The commands you enter to make the basics of this happen are:
R2(config)#ip rsvp pq-profile  
R2(config)#ip rsvp pq-profile voice-like

Both of these are the default commands, so entering these does nothing on a greenfield lab.

"voice-like" looks for a g711ulaw stream or smaller, and admits it to the PQ.
You can also specify a maximum traffic flow (anything smaller than the specification makes it into the PQ).  The command syntax is:

ip rsvp pq-profile <max flow in bytes> <max burst in bytes> <peak-to-average ratio> | ignore-peak-value

A flow will be permitted into the PQ if it is smaller that <max flow in bytes> and <max burst in bytes>, and it's 'burstiness' is under the peak-to-average ratio.  Alternately, you can ignore peak-to-average with "ignore-peak-value" command.

The catch is, I absolutely cannot make this work.  I've tried three different IOS versions (primarily I have been using 3725 12.4(15)T14).  Perhaps all the ones I've tried are bugged - it's not like this is a commonly used feature.  If someone knows what I'm doing wrong, please chime in.  I've moved my reservation down to g729-sizes, used the commands above, and here's what my output always looks like:

R2(config)#do sh ip rsvp install det | s Resource
  Resource provider for this flow:
    WFQ on hw idb Fa0/0:  RESERVED queue 265.  Weight: 16, BW 32 kbps

From what I've seen elsewhere on the Internet, "RESERVED" should be "PRIORITY".

RSVP for CAC + DiffServ
What if you've got a router in the path that only supports DiffServ?  Fear not, we can mix IntServ and DiffServ.

Let's modify R2 to be DiffServ only.

R2(config)#int s0/0
R2(config-if)#no ip rsvp bandwidth
R2(config-if)#int fa0/0
R2(config-if)#no ip rsvp bandwidth
I've cleared the reservation and recreated it.  Just for proof of concept that RSVP can, in fact, skip a hop:

SENDER#show ip rsvp install
RSVP: Serial0/0
BPS    To              From            Protoc DPort  Sport  Weight Conversation
88K    4.4.4.4         1.1.1.1         UDP    16392  16387  6      137

R2#show ip rsvp install

R2#

R3#show ip rsvp install
RSVP: Serial0/0
BPS    To              From            Protoc DPort  Sport  Weight Conversation
88K    4.4.4.4         1.1.1.1         UDP    16392  16387  6      137
RSVP: FastEthernet0/0 has no installed reservations
SENDER and R3 both have the reservation, but have skipped R2.

I spent some time labbing this for proof-of-concept.  My lab configuration consisted of 4.4.4.4 connecting to 1.1.1.1 for a chargen TCP stream.  One of the tricks of labbing this on a router is that the port making the request for chargen is random > 1023, so you actually need to establish the flow in advance of creating the RSVP reservation, in order to determine what the port numbers are.

Let's get the chargen session up first, for the reasons described above.

SENDER:
service tcp-small-servers

RECEIVER:
telnet 4.4.4.4 19 /source-interface Lo0

_`abcdefghijklmnopqrstuvwxyz{|}~ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFG
`abcdefghijklmnopqrstuvwxyz{|}~ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGH
(etc)

RECEIVER#show tcp brief
TCB       Local Address               Foreign Address             (state)
6657E8A0  4.4.4.4.34407               1.1.1.1.19                  ESTAB

OK, so the flow we want to match is 1.1.1.1:19 -> 4.4.4.4:34407

RECEIVER(config)#ip rsvp reservation-host 4.4.4.4 1.1.1.1 TCP 34407 19 FF RATE 60 2
SENDER(config)#ip rsvp sender-host 4.4.4.4 1.1.1.1 TCP 34407 19 60 2

SENDER#show ip rsvp install detail | i Data given
  Data given reserved service: 68807 packets (7928432 bytes)
  Data given best-effort service: 13667 packets (1558038 bytes)
There it is - chargen doesn't generate that much traffic, but I had this running for a while already. 

We still have RSVP off on R2.  Now let's make this thing work!  On Serial0/0 on SENDER:

SENDER(config-if)# ip rsvp precedence conform 5 exceed 2

This will mark IPP on the packets processed by RSVP.

R2:
class-map match-all ipp_5
 match  precedence 5

policy-map no_rsvp
 class ipp_5
  priority 60

interface FastEthernet0/0
 service-policy output no_rsvp

Pretty straightforward...

R2#show policy-map int
 FastEthernet0/0

  Service-policy output: no_rsvp
    Class-map: ipp_5 (match-all)
      17432 packets, 2286130 bytes
      30 second offered rate 14000 bps, drop rate 0 bps
      Match:  precedence 5      Queueing
        Strict Priority
        Output Queue: Conversation 264
        Bandwidth 60 (kbps) Burst 1500 (Bytes)
        (pkts matched/bytes matched) 11238/1443274
        (total drops/bytes drops) 792/102116

    Class-map: class-default (match-any)
      3432 packets, 204552 bytes
      30 second offered rate 1000 bps, drop rate 0 bps
      Match: any

That's about all there is to it.  IntServ + DiffServ combined!

RSVP for CAC + WRED
You've seen me use "LOAD" and "RATE" in the reservation commands above.

ip rsvp reservation-host 4.4.4.4 1.1.1.1 UDP 16392 16387 FF LOAD 88 2
ip rsvp reservation-host 4.4.4.4 1.1.1.1 TCP 34407 19 FF RATE 60 2

LOAD is supposed to be a bandwidth guarantee
RATE is supposed to be a latency (priority) guarantee

Best I can tell IOS doesn't care what you put in that field, it will accept the reservation if RSVP is enabled, regardless of if it can guarantee latency.  From what I've read elsewhere on the Internet, the expectation is if you're accepting RATE, you should be using WRED to guarantee it. 

Here's how you use WRED with RSVP.

First, you remove WFQ.  WRED and WFQ don't play together.
See my post on WRED for more info on the inner workings of WRED.
http://brbccie.blogspot.com/2012/12/the-nitty-gritty-of-wred.html

SENDER(config)#int s0/0
SENDER(config-if)#no fair-queue

Now enable WRED

SENDER(config-if)#random-detect
SENDER(config-if)#random-detect precedence rsvp 76 77 1
\
The above command makes a lot of assumptions (again, see my post above for more details on these topics); it assumes you want to use precedence-based WRED, it assumes that your output hold queue is default for this serial interface (75), it assumes the rest of your flows are appropriately tuned for drop depth & probability.

That all said, this command will assure that RSVP traffic is never randomly dropped; and will drop all other flows first.  The "never randomly drop" status comes because the hold-queue is only 75 deep; telling RSVP to start dropping at 76 effectively disables it for that traffic class.

The show command confirms:

SENDER#sh ip rsvp install detail | i Resource provider
  Resource provider for this flow: RED

Misc Stuff
RSVP has two incredibly useful debug commands:

debug ip rsvp
This debug allows you to see the signaling taking place, specifically, PATH, RESV, and TEAR.  If your reservation isn't making it across the network, this is the command you want.  Be sure to turn it up on every router in the path, often the hop it's dying at isn't the one you'd guess.

debug ip rsvp traffic-control
This debug shows the resource provider assignment.  Sadly, it's not helped me any in finding out why I can't create a PQ, but it is otherwise helpful.

What if you have RSVP enabled, but don't want a specific "neighbor" sending you RSVP requests?
The first thing to understand is that there is no predefined "neighbor" relationship, it's all adhoc as reservations are created.  However, if you don't want to process RSVP packets from certain IPs, you use:

ip rsvp neighbor <standard ACL>

The usage on this is kind of goofy in my opinion.  The filtering is on who generated the request, so the concept of it being a "neighbor filter" is really only applicable if you're talking about edge filtering.

Take our scenario from the top of the document.  If you were on R3 and wanted to stop the reservation from 1.1.1.1, logic tells me in a "neighbor statement", you'd want to block 192.168.1.2, the link to R2.  This is incorrect.  You'd want to filter 1.1.1.1, the source of the request.  So it's more of a "source filter" than a "neighbor filter".

DSBM for RSVP
RSVP doesn't play well on multiaccess networks.  If you have a series of routers connected via a switch, that switch may be oversubscribed on egress on to one of the routers.  Take this scenario for example:


Consider that both SENDER1 and SENDER2 are each sending a 75Mb RSVP stream towards RECEIVER1.  All the links connected to SW1 are 100Mbit.  R1 will approve the reservation, as will R2, and the egress port from SW1 -> R3 will become congested.  RSVP will fail from a CAC and.queueing standpoint.

Enter DSBM - Designated Subnet Bandwidth Manager.   The concept is simple, one router on the multiaccess segment gets elected to track all the reservations and make sure this doesn't happen.

Configuration is simple:
R1:
interface FastEthernet0/0
 ip rsvp bandwidth 75000
 ip rsvp dsbm candidate 120 
 ip rsvp dsbm non-resv-send-limit rate 40

This config would only allow an aggregate 75k on to the multiaccess segment, would establish R1 as a DSBM candidate (120 is an optional priority - higher is better), and only allow 40KBytes of traffic on to the segment that was NOT RSVP traffic.

Turning off the RSVP queueing scheduler
I haven't got a complete picture of what this feature does, but I felt it needed to be included (seems a likely lab topic).  If you want to use RSVP for CAC only, there's no reason to have the process running that attempts to inspect & queue traffic.  You can disable it with:

ip rsvp data-packet classification none

Burst Policing
With RSVP, burst is handled on a per-interface basis.  If you want to configure how much an interface (not a flow) can burst over its requested rate, you configure:

ip rsvp burst policing [percentage]

The default is 200, range can be 100 to 700.

Cheers,

Jeff