Buffer-to-buffer Credits in Fibre Channel

In a Fibre Channel (FC) or Fibre Channel over Ethernet (FCoE) implementation, dropping frames isn’t allowed. Hence the name “lossless fabric.” This means in an FC fabric, a device must never send frames unless the intended receiver is able to receive them.

To this end, buffer-to-buffer credit system is employed as a flow control mechanism in FC. The buffer-to-buffer credits are negotiated between two adjacent ports on two devices.

SwitchA——————SwitchB

In this hardware-based flow control mechanism, the number of frames a port on SwitchA can send to an adjacent port on SwitchB, without having to wait for an ACK, is equal to the number of credits the port on SwitchB has managed to accumulate thus far. (Sounds kind of like a loaded sentence. Let’s elaborate below.)

To keep count of the number of buffer credits available, we’ll have to briefly take a look at Primitive Signals.

Fibre Channel has 5 layers: FC0, FC-1, FC-2, FC-3 and FC-4.

The Signaling Protocol Layer, FC-2, is focused on breaking up the data to be transmitted in to frame size elements, encapsulating the data in to frames, controlling the order in which the frames will be transmitted and reassembling the frames after transport. In short, FC-2 layer defines important functions such as flow control, encapsulation of data in to frames and exchange of sequences.

To accomplish the responsibilities specified above, FC-2 layer uses:

  • Frame Delimiters: Specify the beginning (Start-of-Frame SOF) & end (End-of-Frame EOF) of a frame.
  • Primitive Signals: Signal placed on wire before the Start-of-Frame (SOF) delimiter.
  • Primitive Sequences: To notify about failure of the link.

There are two types of primitive signals here i.e IDLE and R_RDY (Receiver Ready.) For flow control, the R_RDY primitive is the relevant primitive we are interested in.

R_RDY (Receiver Ready): This is a primitive signal sent by the receiver to the sender. By sending an R_RDY, the receiver is basically signaling to the sender: “I have a buffer space I can hold a frame in if you wish to send me one.”

In simple terms, the Tx BB_credit value on this side of the link tells a story of how many frames the device on the other side can receive.

SwitchA(Tx BB_credit=8)——————–SwitchB(Tx BB_credit=17)

SwitchA can send 8 frames to SwitchB without having to wait for an ACK signal.
SwitchB can send 17 frames to SwitchA without having to wait for an ACK signal

For each frame an interface transmits, the buffer-to-buffer credit is decremented by one. For every R_RDY primitive signal received on the interface the buffer-to-buffer credit is incremented by one; which implies the interface gained the ability to transmit one more frame.

If the buffer-to-buffer credit value reaches zero, the port cannot transmit any frame until it receives an R_RDY primitive signal.

SwitchA(config)# interface fc1
SwitchA(config-if)# switchport fcrxbbcredit X
To display the BB credit information, use the show interface fcX/Y bbcredit command.
SwitchA# show interface fc1/1 bbcredit
fc1/1 is up
Transmit B2B Credit is 8
Receive B2B Credit is 17
…
17 receive B2B credit remaining
8 transmit B2B credit remaining
SwitchB# show interface fc1/1 bbcredit
fc1/1 is up
Transmit B2B Credit is 17
Receive B2B Credit is 8
…
8 receive B2B credit remaining
17 transmit B2B credit remaining

This method of flow control is the buffer-to-buffer flow control and applies on Class 3 frames. A different mechanism in fibre channel is end-to-end flow control. End-to-end flow control follows distinct concepts and implementation (applies on Class 1 frames) to ensure a lossless fabric.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s