Packet Ordering - IDT 89HPES64H16G2 User Manual

IDT Switch Core
Notes
PES64H16G2 User Manual
in the appropriate IFB queue. After being queued in an IFB
data transferred through the crossbar interconnect is transferred in a continuous TLP manner (i.e., the data
path is never multiplexed).
This choice of datapath width implies that the crossbar has 20% higher throughput than the throughput
required to service all ports. This "over-speed" ensures that inter-port messages (i.e., internal messages
exchanged by ports for switch management) do not affect the throughput of the PCIe links.
On the egress interface, data in the EFB is read by the port's data link layer (i.e., DL) when it is chosen
to be transmitted on the link. If the port is in merged mode, the DL allocates all clock cycles to read data
from the EFB. However, depending on the negotiated link width not all clock cycles may be used to transfer
data. If the port is in bifurcated mode, the DL reads data from the appropriate EFB (i.e., each port has a
dedicated EFB). Again, depending on the negotiated link width, not all clock cycles may be used to transfer
data.

Packet Ordering

The PCI Express specification 2.0 contains packet ordering rules to ensure the producer/consumer
model is honored across a PCIe hierarchy and to prevent deadlocks. The Switch Core performs packet
ordering on a per-port basis, at the output of the ingress and egress buffers of each port (refer to Figure
3.1).
Applying ordering rules at the output of the ingress buffer (i.e., before the crossbar) is done to ensure
that packets are ordered regardless of their destination port. This guarantees that the producer/consumer
model is met when the data transfer involves any number of peers.
Applying ordering rules at the output of the egress buffer is done to allow packets in the EFB to be re-
ordered for deadlock prevention and efficient transmission on the link without violating the PCIe ordering
rules. Without this ordering logic, packets in the EFB would need to be transmitted in the order they were
received by the EFB. If the oldest packet in the EFB lacked sufficient link credits for its departure, head-of-
line blocking would occur at the EFB. The presence of ordering logic at the EFB reliefs potential head-of-
line blocking by allowing other packets to be transmitted, as long as ordering rules are not violated.
Table 3.4 shows the ordering rules honored by the Switch Core. Note that the PES64H16G2 honors the
relaxed-ordering attribute in packets as shown in the table.
1.
Please refer to section Cut-Through Routing on page 3-6 for further information on conditions for cut-through
transfers to occur.
1
3 - 4
and undergoing ordering and arbitration, all
April 5, 2013