DCD Design Note 141.0

CDF Run-II On-line Network Upgrade

by

David TANG

5/25/99

Enter the document body after this page break. Do not remove the page break. This text will not print.

I. CDF On-line Components:

There are eight major components of CDF On-line computing in the CDF Assembly Building, from a networking perspective:

1. The File Server-Data Loggers.
2. Collision Hall Front End VME CPUs.
3. Collision Hall Diagnostic ports.
4. DAQ Front End VME CPUs.
5. DAQ Level 3 I/O Nodes.
6. Control-Monitor Systems.
7. Development Systems.
8. General Office Systems.

The requirements for CDF Run-II On-line network support are driven by these eight major components.

II. CDF On-line Switch Port Requirements:

There are two switch capacity issues of concern for the CDF On-line network, the number of switch ports needed, and the aggregate maximum network traffic the switch fabric must be able to handle at any particular moment. The number of switch ports is further divided into switch port types, namely gigabit ethernet or fast (10/100TX&FX) ethernet. The port requirements imposed on the network by the major CDF On-line components are listed below:

Component ports ports ports

Up-links 2

Collision Hall Diag. ports - 8 -

Collision Hall Front end VME CPU - 64 -

DAQ Front end VME CPU 60 - -

DAQ Level 3 I/O nodes 16 - -

File Server/Loggers 10 - -

Control-Monitor Systems 30 - -

Development Systems 20 - -

General Office Systems 50 - -

A small number of the components listed may need gigabit ethernet switch ports in the future. Initially the components will generat modest maximum data rates and will likely initially be equipped with only fast ethernet interfaces. However, if these components (the File Server and Data Loggers…) are equipped with gigabit ethernet interfaces at some future date, additional gigabit switch ports can be procured at a later date. The gigabit uplinks into the CDF Off-line LAN, and the general site LAN (for off-site access) may be significantly underutilized, but reflect a general philosophy to over-engineer links into the core network in order to provide very high quality response & performance.

III. Proposed CDF Run-II On-line Network:

The proposed implementation for the CDF On-line network is a single, high capacity, high performance switch/router to support all CDF Run-II On-line computing systems at CDF Assembly Hall. Attaching all CDF On-line computing systems in this location to a common switch provides the simplest overall network architecture, and eliminates the potential bottlenecks inherent with trunk links needed to interconnect different switches. While the different components of CDF On-line computing would share the same switch fabric, they would not necessarily be part of the same logical LAN. The switch would be configured with logically separate partitions to preserve some level of router isolation between the various CDF On-line systems. The routing between most of the different switch partitions would be handled within the switch, not via multiple connections to an external router.

The Cisco Catalyst 6509 is proposed as the switch platform. It has a 32Gb/s capacity switch fabric, an order of magnitude greater than CDF’s estimated aggregate maximum data rate. The 6509 has 8 slots available for switch port modules and routing module. 48-port 10/100B-TX modules, 24-port 10/100B-FX, and 8-port 1000B-SX modules are available today, and a 16-port 1000B-SX module will be available shortly. The 6509 would be located in CDF Assembly Building 3^rd floor Data Communications rack.

The proposed configuration of the 6509 fills the 8 slots of the 6509 chassis. However, if in the future the switch port capacity becomes an issue, we will install a smaller switch such as the Cisco Catalyst 5000( a 5-slot switch) to provide additional 10/100B-TX and 10/100B-FX switch ports, with the capability of gigabit ethernet up-link. This second switch would be used for office and Development systems.

The proposed network configuration for the CDF Run-II On-line network is shown in figure 1.

figure 1.

The Catalyst 6509 would be configured into two partitions, the Front End VME CPUs partition, and the general interactive partition which would include the File Server/Data Loggers and the rest of the CDF Run-II On-line systems. Traffic between the two partitions would be routed internally within the switch/router via layer-3 switching. Traffic external to those two partitions would be routed externally via the "core" router in FCC2E.

IV. Procurement Specifications & Issues:

The Catalyst 6509 would be procured with a configuration equipped with LAN ports to accommodate the CDF Run-II On-line computing tests scheduled for July & August of this year. The rest of the modules (i.e. daughter-board router module for the central supervisor) can be purchased at a later when it become available later this fall.

The proposed procurement configuration is shown below. The cost of the Catalyst 6509 equipped as described below is $ xx.0k, with supervisor (main processor…) module and redundant power supplies.

Module Type Number Ports/module Total # of ports

Supervisor 1000B/SX 1 2 2

10/100B-TX (RJ21) 4 48 192

10/100B-FX 3 24 72

Routing Module 1 0 0

The Catalyst 6509 supervisor module does have two 1000B-SX ports, these ports can be connected to multiple Gigabit up-links or Gigabit switches as needed.

The configuration for the switch for CDF Run-II to be four 48-port 10/100B-TX modules, three 24-port 10/100B-FX module, and one routing module. Total ports available would be 264 fast ethernet ports and 2 gigabit ethernet ports ( these two ports are on the supervisor module).

V. Implementation Issues:

The new Catalyst 6509 would be installed within 30 days of delivery, and made available for the various tests scheduled as part of the preparations for Run-II computing support.