The D0 Online systems are categorized as follows:

1. Host Servers. Two of the three AlphaServer 4000’s are in place on DAB2. Each will have 3 network interfaces, 2 of which will operate at 1Gbps.
2. L3 Filter Nodes. Approximately 48 systems, eventually may expand to 100. Most or all will have two 10/100Mbps interfaces, one to handle event data, the other for monitoring and control.
3. MCH Front Ends. Approximately 24 systems with 10/100Mbps interfaces in the Moving Counting House. Some systems will deal with L2 Triggering and will need to be protected from broadcast traffic.
4. DAB2 Farm Nodes. Approximately 24 linux systems with 10/100Mbps interfaces.
5. Control Room PC’s. Approximately 24 systems with 10/100Mbps interfaces in the DAB1 Control Room.

Because the MCH physically travels from one position to another, power and communication lines must be disconnected and reconnected during the process. Rather than do this for 48 fast ethernet connections, a high performance workgroup switch (Catalyst 2948G) with a gigabit uplink would be installed in the MCH. Only the uplink need be reconnected after MCH repositioning.

The switch port requirements and estimated data rates for the DAB D0 Online components are as follows:

A single-chassis high performance switch would be placed in the DAB2 Computer Room, augmented with a structured wiring system (Design Note 127) capable of delivering 10/100/1000Mbps to each computing node. A satellite switch with a high bandwidth uplink to the main switch would be located in the Moving Counting House. All nodes in the component categories listed above would connect to dedicated ports in either of these switches. With one possible caveat, the proposed switch backplanes will have enough bandwidth to provide a virtually non-blocking switch fabric for all traffic between the connected ports.

The main DAB D0 Online switch will be a Cisco Catalyst 6509 9-slot chassis with dual redundant power supplies. The 6509 has a 32Gbps backplane and 8 slots available for interface modules, the ninth being reserved for the supervisor module. The interface modules available to date include an 8-port Gigabit Ethernet module, a 24-port fiber-based Fast Ethernet (100BaseFX) module (MT style connectors), and two versions of a 48-port 10/100BaseTX module (one with RJ45 connectors, another with telco style RJ21 connectors). A 16-port Gigabit Ethernet module is forthcoming. The requisite supervisor module comes with two 1Gbps port slots. All gigabit port slots on any module are configurable for short-haul (1000BaseSX) fiber or long-haul (1000BaseLX) fiber using replaceable GBIC inserts.

Initially, the 6509 will be configured with an 8-port gigabit ethernet module and two 48-port 10/100 (RJ45) ethernet modules. One 1000BaseLX and four 1000BaseSX GBICs will be installed in the 6509’s gigabit ports.

The 6509 will be logically partitioned into 3 broadcast domains tentatively labeled RIP, Event Data and Interactive. The RIP partition will consist of four gigabit ethernet ports and will provide a fabric between the unix hosts and the RIP switch in FCC. The Event Data partition will contain three gigabit ethernet ports and 48 10/100BaseTX ports for connection between the unix hosts and the L3 Data nodes. The general purpose Interactive partition will include a gigabit port for the MCH switch, and approximately 75 10/100BaseTX ports for connections to the L3 Control nodes, linux farm, control room PC’s and the 3 unix hosts (at 100Mbps).

Three unix-based Host Servers will be connected to 6 dedicated gigabit ports on the 6509 over local multimode fiber circuits. One 1000BaseLX port on the 6509 will be patched to the FCC Run-II switch over singlemode fiber. One 1000BaseSX short-haul gigabit port will connect to the MCH switch over local multimode fiber.

The linux farm systems, L3 Nodes, and Control Room PC’s will be connected to the 6509’s 10/100 ports over local structured wiring. In addition to their gigabit adapters, the 3 unix hosts will each have a 10/100BaseTX interface connected to the 6509 over copper.

The satellite switch in the MCH will be a Cisco Catalyst 2948G, a fixed-configuration chassis with two gigabit ports and 48 10/100BaseTX (RJ45) ports. One gigabit port will contain a 1000BaseSX GBIC for its uplink to the 6509.

The 2948G will be logically partitioned into 2 broadcast domains, one of which will contain the uplink to the 6509. One partition will be used for MCH front end systems, the other for L2 trigger nodes. The latter are to be protected from broadcast traffic, yet require connectivity outside of their partition. A small router will be connected between the partitions to achieve the required isolation.

A primary requirement of this network is to restrict traffic over the RIP link between DAB and FCC to production data streaming and files transfers only, and exclude all other traffic (e.g., control, monitoring, miscellaneous). It may be concluded that an ancillary link is needed between the 6509’s Interactive partition and the FCC Run-II 6509’s Interactive partition, and that a routing function (perhaps "policy routing") will be necessary to discriminate RIP and non-RIP traffic accordingly across these two links. If this routing function is performed externally to the 6509 it must be done by a high-performance router with gigabit interfaces. For the time being our in-house Cisco 8540 may serve the need, but it makes better sense to deploy the routing function within the FCC Run-II 6509. Unfortunately, the Cisco multilayer switching module that would fill this need is not yet available.

The Catalyst 6509 will be ordered with a minimum number of 10/100 modules, enough to accommodate the Run-II computing tests scheduled for July and August. Should more 10/100 ports be required (e.g., additional L3 Filter Nodes arrive), a third 48-port 10/100 module will be ordered.

The proposed initial procurement breakdown is shown below.

The new Catalyst 6509 for DAB D0 Online would be installed within 30 days of delivery and made available for the various tests scheduled as part of the preparations for Run-II computing support.