Transit Note #27
MBTA: Network Initialization

Andre DeHon

Original Issue: July 1990

Last Updated: Tue Nov 9 13:25:09 EST 1993

Introduction

This note suggests a ``simple'' scheme for getting all of the nodes in an MBTA machine to agree upon the number of dummy network cycles per real network cycle and agree upon when the real network cycles occur.

Problem

The basic MBTA boot sequence uses the network to load and boot each node (tn20) [DeH90a]. This means that network communication to a node is necessary before the node processor can boot to configure a node. In general, we would like to be able to configure the number of dummy cycles dynamically at boot time. Difficulty arises since network communication is necessary to dynamically communicate the number of dummy cycles to the node and knowledge of the number of dummy cycles is necessary to properly interpret network data. This problem is further complicate by the need for nodes to agree which cycle is the real network cycle.

Two-Phase Initialization

A two-phase initialization scheme can be used to bootstrap a running MBTA machine.

When the machine powers up, a hardware initialization is asserted, . forces the network to reset to its default idle configuration and forces the network interfaces to their startup configuration. The network interfaces, net-out (tn23) and net-in (tn24), each start up assuming some default value for the number of dummy cycles. This value will be a small number ( e.g. zero, if the node can handle traffic at full network speed).

After a hardware initialization the machine is ready to be booted. The network can be used to communicate startup data to all network nodes. Among the data that is communicate to each node is the desired value for the number of dummy network cycles. This value should get written into a specific location in each node's memory ( DUMMY_CYCLES). As part of the initialization sequence of each node, the processor will write this value to each of its network interfaces to configure them for proper emulation.

Once all nodes have successfully booted and the MBTA machine is ready to begin multiprocessor emulation, the soft initialization, , is asserted. The assertion of will cause each network interface to update its concept of the number of dummy network cycles to the value written during configuration. The network cycle immediately following the deassertion of will be treated as the first real network cycle; this guarantees all nodes agree on when the real network cycle occurs.

Caveats

Skew in is almost as critical as clock skew. If the network interface components on different nodes see deasserted during different clock cycles, the nodes will not agree about where the real network data is in transactions and thus be unable to communicate correctly. should be distributed from a common source in the same manner as the network clock.

If the default number of dummy cycles is anything other than 0, the same care must be taken in routing . Here, also, nodes should agree on when the real network cycle occurs. If the default number of dummy cycles is 0, then the distribution of is not critical since there are no dummy cycles; real network cycles are trivial to identify since all network cycles are real.

Transit Note #27
MBTA: Network Initialization

Introduction

Problem

Two-Phase Initialization

Caveats

See Also...

References

Transit Note #27 MBTA: Network Initialization

Introduction

Problem

Two-Phase Initialization

Caveats

See Also...

References

Transit Note #27
MBTA: Network Initialization