MBTA: Network Interface (Output)
Andre DeHon
Original Issue: July 1990
Last Updated: Tue Nov 9 13:07:55 EST 1993
Net-out provides an interfaces between the processor on an MBTA
node and an RN1 based network. Net-out deals with all network
traffic originating at the node. This note
describes the interface to and composition of net-out.
For MBTA, all network activity originating from a node is under the control of the node processor. That is, there are no real split transactions in MBTA. The processor must be actively involved in handling network transactions. Hopefully, split transactions can be emulated at the software level.
Since the node processor must handle network transactions, it must explicitly recognize and handle non-local memory references. Most likely, the processor will store its node number in a global register when booted. When performing an emulated memory operation, the processor can check the target address against its processor number to establish if the operation is local or remote and handle it accordingly.
Net-out handles most of the low-level issues of communication. It is intended to handle the portions of the network interface which must be done in hardware and are well understood now. Net-out is especially intended to handle operations which need to be implemented efficiently in hardware in order to obtain a reasonable level of performance. To this end, net-out handles:
Net-out looks like a memory mapped peripheral in the processor's
address space. While net-out only really has a single data port,
net-out is mapped to several addresses. These distinct addresses are
used by the interface so that different function and meaning can be
attributed to memory operations. These addresses are shown in
Table . Only three address bits are shown in
Table
since only a small number of addresses must be
distinguished by net-out. The node will map these addresses as word
address somewhere in the memory mapped i/o address range ( e.g.
NO_CONFIG
0xFE000114).
Closely related to net-out is its associated FIFO. The FIFO is also
mapped into the processor's address space. It has two addresses of
interest as shown in Table .
These addresses make available a number of basic primitives for interfacing
with net-out and monitoring its operation. These basic primitives
and their relation to the visible addresses are shown in
Table .
The general format of an operation is:
DST LEN
OP
FUNCTION
Where each portion is one byte in length. DST specifies the destination node for the specified operation. LEN specifies the length in words of the operation. OP specifies the operation to be performed at the remote node. FUNCTION specifies a function to be performed by the local node. Operations do not always use all of the fields of the operation.
The function is the most basic part of the operation command. It is the
only field which is interpreted for all operations. Functions are shown in
Table .
NETOPSTAT and NETOP state launch one of the network operation described in
Section and at different levels in (tn19) and
(tn20). These two operations are the only two which interpret the
high three bytes of the operation command. The distinction between
NETOPSTAT and NETOP allows network operations to specify whether or not the
processor wishes to be given connection status reports. Operations
initiated with NETOPSTAT will feed one status word back to the processor
for each connection attempt. Operations initiated with NETOP do not feed
connection status back. This gives MBTA nodes two mechanisms for dealing
with connection status words -- using NETOPSTAT to initiate the operation
and reading the NO_STATUS address. This allows maximum freedom for
deciding which is most appropriate in a given situation. We may wish to
eliminate one of these mechanism in the future.
NETOPNOTURN launches a remote operation which does not turn the network and get any reply or status information. ENDNOW instructs net-out to close an open connection that the processor does not wish to use. RESPONDFINAL and RESPOND initiate remote operation responses. RESPONDFINAL closes the connection after sending data whereas RESPOND turns the connection around again for another reply.
ABORT instructs net-out to drop the current operation and return to its idle state as soon as possible.
OP specifies the network operation to be performed. Operation
encodings are shown in Table and described further in
(tn21).
LEN specifies the length in words of data to be retrieved during a network read operation. At present, no other operation uses the LEN field.
DST specifies the destination processor for the network message.
Net-out has a number of configurable options. It is possible to
specify the number of dummy cycles between real network data by setting
dummy-cycles.
The number of network stages can be set by setting stages.
Figure
shows the composition of the configuration word.
Individual portions of the word cannot be set independently. To change
just part of the configuration, read the configuration, reset the desired
bits, and write the configuration back. The unused bits in the
configuration word are available for other configuration options which may
come up during design and prototyping. Space is specifically left next to
the number of dummy cycles so this parameter can be expanded if early
experience with MBTA indicates the number of allocated bits is
insufficient.
The status word indicates the result of the most recent attempt to open a connection through the network. The status word is formatted as:
That is, the status word contains a sequence of two bit status indications
from each network stage, , and a two bit status indication from the
net-in interface at the destination node,
. The
status for net-in immediately follows the status from the final
network stage. Tables
shows the meaning of each pair of
status bits. The status bits for all stages following a blocked stage are
meaningless. The lowest-order bit of the status word, FAIL,
indicates whether or not the connection was successful. If FAIL is
set, the connection failed; if FAIL is clear, the connection was
successfully made. The processor needs only to look at FAIL if it
only wants to know if a particular connection attempt was successful. The
bits in between
and FAIL are currently unused and are
undefined.
Reading NO_STATE will return the state of net-out. The format and meaning of this word will be defined as net-out is implemented. Some subset of these bits should indicate what net-out is expecting from the processor. This may also be useful for keeping the processor in synch with net-out. The state as a whole should be useful in diagnostic testing.
When errors occur such that net-out is forced to signal the processor
that an error has occurred using its line, the
processor should be able to determine the error by reading NO_STATE.
The current list of possible errors is shown in Table
.
N.B. All of the errors shown so far are essentially fatal. When one
of these errors occurs, either the processor and net-out are in
inconsistent states or there is a bug in the source program. The assertion
of indicates that such an error has occurred;
the processor should halt and signal the error to the host so the source of
the error can be located and debugged.
The address NO_ACK is used to check the successful completion of an
operation. As such it is used in two slightly different ways depending on
the network operation performed. After any operation which turns the
network around for an acknowledgement but not for data ( i.e.
noop, reset, or write), it indicates whether or not the ack
indicated the success or failure of the operation. After any operation
which sends a response over the network ( i.e. read and
ROP), it indicates whether or not the reply checksum was correct. Reading
NO_ACK gives the processor an entire word, but only the lowest bit is
important.
If this bit is clear, the operation failed ( i.e. the ack was false
or the checksum was invalid). If this bit is set, the operation succeeded
( i.e. the ack was true or the checksum was valid).
Figure shows a diagram of the basic connections from and to
net-out. Figure
shows net-outs's companion FIFO
buffer; functionally, the FIFO will be a 32-bit wide FIFO buffer with
standards controls like an SN74ACT7201A [Tex89] or a
CYC420
[Sem90]. Table
describes the signals used by
net-out. Note that the Figures
and
are not
specific about signal polarity; Table
, however, does
indicate signal polarity. Net-out is shown configured as part of an
MBTA node in (tn25). Table
summarizes the signal pin
requirements for net-out.
Net-out is primarily intended to support the network transactions described in (tn19) and [DeH90b]. It communicates over the network with the streams described in (tn21). The processor interface described above is intended to allow the processor to provide data transmissions for net-out and control the higher level details of these transactions. This section delves into the specifics of how the processor is expected to use net-out to generate the network transactions.
In this section the following conventions will be used to distinguish required and optional processor operations:
Net-out will generate one status word for each attempt to make a connection through the network. As a result, the number of status words generated can be any number greater than zero. However the processor is checking status, it must use the lowest bit of the status word, FAIL, to identify when to expect real data to arrive.
All the examples in this section concentrate on the i/o operations between the processor and net-out. Intervening computation by the processor is categorically omitted. In emulation mode, the processor will probably want to check the state of net-out by reading NO_STATE to make sure that the operations it performs will complete within a well-defined amount of time.
A noop operation proceeds as follows:
A simple noop network operation without status would be issued as:
If the processor wanted to be given status, it would proceed like:
Alternately, if the processor wanted to poll for status at its leisure, it could get a similar effect by doing the following:
A reset operation should proceed as follows:
A network read operation is performed as follows:
Network write operations proceed as follows:
Alternately, the operation may proceed as follows:
Remote operations ( ROP) have the most generality and are potentially the most complicated. The basic flow is described as follows:
The sequence allows the network to be turned around as many times as the software for a particular remote operation desires. The connection can be dropped from either end of the network as appropriate.
Network operations which never turn the network around are generally uninteresting since they do not get any indication as to whether the connection was successful. For maximum flexibility, remote operations are provided with this option as the simplest ROP.
Many remote operations will probably only want to issue a request and get a response. These proceed much like reads or writes:
Often, it may be necessary to hold more of a dialogue between remote processor. In this example, the network is turned around twice before completing the operation.
The network can be turned around again as shown in the following example:
The network can be turned around as many times as is needed to implement a particular remote operation
Sometimes, it may be desirable to allow an optional response. That is, one node may turn the connection around so the other can reply. The processor replying may have the option to just close the connection without sending any further data. This is a case where the ENDNOW operation may be used.
This will be fleshed out as implementation proceeds.