MBTA: Network Interface (Input)
Andre DeHon
Original Issue: July 1990
Last Updated: Tue Nov 9 13:14:58 EST 1993
Net-in provides an interfaces between an RN1 based network and
the processor and memory on an MBTA node. Net-in deals with all
network traffic destined for the node.
This note describes the interface to and the composition and behavior of
net-in.
Net-in is designed to handle the most primitive operations directly without need for intervention from the processor. For more complicated operation, it simply serves as an interface between the processor and network, acting under the direct control of the processor.
Net-in handles virtually all of the low-level issues of communication. It is intended to handle the portions of the network interface which must be done in hardware and are well understood now. Net-in is especially intended to handle operations which need to be implemented efficiently in hardware in order to obtain a reasonable level of performance. To this end, net-out handles:
Many things are happening on each node at various speeds. To fully follow the subsequent sections, it may be useful to understand a few cycle time definitions.
Network Cycle One cycle of the network clock. During this time a single byte of data may be transmitted to or from the network through each network port. When RN1 is running at its full target speed of 100MHz, this cycle time will be 10ns.
Real Network Cycle Cycle during which real data is being sent to or received from the network. See (tn25) for a description of real and dummy network cycles. In general each real cycle is followed by zero or more dummy network cycles.
Dummy Network Cycle Cycle during which dummy data is being sent to or received from the network. See (tn25) for a description of real and dummy network cycles.
Memory Cycle One read or write access to memory. The current plan is to make memory cycle time fast enough to service two network ports. Since memory is 32-bits wide and the network is 8-bits wide, this cycle time must be at most twice the network cycle time. At the full network speed mentioned above, memory cycle time will need to be 20ns.
Processor Cycle One cycle of the processor clock. This will be 60ns, 40ns, 30ns, or 25ns depending on the speed grade of 80960CA/CB available and used. A 40ns processor cycle would be easiest to work with since it is an even multiple of the network cycle time (10ns) and the memory cycle time (20ns). We may want to be able to make use of faster processor cycle times when they become available.
Node or Emulation Cycle One cycle of emulation of all the hardware on the node. This is the time it takes for the processor to pretend to be each piece of hardware which one is pretending is on the node. During one emulation cycle there are four real network cycles. The number of dummy cycles is chose so that there are exactly four real network cycles for each emulation cycle.
While there will be a single design for net-in, it may be used in a few slightly different ways.
Master Net-in is the authoritative net-in on the node.
Slave Net-in is the net-in which takes cues from the master net-in
To solve this potential problem, the net-ins are each given exclusive
access to the memory bus for one memory cycle during each node cycle
provided it wants the memory cycle. This allows fair emulation during node
emulation since one memory cycle is given to each network port per
emulation cycle. It also allows full-speed operation during high speed
network testing. Table shows the relation of net-in
memory cycles to network cycles for high speed network testing. Note, that
this does starve the memory system bandwidth when both net-ins
constantly require access to memory. Table
shows this relation
during emulation.
Net-in will be counting dummy cycles already in order to identify real network cycles. Thus, it makes most sense to let net-in signal the end of an emulation cycle and when net-ins get the memory bus. Before the end of the processor cycle prior to the cycle when the net-ins may get the bus, the master net-in asserts EC, to indicate the end of an emulation cycle.
Each net-in which wants to use the memory bus should have its want output, WO, asserted prior to the assertion of EC. While EC is asserted and WO is asserted by at least one of the net-ins, the bus controller allows the net-ins access to the memory bus. The master net-in holds EC high until both it and the slave net-in relinquish their WOs. The master net-in uses the memory bus in the memory cycle immediately following its assertion of EC if it needs to. If it needs to access memory, the slave net-in does so when EC is asserted by the the master but WO is not. Each net-in monitors the other's WO via its own WI. The slave monitors EC from the master via its EC. (See (tn25) for the details of interconnecting net-ins into a node.)
Figures ,
, and
shows the rough
timing of the EC and WO signals. WO is asserted at some
point during the emulation cycle before the last network cycle. EC
becomes asserted at the beginning of the last network cycle of an emulation
cycle. It remains asserted while the net-ins need to access memory,
dropping just prior to the end of the last memory cycle. Each WO
drops just prior to the end of the memory cycle its net-in uses.
Net-in is intended to autonomously handle all of the low-level network transactions described in (tn19) and [DeH90b] except for the ROP, remote operation, transaction which implicitly requires the processor's control.
These transactions require no node resources.
When a NOOP network transaction is received, net-in drops the
connection after returning its status and checksum bytes. See
Section for information on the status and checksum bytes.
Upon receiving a RESET network transaction, net-in initiates a
reset sequence for the node processor and holds the network connection
open. Net-in asserts for the appropriate number of
cycles (see (tn25)) then releases
to effect a
processor reset. When net-in first powers up,
should
be asserted. After performing the reset sequence, net-in monitors
the
signal. If the processor succeeds in booting,
net-in returns an ack_t; otherwise it returns an ack_f
(see (tn21)).
Net-in can directly handle the raw memory transactions described in
(tn21). This along with the RESET transaction allow the node to
be booted over the network without EPROMs (tn19). These memory
transactions require access to the node memory. This access is obtained as
described in Section . The format of data received and
transmitted over the network during any of these transactions is given in
(tn21).
Upon receiving a READ transaction, net-in returns the requested words at the emulation rate ( i.e. one word per emulation cycle). Following the last word, net-in sends a forward checksum byte on the data transmitted before closing the connection.
WRITE transactions are handled similar to read operations. One word is written into memory each emulation cycle. When the network is turned around following the transmission of the write data, net-in transmits an ack to indicate whether or not the write completed successfully. ack_f may occur if the host interface takes over the node's bus or if the processor aborts the net-in operation.
Where net-in and net-out perform similar functions, the
interface to net-in is intended to be compatible with
net-out's. Net-in looks like a memory mapped peripheral in the
processor's address space. While a net-in unit really only has a
single data port, each net-in is mapped to several addresses. These
distinct addresses are used by the interface so that different function and
meaning can be attributed to memory operations. These addresses are shown
in Table . A pair of physical addresses is shown for
each logical address in Table
since the master and slave
net-in are mapped into different addresses. Net-in actually
only deals with 22-bit word addresses. For simplicity
table
shows 24-bit byte addresses where the low two bits
are always zero since the addresses are word aligned. These addresses will
be mapped into some 16 megabyte region of address space ( e.g.
NI_OPERATION = 0x000200
0xFE000200).
These addresses make available a number of basic primitives for interfacing
with net-in and monitoring its operation. Again, these operations
are very similar to their net-out counterparts. These basic
primitives and their relation to the visible addresses are shown in
Table .
The general format of an operation is:
0x00 0x00
0x00
FUNCTION
Since net-in has no need for the operations defined by net-out which used the top three bytes of the operation word, these three bytes are unused. These bytes should be all zeros, as shown, in case a use for them arises later. As for net-out FUNCTION specifies a function to be performed by the local node.
The function is only field which is interpreted net-in operations.
Functions are shown in Table .
DATAEND indicates that no more data will be supplied by the processor for the network transaction in progress. ENDNOW instructs net-in to close an open connection that the processor does not wish to use. RESPONDFINAL and RESPOND initiate remote operation responses. RESPONDFINAL closes the connection after sending data whereas RESPOND turns the connection around again for another reply.
ABORT instructs net-in to drop the current operation and return to its idle state as soon as possible.
Net-in has a number of configurable options. It is possible to
specify the number of dummy cycles between each real network data cycle by
setting dummy-cycles. The number of
network stages can be set by setting stages. The node number can be
set by setting node-number. Net-in can be configured such that
it even lets the processor deal with primitive reads and writes by setting
polled-only; by default, polled-only will be deasserted.
Figure
shows the composition of the configuration word.
Individual portions of the word cannot be set independently. To change
just part of the configuration, read the configuration, reset the desired
bits, and rewrite the configuration back. The unused bits in the
configuration word are available for other configuration options which may
come up during design and prototyping. Space is specifically left next to
the number of dummy cycles so this parameter can be expanded if early
experience with MBTA indicates the number of allocated bits is
insufficient.
Reading NI_STATE will return the state of net-in. The format and meaning of this word will be defined as net-in is implemented. Some subset of these bits should indicate what net-in is expecting from the processor. This may also be useful for keeping the processor in synch with net-in. The state as a whole should be useful in diagnostic testing.
Net-in state should indicate the following in readily accessible form:
When errors occur such that net-in is forced to signal the processor
that an error has occurred using its line, the
processor should be able to determine the error by reading NI_STATE.
The current list of possible errors is shown in Table
.
N.B. All of the errors shown so far are essentially fatal. When one
of these errors occurs, either the processor and net-in are in
inconsistent states or there is a bug in the source program. The assertion
of indicates that such an error has occurred;
the processor should halt and signal the error to the host so the source of
the error can be located and debugged.
The address NI_ACK is used to check the successful completion of an
operation. After any operation which sends a response over the network
( i.e. ROP), it indicates whether or not the reply checksum
was correct. Reading NO_ACK gives the processor an entire word, but
only the lowest bit is important. If this bit is clear, the operation failed (
i.e. the checksum was invalid). If this bit is set, the operation
succeeded ( i.e. the checksum was valid).
The 0b11 encoding is unused to remain consistent with RN1's status indication. It is possible to fail to make a connection through net-in for any of the following reasons:
Figure shows a diagram of the basic connections from and to
net-in. Table
describes the signals used by
net-in. Note that the Figure
is not
specific about signal polarity; Table
, however, does
indicate signal polarity. Net-in is shown configured as part of an
MBTA node in (tn25). Table
summarizes the signal pin
requirements for net-in.
The processor interacts with Net-in primarily to service ROP network transactions (tn19) [DeH90b]. These transactions allow almost arbitrary network transactions to be handled at the software level. Net-in communicates over the network with the streams described in (tn21). The processor interface described above is intended to allow the processor to provide data transmissions for net-in and control the higher level details of these transactions. This section delves into the specifics of how the processor is expected to use net-in to service remote operations.
All the examples in this section concentrate on the i/o operations between the processor and net-in. Intervening computation by the processor is categorically omitted. Breaks between emulation cycles are indicated in the examples by a pair of horizontal lines.
The basic flow of processor interaction with net-in for an ROP is shown below:
The sequence allows the network to be turned around as many times as the software for a particular remote operation desires. The connection can be dropped from either end of the network as appropriate.
Network operations which never turn the network around are generally uninteresting since they do not get any indication as to whether the connection was successful. For maximum flexibility, remote operations are provided with this option as the simplest ROP.
Many remote operations will probably only want to issue a request and get a response. These will proceed more as follows:
Often, it may be necessary to hold more of a dialogue between remote processor. In this example, the network is turned around twice before completing the operation.
The network can be turned around again as shown in the following example:
The network can be turned around as many times as is needed to implement a particular remote operation
Sometimes, it may be desirable to allow an optional response. That is, one node may turn the connection around so the other can reply. The processor replying may have the option to just close the connection without sending any further data. This is a case where the ENDNOW operation may be used.
This will be fleshed out as implementation proceeds.