TCP networking in Oberon Workstation 1.3.1+

The TCPNet.Mod module and TCPNet Device in Oberon Workstation provide client and server internet connectivity to Oberon programs. Each of the (max 32) connections works with 2 regular Oberon files. One file receives TCP data, the other supplies data to be transmitted.

The TCPNet Device works concurrently with RISC5 execution. As data transfers to the input file occur beyond the (Oberon) file-buffer-end, and data transfers from the output file are just read-only, the TCP data stream events are not perceived during regular Oberon command processing, and they do not involve Oberon's main memory.

To make the TCPNet Device's progress available to Oberon, a (periodic) synchronizing update is performed inside the TCPNet.Mod module, using an Oberon Task. This Task obtains a service request word from the Device, where each bit represents a connection that needs attention, and then updates that connection.
The update involves 3 main actions:

Using TCPNet.Mod

To create a connection to a server

PROCEDURE Open* (conn: TCPConn; in, out: Files.File;
		address: ARRAY OF CHAR; port: INTEGER; 
		handler: Handler; VAR res: INTEGER);

This procedure requests the establishment of a connection conform the given parameters. The action occurs asynchronously by macOS system calls to its sockets library. When the requested connection succeeds, streaming is potentially immediately started if either local or remote sides have data to send. The establishment of the connection (among other events) is reported via status calls to your handler procedure.

a NEW-created TCPNet.TCPConn record. to hold the connection parameters.
in, out
the receive (in), and transmit (out) files for the connection. The files do not need to be empty, and are not required to be registered. Transmission of the out file always starts at position 0. Received bytes are always appended at the end of the in-file. The files are assigned to the connection for its entire lifetime (i.e. till the connection is closed). Each in- or out file must not be attached to more than 1 connection.
The server address. As DNS lookup is provided in the TCPNet Device, this parameter may be in domain name- or IPv4 form, e.g.:
The address string is max. 63 chars (excluding trailing null)
Port number at the server. This integer is in normal Oberon integer endianness (i.e. not in big-endian network form). Max. port number is 0FFFFH (16 bits).
Client PROCEDURE of type TCPNet.Handler , This procedure will be called if the Network Device has any status changes to report since the previous handler call, which may require a client action. see below for details on handler calls
Result. 0: success (connection requested); <0: failed request (no free handles, or direct parameter error)

Responding to handler calls

Handler* = PROCEDURE (conn: TCPConn; stat:SET): INTEGER;

The handler procedure is periodically called by TCPNet to inform your program of the latest status/progress of your connections, and allows your program to send a network response and/or operate on the new data. The handler typically controls (large) parts of your (network-based) program that are not governed by direct Oberon user-commands.
It is generally safe to call TCPNet.Mod procedures from your handling code, including when they affect other connections (opening, closing), but care should be taken to avoid accessing connections that may be part of a different program (unrelated programs). For example, a web browser and a terminal application should not access each other's connections, which would be nevertheless possible using TCPNet.Enumerate(). A recommended practical safety measure is by making use of the context field in the TCPConnDesc record, which can be extended to a unique type reserved for a particular program.

The unique connection that this status call relates to. Each connection may have its own unique handler procedure, or a handler procedure may be shared by multiple connections.
To discriminate between possible multiple connections, if direct comparison of the conn parameter is not wanted or possible, the context field of the connection record can be used to store data that can uniquely identify the connection on behalf of which the handler is called.
Status. A set of flags that report events that occured on the connection. Multiple flags may be set.
hostc IN stat
The connection was successfully established. This flag is only reported once for a connection.

You may use the TCPConn.laddr and TCPConn.lport fields of the connection record to inspect local IP address, and local port. The TCPConn.raddr and TCPConn.rport contain the remote peer address and port. Addresses are formatted in IPv4 dotted quad notation. Port numbers are in Oberon native integer form.

newrx IN stat
Some data was received and appended to the in-file since the previous handler call. The length of the in-file has increased correspondingly.
newtx IN stat, spacetx IN stat
newtx: some data was transmitted from the output file since the previous handler call. spacetx: the local transmission mechanism reached the end of the output file, and will only resume if you write more data.
Allows progress monitoring, or to withold further data transmission until some condition has has been met, e.g. receiving a password. You do not have to wait for spacetx in order to append more data to the output file. The flags do not guarantee the remote host has received anything yet.
disconn IN stat
The peer host has closed the connection, either forcefully (connection reset) or gracefully (peer closed connection). No further data will be received from this host. The connection on this side should be closed also.
limrx IN stat, error IN stat
Typically fatal errors, that render the connection stale. The proper response is to close the connection. limrx: input stream has reached the max. file size limit; error: an error in connecting, error in sockets/streaming, or a listening connection slot is created without valid listening socket on the requested port.
idle IN stat, wconn IN stat
These flags are set immediately upon opening a listening connection (idle), or normal client connection (wconn), before the connection with a peer is established.
These flags do not cause a service request, so you will normally not see them, unless they are accompanied by the error flag (which does generate a service request).
idle + error: opening a listen connection failed because the requested listening (prototype) socket could not be created;
wconn + error: establishing the connection failed because remote host did not respond, accept, or dns lookup failed.
return value of the handler
The INTEGER return value of the handler is unused in the current implementation. Returning 0 is recommended as a "neutral" value with no effect also in a future version of the software.

To close a connection

PROCEDURE Close*(conn: TCPConn);

Closing a connection is useful when a protocol "cycle" has ended, and no new actions are planned, or an error occurred. Closing saves resources on server and client sides and prevents "leaking" the connection capacity (32 slots) of the TCPNet Device. The Close procedure may be called from within the body of the handler or as part of a normal Oberon command. A system- or program-wide disconnect or a specific server-shutdown can be accomplished with the Enumerate() procedure i.c.w. a EntryHandler procedure that closes connections according to the corresponding properties.
After closing, any references to the connection record and its files are erased in the TCPNet module and TCPNet Device.

The connection record is subject to normal Oberon garbage collection. Since closing the connection has no impact on the in- and out files, these files have kept their content as it existed when closing the connection. No Close, Register or Purge etc. operations are performed on the in- and out files.

Set up a listening connection

PROCEDURE Listen* (conn: TCPConn; in, out: Files.File; 
	port: INTEGER; handler: Handler; VAR res: INTEGER);

This procedure reserves a serving connection slot where potentially a single remote client can connect to. If multiple of such listening slots are created with the same port number, effectively a "server" is established. The capacity of the server is equal to the number of created slots; the capacity may be fixed to a target amount by implementing a refreshing scheme where a new listening connection is created each time another is done being served and therefore closed.

Listening connection slots, once connected, behave essentially the same as directly requested connections. Whereas directly requested connections specify a definite peer address and start contacting the peer immediately, the serving connection slots wait to be contacted by any remote computer.

The same type of handler procedure applies to a listening connection as to a directly created connection (described in "Responding to handler calls"). A single handler for listening connections with a given port number would typically be implemented for that port number.

The parameters descriptions of Listen() are the same as those of Open(), except for a slightly different meaning of port:

Local port number of this server. This integer is in normal Oberon integer endianness (i.e. not in big-endian network form). Max. port number is 0FFFFH (16 bits). Make sure that no server port is used that is already in use by other sever software on your computer. Port numbers 1024-49151 are recommended for your own server(s) or protocols. For a web server, typically use 80.

Server behavior

The TCPNet Device allows a maximum of 4 different serving ports to be in use, thus supporting 4 servers to be working at once.

As long as one or more listening connections on a given port exist (idle or already connected), the listening socket for that port is kept alive by the TCPNet Device. New connections are accepted and reported (via the hostc flag in the handler) until no free (idle) listening slots remain.
Without free listening slots, new visitors will still be accepted however, but "queued" instead, until new free listening slots are created. With this mechanism, in a busy server with short lived connections, the "availability" perceived by visitors is improved. Their connection request is not outright refused but the effective communication with Oberon is only delayed.

When all listening connections for a given port (idle and/or connected) are closed, the listening socket is automatically removed by the TCPNet Device, so any connection attempts will then result in a "connection refused" response.
Note that after the listening socket is removed, macOS inhibits the creation of a new listening socket for the same port during a few tens of seconds. If in that holdoff period you attempt to create new listen connections they will fail (i.e. lead to error flagged calls to your handler).

When testing server operation during development, the macOS terminal command 'lsof -i' might be useful to list the current connections.

Operations on connections

PROCEDURE Send*(conn: TCPConn);

The TCPNet.Send() procedure is used to notify the TCPNet Device of new data appended to the output file; it prompts the TCPNet Device to resume transmission if it was pausing at the end of the previously appended data.

Explicitly prompting the transmission allows a client-determined accumulation of data in the output file before it is sent. E.g. in a chat program, the choice could be made to send each typed character individually, or to send only entire lines of text.

When data to be sent is produced within a handler body, the Send() procedure is not needed, since transmission of any appended new data in that case is implicitly taken care of by the service request mechanism of TCPNet.Mod

PROCEDURE InFile*(conn: TCPConn): Files.File;
PROCEDURE OutFile*(conn: TCPConn): Files.File;

Return the input- and output files of the connection.

PROCEDURE Enumerate*(proc: EntryHandler);

Calls the supplied EntryHandler procedure successively with all current connection records kept by TCPNet.Mod and the TCPNet Device, until one sets its continue parameter to FALSE. Typically, to filter the connection records, your EntryHandler would inspect the TCPConn fields and in particular the custom extension type and contents of its context field