<html><head><meta name="color-scheme" content="light dark"></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">\section{Using Goby (pAcommsHandler) with MOOS-IvP} \label{sec_acomms_example}

\subsection{Launch a one-way point-to-point network}
We're going to start with a basic two community network that is a simple version of a common underwater scenario - a ``sender'' community (often a subsea node collecting data), and a ``receiver'' (typically a topside on a ship). In reality, communications are bi-directional as we want to command the vehicle from the topside as well, but typically the data flow is lopsided, weighted heavily towards the data-collecting node.

Find the mission named ``acomms1'':
\begin{verbatim}
cd moos-ivp-extend-acomms/missions/acomms1
\end{verbatim}

Launch it:
\begin{verbatim}
./launch.sh
\end{verbatim}

You will see two communities launch: the \textbf{sender} on the left side of the screen and the \textbf{receiver} on the right side. Both communities have a window for pAcommsHandler (the MOOS interface to the \verb|goby_acomms| library) in \verb|show_gui: true| mode that provides a large amount of diagnostic information. The other applications include:

\begin{itemize}
\item pAcommsExample: This application, running on the sender community, is continuously publishing two message types containing an index, a random number or boolean (placeholder for real data), and the sender of the message (currently my MIT id, \verb|tes| or Henrik's id, \verb|henrik|). The process configuration block (in \verb|meta_sender.moos| for this application lists two variables of interest: 
\begin{verbatim}
post_interval1 = 4
post_interval2 = 1e100
\end{verbatim}
\verb|post_interval1| is the period (in seconds) between posts for messages from me (TES), and \verb|post_interval2| is the period between posts for messages from Henrik (HENRIK). For now, I set \verb|post_interval2| very large, effectively disabling messages from Henrik.
\item uXMS: You are familiar with this application. On the sender community, it will show the output of pAcommsExample (MOOS variables \verb|ACOMMS_EXAMPLE1_OUT| and \verb|ACOMMS_EXAMPLE2_OUT|). On the receiver community, it shows the equivalent received variable after sending through the (simulated) acoustic link. There will be an instance of uXMS running for messages from me and from Henrik.
\item pCoroner: Similar to uProcessWatch, shows if any applications known to pAntler are not currently running.
\end{itemize}


Some things to note before we continue:
\begin{itemize}
\item Vehicles are identified by an unsigned integer (or ``modem id'') that is much like a MAC address on your computer's network card. String names like ``gilda'' are too costly in message space to use. We can use lookup tables, however, to tie modem ids to names. The modem id ``0'' is reserved to indicate ``broadcast'' in Goby. All vehicles will decode messages sent to ``0''.
\end{itemize}

\subsection{pAcommsHandler terminal output}

pAcommsHandler is the focus here, so watch the text in the terminal window to get a sense of what is going on. Maximize it as needed. From the top left, going to the bottom left, and over to the right column, the diagnostic displays indicate the modules of \verb|goby_acomms| in order from application layer to link layer (see Fig. \ref{fig:structure}):

\begin{figure}
\begin{center}
	\includegraphics[width=8.4cm]{structure.pdf}
	\captionof{figure}{The software structure of Goby, showing the approximate OSI network layer on the left. In this lab, we will focus mostly on the DCCL and ModemDriver components.}
	\label{fig:structure}
\end{center}
\end{figure}

\begin{itemize}
\item \textbf{Ungrouped messages}: messages that don't have a clear home.
\item \textbf{pAcommsHandler}: messages specific to the interaction between Goby and MOOS. In this window you will basically see a copy of what pAcommsExample is publishing.
\item \textbf{goby::acomms::dccl::encode}: shows the Dynamic Compact Control Language (DCCL) encoder output. When a message is encoded for sending, a line is printed here indicating that.
\item \textbf{goby::acomms::dccl::decode}: shows the Dynamic Compact Control Language (DCCL) decoder output. When a message is decoded after receiving, a line is printed here indicating that.
\item \textbf{goby::acomms::queue::push}: shows when a message is pushed (added) to the Goby dynamic priority queue. 
\item \textbf{goby::acomms::queue::pop}: shows when a message is popped (removed) from the Goby dynamic priority queue. 
\item \textbf{goby::acomms::priority}: shows the results of the priority contest that is held between all queues when it is time to send a message.
\item \textbf{goby::acomms::queue::out}: Messages showing the output of the queue data to the ModemDriver (link layer).
\item \textbf{goby::acomms::queue::in}: Messages showing the input of the ModemDriver to the queuing layers. Messages are not queued upon receipt, but are rather immediately decoded.
\item \textbf{goby::acomms::queue::amac}: Acoustic Medium Access Control (MAC) unit. This governs the time schedule when a given node is allowed to send. In this example, we have a single slot: vehicle 1 sends to the destination of the next message.
\item \textbf{goby::acomms:modemdriver::out}: Raw outgoing messages to the modem hardware. In this case of this example, we are using a driver that passes messages through MOOS (using the uField \verb|NODE_MESSAGE| variables). Switching to a real modem such as the WHOI Micro-Modem is only a matter of changing a few configuration variables.
\item \textbf{goby::acomms:modemdriver::in}: Raw incoming messages from the modem hardware.
\end{itemize}



\subsection{pAcommsHandler MOOS API}\label{sec:MOOSAPI}

pAcommsHandler writes a variety of messages to MOOS variables that by default begin with \verb|ACOMMS_|. The contents of all these messages are human-readable representations of Protobuf messages. Open uMS or uXMS and take a look at these variables (these are the default names, all of which can be configured in the MOOS file if desired). Some will only be published on one community or the other:

\begin{itemize}
\item \verb|ACOMMS_RAW_INCOMING|: Shows the serial or TCP feed \textbf{from} the current acoustic modem. The uField Driver uses a hexadecimal representation of the Protobuf encoding of \\goby::acomms::protobuf::ModemTransmission. The WHOI Micro-Modem uses an NMEA-0183 interface as you'll see in the next section.
\item \verb|ACOMMS_RAW_OUTGOING|: Shows the serial or TCP feed \textbf{to} the current acoustic modem. 
\item \verb|ACOMMS_MODEM_RECEIVE|: Abstracted message from the Modem driver. This message will look the same regardless of which low-level driver is being used. The \texttt{frame} field contains a DCCL encoded message.
\item \verb|ACOMMS_MODEM_TRANSMIT|: Abstracted message to the Modem driver. This message will look the same regardless of which low-level driver is being used. The \texttt{frame} field contains a DCCL encoded message.
\item \verb|ACOMMS_QUEUE_RECEIVE|: Decoded received message. This should look very much like the final published message.
\item \verb|ACOMMS_QUEUE_TRANSMIT|: Unencoded sent message. 
\item \verb|ACOMMS_QSIZE|: Indicates whenever a queue changes size (due to a message being pushed or popped).
\item \verb|ACOMMS_MAC_INITIATE_TRANSMISSION|: Indicates the beginning of a transmission.
\end{itemize}

The uField Driver also uses a few more MOOS variables to do its work:
\begin{itemize}
\item \verb|NODE_MESSAGE_LOCAL|: posted to allow the uField tools to route the message via the shoreside to another vehicle. This gives us some rudimentary network simulation capabilities (primarily range-dependent packet dropping).
\item \verb|ACOMMS_UFIELD_DRIVER_OUT|: copy of the \verb|string_val| field of the message posted \verb|NODE_MESSAGE_LOCAL| in the same encoded format as will be published to \verb|ACOMMS_UFIELD_DRIVER_IN| on the receiving end.
\item \verb|ACOMMS_UFIELD_DRIVER_IN|: posted by uFieldMessageHandler after routing the message originally posted to \verb|NODE_MESSAGE_LOCAL|. This is subscribed to by the Goby uField Driver.
\end{itemize}


\subsection{Comparing encoding representations}

Look at the terminal output of \verb|pAcommsExample|. After each message, the size is written if the message were encoded using a std::string (``TextFormat''), the built-in Protobuf encoder, and DCCL. It should be clear from this that DCCL offers the smallest packet size. The drawback is the extra work precisely defining the origin of our messages' data. We will look at this briefly in the next subsection (\ref{sec:dccl15}), and substantially more using a probabilistic framework in Section \ref{sec_ctd_encode}.

\textbf{Using this mission, please answer the first question (and all its parts) below in Section \ref{sec:q15}.
}
\subsection{DCCL Messages: AcommsExampleMessage1 and AcommsExampleMessage2}\label{sec:dccl15}

The messages AcommsExampleMessage1 and AcommsExampleMessage2 are Google Protocol Buffers message defined in
\begin{verbatim}
moos-ivp-extend-acomms/src/protobuf/acomms_example.proto
\end{verbatim}

To allow DCCL to encode this message, some custom message (\verb|dccl.msg|) and field extensions (\verb|dccl.field|) are defined to allow us to attach further meta-data to the message definition. This additional information is used to produce an even smaller encoded message than the default Protobuf technique. Only a subset of the extensions are used here. The full set is given in \url{http://gobysoft.org/doc/2.0/dccl__option__extensions_8proto_source.html}, and the requirements of the various built-in encoders/decoders (``codecs'') is given on this page: \url{http://gobysoft.org/doc/2.0/acomms_dccl.html#dccl_options}.

Normally, integers are stored in 32 or 64 bits, but most of the time we're actually sending messages from a small subset of the available $2^{32}$ or $2^{64}$ integers. Thus, the built-in DCCL codecs use arbitrarily bounded integers to send data\footnote{While slightly out of date, this paper gives the algorithms for the built-in codecs: \url{http://gobysoft.com/dl/dccl_oceans10.pdf}}. For example, if you set
\begin{verbatim}
(dccl.field).max = 99, (dccl.field).min = 0
\end{verbatim}
the default DCCL codec for \verb|int32| creates a 7-bit integer\footnote{$\hbox{ceil}(\log2(99-0+1))=7$} to store it, a savings of 25 bits or 78\% over using a 32-bit integer. This is how the field named \verb|random| in \verb|AcommsExampleMessage1| is being encoded. 

\subsection{Examine a special purpose encoder: VarIntCodec} \label{sec:varint}

While this simple technique for reducing the size of our messages is a useful first attempt to send tightly compressed data, we can often do better than this. The key to this improvement is to understand the underlying \textit{probability distribution} of the data (or ``symbols'') to be sent. Let's assume that the small numbers (say 0-127) are sent much more frequently than large numbers, but we still want to be able to handle larger numbers as the need arises. One way to do this is with a variable integer (``varint'') encoder\footnote{as an interesting side note, variable integer encoding is how Google Protocol buffers default encoding works: see \url{http://code.google.com/apis/protocolbuffers/docs/encoding.html}}, which uses one byte for the values 0-127 and two bytes for 128-32768. The final bit is used for indicating the size of the range (large or small). For example, the small number 50 would be encoded as:
\begin{verbatim}
0110010 0
   ^    ^
   |    |
  d50   0 means short form (1 byte = 8 bits)
\end{verbatim}
On the other hand, the large number 500 would be
\begin{verbatim}
000000111110100 1
       ^        ^
       |        |
     d500       1 means long form (2 bytes = 16 bits)
\end{verbatim}

Note that the first bit read in DCCL is the \textit{least significant bit} (i.e. $2^0$) so the first bit encountered tells the decoder whether the integer is written in long form or short form.

Such a codec can be written outside the Goby project and imported as a plugin. To do so, we define some name for our codec and write it into the Protobuf message (in this case in the \verb|index| field of AcommsExampleMessage2):
\begin{verbatim}
(dccl.field).codec="lab12_varint"
\end{verbatim}

To write our codec, we subclass DCCLTypedFieldCodec\footnote{\url{http://gobysoft.org/doc/2.0/classgoby_1_1acomms_1_1DCCLTypedFieldCodec.html}} for singular fields (\verb|optional| or \verb|required| in Protobuf parlance) or DCCLRepeatedTypedFieldCodec\footnote{\url{http://gobysoft.com/doc/2.0/classgoby_1_1acomms_1_1DCCLRepeatedTypedFieldCodec.html}} for \verb|repeated| fields (which are basically vectors). This is done for the varint codec in
\begin{verbatim}
moos-ivp-extend-acomms/src/codecs/varint.h
moos-ivp-extend-acomms/src/codecs/varint.cpp
\end{verbatim} 
You will want to look at this code to get an idea of how a simple variable-length codec looks in advance of writing a slightly more complicated one in section \ref{sec_ctd_encode}.

Finally, we must load our codecs using the code given in 
\begin{verbatim}
moos-ivp-extend-acomms/src/codecs/codec_load.cpp
\end{verbatim}
All this code is compiled into a dynamic library (\verb|lab12codecs.so/dylib|) that is loaded by pAcommsHandler\footnote{using dlopen}, which in turn loads the codecs into the Goby DCCL module. This extensibility allows for tailoring acoustic messaging to specific mission needs, \textit{without needing to modify the Goby source code}. However, this level of effort isn't always required, as the built-in codecs provide useful functionality for deployments with slightly less rigorous data compression requirements.

\subsection{Questions}\label{sec:q15}

Some quick questions for you to consider and hand in by the end of the lab on Thursday.
\begin{enumerate}
\item pAcommsHandler uses a dynamic priority queue to buffer different message types. Each queue is configured with a time-to-live (\verb|ttl|) and a base priority (\verb|value_base|). This question deals with a few basic elements of queuing different messages, especially when the link becomes overcapacity (more messages are available than can be sent). The queue configuration is in \verb|plug_pAcommsHandler.moos| under \verb|queue_cfg { message_entry  { } } |. You may find it helpful for these questions to look at the \verb|goby::acomms::queue::push| window which shows the queue size after each message is pushed.
\begin{enumerate}
\item With \verb|post_interval1 = 4| in pAcommsExample's ProcessConfig block, you will note that all of the messages get through to the receiver over the simulator acoustic link (which has the bitrate of 32 bytes/10 seconds = 25.6 bps). Given the size of the DCCL messaages shown by pAcommsExample, what is the lowest \verb|post_interval1| (highest rate) that will allow all messages to get through? Given this interval setting, what is the smallest \verb|max_queue| needed for the AcommsExampleMessage1 type? Set \verb|post_interval1| to this value and test that your answer works.
\item Set \verb|max_queue| to some large value (e.g. 100) for both messages, but now set \verb|post_interval2| to the same value as \verb|post_interval1|, enabling the generation of messages from Henrik at the same rate as messages from me. This scenario more closely approximates the situation on a real underwater vehicle (but of course with non-trivial messages). What happens to the latency at the receiver (defined as the difference from the time that a message with \verb|index: N| is published on the sender to the time it is received)? In other words, if $L(N)$ is the latency of the message with \verb|index: N|, what (approximately) is $L(N)$? Why? List two ways you might resolve this problem.
\item Try setting \verb|newest_first: true| (FILO) for both message queues. Give an example of a message that would this be preferable to use \verb|newest_first: true|? In what cases would \verb|newest_first: false| (FIFO) Ebe better?
\end{enumerate}
\ifx\answer\undefined
\else
{\color{red}
}
\fi

\item If we were using the varint codec given in section \ref{sec:varint} and our numbers were almost always $large$, we would use 16 bits to encode symbols that only require 15 bits to encode (because we use 1 bit to indicate the size domain). Thus, there is some probability, $p_0$, where $P(small) &gt;= p_0$ that makes use of the varint codec worthwhile (i.e. it will use fewer bits \textit{on average}). What is that value $p_0$? As before, $large$ is defined as values [128, 32768) and $small$ is [0, 128).

\ifx\answer\undefined
\else
{\color{red}
Define $\bar{s}_{varint}$ as average size in bits of message using varint codec, $\bar{s}_{normal}$ using normal bounded integer. We know:
\begin{eqnarray*}
\bar{s}_{varint} = 8P_{small} + 16P_{large}
\end{eqnarray*}
and
\begin{eqnarray*}
\bar{s}_{normal} = 15
\end{eqnarray*}

Due to the fundamental rules of probability
\begin{eqnarray*}
P_{small} + P_{large} = 1
\end{eqnarray*}

We want to know $P(small) &gt;= p_0$ such that 
\begin{eqnarray*}
\bar{s}_{varint} &amp;&lt;=&amp; \bar{s}_{normal}\\
8P_{small} + 16P_{large} &amp;&lt;=&amp; 15\\
8p_0 + 16 (1-p_0) &amp;=&amp; 15\\
-8p_0 &amp;=&amp; -1\\
p_0 &amp;=&amp; \frac{1}{8}
\end{eqnarray*}

}
\fi

\end{enumerate}



</pre></body></html>