<html><head><meta name="color-scheme" content="light dark"></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">\documentclass[onecolumn,letterpaper,11pt]{article}

%=================================================================
\input{common_latex/common_latex_packages}
\input{common_latex/common_latex_definitions}
\input{common_latex/common_latex_help_format}
%=================================================================

\begin{document}

\wikidoc{file=IvPTools.PLogger}
\wikidoc{auth=Paul Newman}

%=================================================================
\section{Logging Data with pLogger}
%=================================================================
\label{sec_moos_plogger}

The \app{pLogger} process is intended to record the activities of a
MOOS session. It can be configured to record a fraction of or every
publication of any number of MOOS variables. It is an essential MOOS
tool and is worth its weight in gold in terms of post-mission
analysis, data gathering and post-mission replay.

\pskip
The configuration of \app{pLogger} is trivial and consists of
multiple lines with the following syntax:

\vspace{0.1in}
\begin{fileverb}
 Log =  varname @ period [NOSYNC],[MONITOR]
\end{fileverb}
\vspace{0.1in}

\noindent
where \psvar{varname} is any MOOS variable name and \psvar{period} is
the minimum interval between log entries that will be recorded for the
given variable. For example if \psvar{varname = INS\_YAW} and
\psvar{period = 0.2} then even if the variable is published at 20Hz it
will only be recorded at 5Hz. The optional \var{NOSYNC} flag indicates
that this variable should not be recorded in the synchronous logs (see
section \ref{sec:LogTypes}). The optional \var{MONITOR} flag tells
\app{pLogger} to send a notification if this variable isn't logged at
least every 10 seconds. The notification occurs under the
\mvar{MOOS\_DEBUG} variable. If you are running \app{iRemote} (which
subscribes to this variable automatically) you'll see a warning
printed to the screen. This can be pretty useful when runnning a
complicated system and you really do want notification that an
important variable isn't being logged (probably because the process
producing the variable is kaput in some sense.)

%=================================================================
\section{Logging Session}

The logger supports the notion of Logging Sessions. For each log
session the Logger will create a new directory and place all the
logged files within that directory. These are typically (see later
sections) an alog file, an slog file, a system log file, a copy of the
mission file (moos file), and a copy of the behavior file (bhv file).
A new log session can be created by writing the variable
\mvar{LOGGER\_RESTART} to the \app{MOOSDB} or if you are using
\app{iRemote} by pressing shift-g.

%=================================================================
\section{Log File Types}
\label{sec:LogTypes}

The logger records data in two file formats - synchronous ("slog"
extensions) and asynchronous ("alog" extensions). Both formats are
ASCII text -- they can always be compressed later and usability is
more important than disk space. The two formats are now discussed.

%=================================================================
\subsection{Synchronous Log Files}

Synchronous logging makes a table of {\em numerical} data. Each line
in the file corresponds to a single time interval. Each column of the
table represents the broad evolution of a given variable over
time. The time between lines (and whether synchronous logging is even
required) is specified with the line

\vspace{0.1in}
\begin{fileverb}
 SyncLog = true/false @ period
\end{fileverb}
\vspace{0.1in}

\noindent
where {\em period} is the interval time. If there has been no change
in the numeric variable between successive time steps then its value
is written as \var{NaN}. It is important to note that synchronous
logs do not capture all that happens - they sample it. Synchronous
logs are designed to be used to swiftly appraise the behaviour of a
MOOS community by examining numeric data in a tool such as Matlab or a
spreadsheet. The \var{MOOSData} Matlab script reads in these files
and with a single mouse click can display the time evolution of any
logged variable.

%=================================================================
\subsection{Asynchronous Log Files}

Asynchronous logging is thorough. The mechanism is designed to be
able to record {\em{every}} delta to the MOOSDB. The use of the
period variable allows the mission designer to back off from this
ultimate limit and record variables at a maximum frequency. The
key properties of asynchronous can be enumerated as follows:

\begin{packed_enumerate}
\item Records both string and numeric data
\item Records data in a list format - one notification per line
\item Entries only made when variable is written
\end{packed_enumerate}

Asynchronous log files are designed to be used with a playback tool
(for example \app{uPlayback} or other purpose-built
executable). Although the handling of strings and numeric data adds a
slight overhead to such a program's complexity the utility gain from
being able to slow, stop and accelerate time during a post-mission
replay/reprocessing session is simply massive. Turn asynchronous
logging on with:

\vspace{0.1in}
\begin{fileverb}
 AsyncLog = true
\end{fileverb}
\vspace{0.1in}

\noindent
in the logger configuration block.

%=================================================================
\section{System Log File}

There is a third kind of log file that is produced - a system log file
(ylog). This file only contains data contained in \mvar{MOOS\_SYSTEM}
and \mvar{MOOS\_DEBUG} messages. The later can be written to by
calling the \func{CMOOSApp::MOOSDebugWrite} method from any
\var{CMOOSApp} derived class. The thinking behind the ylog files is
that its pretty useful to be able to browse through a text file of
events to see when and if things went wrong in a mission. Processes
can write to \mvar{MOOS\_SYSTEM} and or \mvar{MOOS\_DEBUG} if
something happens which they think is salient. Think of this as
\var{/var/log}.


%=================================================================
\section{Dynamic Logging Configuration}

Post V7.0.1 releases of MOOS include a dynamic logging
ability. External parties can, at run time request the logger to start
logging particular (named) variables. The logger subscribes to
messages called \mvar{PLOGGER\_CMD} (if the Logger is being run under
the MOOS name "pLogger" otherwise subsitute the relevant name) and by
correctly formatting the string data of this message dynamic logging
can be invoked.

\vspace{0.1in}
\begin{fileverb}
 LOG_REQUEST =  varname @ period [NOSYNC],[MONITOR]
\end{fileverb}
\vspace{0.1in}

\noindent
Note that this string format is identical to that found in the mission
file modulo \var{LOG} being replaced with \mvar{LOG\_REQUEST}. Now
dynamic logging sits naturally with "alog" (asynchronous) files but
not so well with "slog" files. These files are easy to use(and hence
popular) because they are rectangular numerical array written to text
file (easily parsed by Matlab's load command). But dynamic logging
means that by the time a mission ends an unknown number of variables
(columns) will occur in every line of the slog. To address this issue
the logger can be configured (in its mission file configuration block)
with a line like

\vspace{0.1in}
\begin{fileverb}
 DynamicSyncLogColumns =  30
\end{fileverb}
\vspace{0.1in}

Here 30 columns are being reserved for variables that are requested at
run time. As dynamic requests are received and processed by the logger
these unclaimed columns are consumed until none remain. At this point
any future dynamic logging requests will still be accepted but the
logged variables won't appear in the slog file (they will of course
appear in the alog file). Unclaimed slots will be labelled
\var{DYNAMIC\_X} until claimed and will always have \var{NaN} entries.


%=================================================================
\section{Specifying Log File Names and Locations}
\label{Sec:Places}

Each time the logger starts creates (if required) a new directory in
the logging root directory (see below) and performs logging within that
directory. \app{pLogger} is quite flexible in terms of log file
configuration and is controlled by the following variables.  

\begin{packed_itemize}[1]

\item \var{GlobalLogPath}: This is a {\bf file scope variable} (ie not
  in any process configuration block). If it is present in a mission
  file then it specifies the root directory in which log files will be
  created.

\item \var{Path}: specifies the root logging directory but only used
  if \var{GlobalLogPath} is not set (see above).  This path must exist, 
  \app{pLogger} will not create it on startup.  If the path does not 
  exist, \app{pLogger} will default to the current working 
  directory (`.') as described below.

\item \var{File}: this is the stem file name given to logged files
  (alog/slog and ylog)

\item \var{FileTimeStamp}: if this is set to true, the name of each
  logged file (and created containing directory) will be the
  concatenation of the \var{File} variable and a time stamp. If
  \var{FileTimeStamp} is false then each time the logger is run it
  will write to the same set of log files {\bf and destroy the
    original contents}. This is by design, when developing it's often
  useful to not have useless log files take ever more space on your
  machine.
\end{packed_itemize}

\noindent
Note that if the logger is run without a mission file it starts
logging to the local directory with file time stamping enabled. See
Listing \ref{plogger_config} for a typical logger configuration
block.

\pskip

If for some reason the logger was unable to start logging to a
location specified in the log file it trys, as a last resort, to open
a log directory in the current working directory (`.'). If this
fallback fails then all is lost, we are without hope and the Logger
exits. If you are running a mission from pAntler you'll be notified
that the logger has quit.

%=================================================================
\section{Mission Backup}
Simply having the {\em alog} and {\em slog} files is not enough to
evaluate the mission. One also needs the things that {\em caused} the
data to be recorded, namely the \var{*.moos} mission file or the
\var{*.bhv} behavior file. To this end the \app{pLogger} process takes
a copy of these files and places them (name appended with a time stamp
if desired) within the logging directory. The files extensions are
renamed to \var{*.\_moos} and \var{*.\_bhv} respectively.

%=================================================================
\subsection{Additional File Backup on Demand}

It is quite within the bounds of reason that a system using MOOS makes
use of configuration (or other files) which has content which should
not reside inside a *.moos or *.hoof file.  \app{pLogger} provides a
mechanism to back these files up on request at run time. Post V7.0.1,
by writing to the \mvar{PLOGGER\_CMD} variable any MOOS process can
instruct the logger to back a local file up and place a copy in the
current logging directory. See \ref{Sec:Places}. As always the process
\mvar{\_CMD} message is a string and for dynamic file back up should
be formatted as:

\vspace{0.1in}
\begin{moosfileverb}
 COPY_FILE_REQUEST = PathToFile
\end{moosfileverb}
\vspace{0.1in}

for example

\vspace{0.1in}
\begin{moosfileverb}
 COPY_FILE_REQUEST = /home/VehicleQ/CameraCalibration.txt
\end{moosfileverb}
\vspace{0.1in}

\noindent
would log \var{"/home/VehicleQ/CameraCalibration.txt"} to a file called
\var{"CameraCalibration.\_txt"} in the current logging directory. Note the
additional underscore which is in keeping with the backup style used
for mission files. If the file being backed up has no extension a
\var{".\_bak"} is added.

%========================================================================
\subsection{Wildcard Logging}

Post V7.0.1 a new configuration option has been introduced which
instructs the logger to log every change made to the \app{MOOSDB} to
an alog file. (It does this by using a specialised MOOS IPC call to
the DB which is not intended for daily use). By writing

\vspace{0.1in}
\begin{moosfileverb}
 WildCardLogging =  true
\end{moosfileverb}
\vspace{0.1in}

Asynchronous logging will be turned on (even if \pvar{AsyncLog =
 false} is present in the configuration block) and every change to
every variable will be logged to the current alog file. Some points to
note:

\begin{packed_itemize}[1]
\item Wildcard logging only causes variables to be written to the
  current alog file. Synchronous logging is not affected
\item If the log rates for variables have already been been specified
  at configuration then it is these rates that are used
\item If wild card logging is enabled, when the logger spots that
  variables that are not already known to it are being written to the
  MOOSDB it registers to be told about every change made (like
  \pvar{Log = XXX @ 0}).
\item By default wildcard logging is disabled. It was designed to be a
  safety net for situations in which many new variables are being
  created by a software team in a dynamic environment.
\item It may take up to a second for the logger to detect and
  subscribe for new MOOS variables. If new data is written multiple
  times before subscription can complete (i.e within a second), the
  logger will be unable to capture these writes.
\end{packed_itemize}

%=================================================================
\section{Runtime File Backup}

By writing a correctly formatted string value to the MOOS variable
\mvar{PLOGGER\_CMD}, processes within the MOOS community can request
the logger to back up arbitrary files to the current log
directory. For example if the contents of a message with name
\mvar{PLOGGER\_CMD} are set to

\prevgreen
\begin{verbatim}
 PLOGGER_CMD = "COPY_FILE_REQUEST=/home/pnewman/code/TheFile.xzy"
\end{verbatim}
\postverb

\noindent
and published to the \app{MOOSDB} it will result in a copy of
\var{/home/pnewman/code/TheFile.xzy} being placed in the logging
directory under the name \var{TheFile.\_xzy} - note the additional
underscore. If the file in question has no extension then \var{.\_bak}
is added appended to the file name.

%=================================================================
\section{Example Configuration}

\vspace{0.1in}
\begin{listing}{An example pLogger configuration block}
\label{plogger_config}
\begin{moosfileverb}
 1  //------------------------------------------------------------------
 2  // pLogger configuration block
 3  ProcessConfig = pLogger
 4  {
 5      //over loading basic params..lets be fiesty
 6      AppTick  = 20.0
 7      CommsTick   = 20.0
 8  
 9      //all file names begin with this stem
10      File        = SciPark29Mar
11  
12      //where is the root log directory
13      PATH        = /home/doe/MOOSData/SciencePark/
14  
15      //yes we want some sync logging for crude
16      //performance checking
17      SyncLog     = true @ 0.2
18  
19      //yes we want async logging so we can replay
20      // exactly what happened and record strings
21      AsyncLog    = true
\end{moosfileverb}

\begin{moosfileverb}
22  
23      WildCardLogging = false
24  
25      //yes append each created directory log file
26      //with a time stamp DAY MONTH YEAR TIME
27      FileTimeStamp   = true
28  
29      //what do we want to log (zero means capture everything)
30      Log     = LMS_LASER_2D   @ 0 MONITOR
31      Log     = LMS_LASER_3D   @ 0 MONITOR
32      Log     = MARGE_ODOMETRY @ 0
33      Log     = DESIRED_RUDDER @ 0
34      Log     = DESIRED_THRUST @ 0
35      Log     = CAMERA_GRAB    @ 0
36      Log     = GPSData        @ 0.4
37  }
\end{moosfileverb}
\end{listing}


\end{document}

</pre></body></html>