<html><head><meta name="color-scheme" content="light dark"></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">%=================================================================
\section{The aloggrep Tool}
%=================================================================
\label{sec_aloggrep}
\index{aloggrep!Overview}

The \app{aloggrep} tool has two primary purposes. The first is to
prune a given \var{.alog} file into a smaller, yet syntactically valid
\var{.alog} file. It this mode it can also be used for enforcing strict
ordering of entries, and removing duplicate entries, since the \app{pLogger}
app does not guarantee perfect ordering or files without the occasional
duplicate entry. The second primary purpose is to extract data for plotting
in a tool like matlab or similar, or extracting a single value that may
be ingested in a shell script. These use cases are described below.

\vspace{0.1in}
\noindent
A concise form of this documentation is available with \var{aloggrep --help},
and the web version of this content can be quickly opened with
\var{aloggrep --web}.

%=================================================================
\subsection{Using aloggrep to Produce a Reduced and/or Ordered Output}

The \app{aloggrep} tool will prune a given \var{.alog} file by
retaining lines of the original file that contain log entries for a
user-specified list of MOOS variables or MOOS processes (sources). As
the name implies it is motivated by the Unix \var{grep} command, but
\var{grep} will return a matched line {\em regardless} of where the
pattern appears in the line.  Since MOOS variables also often appear
in the string content of other MOOS variables, \var{grep} often
returns much more than one is looking for. The \app{aloggrep} tool
will only pattern-match on the second column of data (the MOOS
variable name), or the third column of data (the MOOS source), of any
given entry in a given \var{.alog} file.

\vspace{0.1in}
\noindent
For example, try running the alpha mission and let the vehicle
traverse the waypoints for a little while. Quit the mission and find
the \var{.alog} file.

\vspace{0.1in}
\begin{consoleverb}
$ cd alpha_4_1_2022_____18_33_46/
$ ls
alpha._bhv                         alpha_4_1_2022_____18_33_46.blog
alpha_4_1_2022_____18_33_46._moos  alpha_4_1_2022_____18_33_46.xlog
alpha_4_1_2022_____18_33_46.alog   alpha_4_1_2022_____18_33_46.ylog
\end{consoleverb}
\vspace{0.1in}

\noindent
Then we can use \cmd{aloggrep} to see what is happening with the
\mvar{DEPLOY} and \mvar{WFLAG} variable. The former is used for
starting the mission, while the latter is incremented each time the
vehicle hits a waypoint:

\vspace{0.1in}
\begin{consoleverb}
$ aloggrep DEPLOY WFLAGalpha_4_1_2022_____18_33_46.alog
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% LOG FILE:       ./alpha_4_1_2022_____18_33_46/alpha_4_1_2022_____18_33_46.alog
%% FILE OPENED ON  Wed Dec 31 19:00:00 1969
%% LOGSTART           41033480654.29403
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
26.28008        DEPLOY               pHelmIvP:HELM_VAR_INIT false 
26.28008        DEPLOY               pHelmIvP:HELM_VAR_INIT false 
26.28008        DEPLOY               pHelmIvP:HELM_VAR_INIT false 
26.28008        DEPLOY               pHelmIvP:HELM_VAR_INIT false 
26.28008        DEPLOY               pHelmIvP:HELM_VAR_INIT false 
26.28008        DEPLOY               pHelmIvP:HELM_VAR_INIT false 
74.65635        DEPLOY               pMarineViewer   true 
74.65635        DEPLOY               pMarineViewer   true 
74.65635        DEPLOY               pMarineViewer   true 
85.23170        WFLAG                pHelmIvP:38:waypt_survey waypoints=1 
116.06017       WFLAG                pHelmIvP:148:waypt_survey waypoints=2 
74.65635        DEPLOY               pMarineViewer   true 
116.06017       WFLAG                pHelmIvP:148:waypt_survey waypoints=2 
140.11575       WFLAG                pHelmIvP:234:waypt_survey waypoints=3 
158.09377       WFLAG                pHelmIvP:298:waypt_survey waypoints=4 
176.15134       WFLAG                pHelmIvP:362:waypt_survey waypoints=5 
74.65635        DEPLOY               pMarineViewer   true 
176.15134       WFLAG                pHelmIvP:362:waypt_survey waypoints=5 
199.22812       WFLAG                pHelmIvP:444:waypt_survey waypoints=6 
230.22727       WFLAG                pHelmIvP:554:waypt_survey waypoints=7 
254.00302       WFLAG                pHelmIvP:640:waypt_survey waypoints=8 
  Total lines retained: 21 (0.05%)
  Total lines excluded: 44238 (99.95%)
  Total chars retained: 1435 (0.04%)
  Total chars excluded: 3873898 (99.96%)
    Variables retained: (2) DEPLOY, WFLAG
\end{consoleverb}
\vspace{0.1in}

\noindent
The last five lines provide a summary contained retained and
excluded. All lines above that are essentially a valid \var{.alog}
file. Note however that there are some duplicate lines, and a couple
lines out of order. To order the output and remove duplicates, try
running with the \var{--sort,-s} and \var{--duplicates,-d} options, or
simply the \var{-sd} option to do both:



\vspace{0.1in}
\begin{consoleverb}
$ aloggrep DEPLOY WFLAG alpha_4_1_2022_____18_33_46.alog -sd
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% LOG FILE:       ./alpha_4_1_2022_____18_33_46/alpha_4_1_2022_____18_33_46.alog
%% FILE OPENED ON  Wed Dec 31 19:00:00 1969
%% LOGSTART           41033480654.29403
26.28008        DEPLOY               pHelmIvP:HELM_VAR_INIT false 
74.65635        DEPLOY               pMarineViewer   true 
85.23170        WFLAG                pHelmIvP:38:waypt_survey waypoints=1 
116.06017       WFLAG                pHelmIvP:148:waypt_survey waypoints=2 
140.11575       WFLAG                pHelmIvP:234:waypt_survey waypoints=3 
158.09377       WFLAG                pHelmIvP:298:waypt_survey waypoints=4 
176.15134       WFLAG                pHelmIvP:362:waypt_survey waypoints=5 
199.22812       WFLAG                pHelmIvP:444:waypt_survey waypoints=6 
230.22727       WFLAG                pHelmIvP:554:waypt_survey waypoints=7 
254.00302       WFLAG                pHelmIvP:640:waypt_survey waypoints=8 
  Total re-sorts: 2
  Total lines retained: 10 (0.02%)
  Total lines excluded: 44238 (99.98%)
  Total chars retained: 723 (0.02%)
  Total chars excluded: 3873898 (99.98%)
    Variables retained: (2) DEPLOY, WFLAG
\end{consoleverb}
\vspace{0.1in}

\noindent
Notice the duplicates are removed, and the entries are ordered. Note also
that, with alog tools, the order of the arguments on the command line do
not matter. In the above example, the report section on the bottom now also
indicates how many lines needed to be re-sorted, in this case 2. 

%-----------------------------------------------------------------
\subsubsection{Creating a New ALog File}

Notice that the output above is not quite a syntactically valid
\var{.alog} file since the report lines at the end are not log
lines. So if output is simply re-directed to a file, the following
would produce an \var{.alog} file with essentially garbage at the end:

\vspace{0.1in}
\begin{consoleverb}
$ aloggrep DEPLOY WFLAG alpha_4_1_2022_____18_33_46.alog -sd &gt; newfile.alog
\end{consoleverb}
\vspace{0.1in}

\noindent
To create a new logfile either suppress the report lines at the end with:

\vspace{0.1in}
\begin{consoleverb}
$ aloggrep DEPLOY WFLAG oldfile.alog -sd --no_report &gt; newfile.alog
\end{consoleverb}
\vspace{0.1in}

\noindent
Or simply provide a second (new) file name and the new file will not include
the report lines:

\vspace{0.1in}
\begin{consoleverb}
$ aloggrep DEPLOY WFLAG alpha_4_1_2022_____18_33_46.alog -sd  newfile.alog
  Total re-sorts: 2
  Total lines retained: 10 (0.02%)
  Total lines excluded: 44238 (99.98%)
  Total chars retained: 723 (0.02%)
  Total chars excluded: 3873898 (99.98%)
    Variables retained: (2) DEPLOY, WFLAG
\end{consoleverb}
\vspace{0.1in}

\noindent
Note the report lines still go to the terminal, but the log files are
written to the new \var{.alog} file.

%-----------------------------------------------------------------
\subsubsection{Filtering based on the Data Source (Publishing App)}

Rather than naming variables to keep, \cmd{aloggrep} can be provided
with the name of one or more apps, and all entries published by these
apps will be retained. 

\vspace{0.1in}
\begin{consoleverb}
$ aloggrep pHelmIvP alpha_4_1_2022_____18_33_46.alog
\end{consoleverb}
\vspace{0.1in}

%-----------------------------------------------------------------
\subsubsection{Wildcard Matching}

Limited simple wildcard matching is supported:

\vspace{0.1in}
\begin{consoleverb}
$ aloggrep NAV_*  file.alog
\end{consoleverb}
\vspace{0.1in}

\noindent
The above will grab all variables beginning with \var{NAV\_}. Placing the
asterisk in any other position will have no effect, e.g., \var{*\_SPEED}
will not grap \var{NAV\_SPEED}.



%=================================================================
\subsection{Using aloggrep to Extract Plottable Data}

An additional use case for \cmd{aloggrep} involves getting data for
plotting. Using the same example as above, suppose we want to plot the
value of \mvar{WPT\_ODO} versus time. This variable represents vehicle
odometry data, published by the waypoint behavior, and reset to zero
on each waypoint.  Using the above arguments we get:

\vspace{0.1in}
\begin{consoleverb}
$ aloggrep alpha_4_1_2022_____18_33_46.alog WPT_ODO
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% LOG FILE:       ./alpha_4_1_2022_____18_33_46/alpha_4_1_2022_____18_33_46.alog
%% FILE OPENED ON  Wed Dec 31 19:00:00 1969
%% LOGSTART           41033480654.29403
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
74.83427        WPT_ODO        pHelmIvP:1:waypt_survey   0.00000      
75.90277        WPT_ODO        pHelmIvP:5:waypt_survey   0.34869      
76.20800        WPT_ODO        pHelmIvP:6:waypt_survey   0.57995      
76.49590        WPT_ODO        pHelmIvP:7:waypt_survey   0.88998      
76.77897        WPT_ODO        pHelmIvP:8:waypt_survey   1.42170      
77.08220        WPT_ODO        pHelmIvP:9:waypt_survey   2.79976      
77.39445        WPT_ODO        pHelmIvP:10:waypt_survey  3.53386      
77.67854        WPT_ODO        pHelmIvP:11:waypt_survey  4.38950      
77.95012        WPT_ODO        pHelmIvP:12:waypt_survey  5.40420   

...

  Total lines retained: 571 (1.29%)
  Total lines excluded: 43688 (98.71%)
  Total chars retained: 43305 (1.12%)
  Total chars excluded: 3832028 (98.88%)
    Variables retained: (1) WPT_ODO

\end{consoleverb}
\vspace{0.1in}

\noindent
The above output does grab the relevant lines, but we'd like to drop
the header lines and report at the end. Using the
\var{--format=time:val} of \var{--tv} option, the results are stripped
down to data lines only and just the time and value columns are
retained:

\vspace{0.1in}
\begin{consoleverb} 
$ aloggrep alpha_4_1_2022_____18_33_46.alog WPT_ODO --format=time:val 
74.83427 0.00000
75.90277 0.34869
76.20800 0.57995
76.49590 0.88998
76.77897 1.42170
77.08220 2.79976
77.39445 3.53386
77.67854 4.38950
77.95012 5.40420
...
\end{consoleverb}
\vspace{0.1in}

%-----------------------------------------------------------------
\subsubsection{Extracting Plottable Data from a Complex Posting}

Not all posted data is in simple numerical format as in case above
with the variable \mvar{WPT\_ODO}. Suppose for example, instead of
\mvar{WPT\_ODO}, our objective is to plot the value of the waypoint
index versus time. This value, in the Alpha mission, is published
as part of a string:

\vspace{0.1in}
\begin{consoleverb} 
$ aloggrep alpha_4_1_2022_____18_33_46.alog WFLAG --format=time:val 
85.23170 waypoints=1
116.06017 waypoints=2
140.11575 waypoints=3
158.09377 waypoints=4
176.15134 waypoints=5
199.22812 waypoints=6
230.22727 waypoints=7
254.00302 waypoints=8
\end{consoleverb}
\vspace{0.1in}


\vspace{0.1in}
\noindent
In this case, we would like to isolate the numerical value from the
string.  Using the \var{--subpat=PATTERN} option we can isolate
numerical values from variable postings made in the common format of:

\vspace{0.1in}

  \var{field1=value1, field2=value2, ..., fieldN=valueN}

\vspace{0.1in}

\vspace{0.1in}
\begin{consoleverb} 
$ aloggrep alpha_4_1_2022_____18_33_46.alog WFLAG --format=time:val --subpat=waypoints
85.23170 1
116.06017 2
140.11575 3
158.09377 4
176.15134 5
199.22812 6
230.22727 7
254.00302 8
\end{consoleverb}
\vspace{0.1in}

\noindent
Note in the above example, the column separator is single white space
character (ASCII 32). If a colon separator is desired, use the \var{--cso}
flag. If a comma separator is desired, use the \var{--csc} flag.


%=================================================================
\subsection{Extracting a First or Final Posting}

In some cases, \cmd{aloggrep} may be used as a tool within a script to
determine mission outcome. If the mission outcome is represented in a
single MOOS variable, e.g., \mvar{MISSION\_RESULT} or
\mvar{MISSION\_SCORE}, then the goal is to isolate just this value.
Using the \mvar{--first} or \mvar{--final} option, we can reduce the
output to just one line.  Using the example above:

\vspace{0.1in}
\begin{consoleverb} 
$ aloggrep alpha_4_1_2022_____18_33_46.alog WFLAG --format=time:val --final
254.00302 8
\end{consoleverb}
\vspace{0.1in}

\noindent
To isolate just the value column, use the \var{--format=val}, or \var{--v}
option:

\vspace{0.1in}
\begin{consoleverb} 
$ aloggrep alpha_4_1_2022_____18_33_46.alog WFLAG --v --final
8
\end{consoleverb}
\vspace{0.1in}

\noindent
The style of output can then be used within a shell script to take action
based on the result, stored in a shell variable:

\vspace{0.1in}
\begin{consoleverb} 
$ FOO=(`aloggrep alpha_4_1_2022_____18_33_46.alog WFLAG --v --final`)
$ echo $FOO
8
\end{consoleverb}
\vspace{0.1in}

\vspace{0.1in}
\begin{consoleverb} 
$ FOO=(`aloggrep alpha_10_5_2022_____14_31_06.alog MISSION_HASH --v --first`)
$ echo $MISSION_HASH
2211-0528V-SOUR-CLUB
\end{consoleverb}
\vspace{0.1in}


</pre></body></html>