<html><head><meta name="color-scheme" content="light dark"></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">\documentclass[onecolumn,letterpaper,11pt]{article}

%===================================================================
\input{common_latex/common_latex_packages}
\input{common_latex/common_latex_definitions}
\input{common_latex/common_latex_memo_format}
%===================================================================

\begin{document}

\begin{labcover}
\labtitle{Example Mission Alpha Obstacle}
\vspace{0.1in}
\labdate{January 2022}
\vspace{0.1in}
\labauth{Michael Benjamin, mikerb@mit.edu}
\labauth{Department of Mechanical Engineering}
\labauth{MIT, Cambridge MA 02139}
%\labtoc{small}
\end{labcover}

\wikidoc{file=Memo.VVMissionAlphaObs}
\wikidoc{auth=mikerb@mit.edu}
\wikidoc{toc=false}

%==================================================================
\section{The Alpha Obstacle Mission - Verifying Obstacle Avoidance}

In the \mis{Alpha Obstacle} mission, a single vehicle is traversing a
set of waypoints, twice, and encounters three obstacles that lie on
its path.  These obstacles are established in the mission plan with
known position and size, i.e., sensor processing is not being tested
here. The primary difference between this mission and the \mis{Alpha
  End Flag} mission is that the vehicle trajectory reflects the
influence of {\em two} behaviors simultaneously.  The Waypoint
behavior and the Avoid Obstacle behavior both need to be operating as
designed. Unlike testing in the previous mission, where a particular
flag posting event was monitored, in this mission the vehicle
trajectory is monitored to ensure that the vehicle does not hit, or
come unreasonably close to the obstacles.

\begin{figure}[H]
  \centering
  \includegraphics[width=0.85\textwidth]{figures/v2_alpha_obsavd_final.png}
  \caption{\mmis{The Alpha Obstacle Mission}: A vehicle traverse a set
    of three waypoints while periodically encountering an obstacle in
    its path. Both the Waypoint and AvoidObstacle behaviors are
    running and reaching a compromise trajectory using multi-objective
    optimization. The AvoidObstacle behavior is configured to publish
    a flag each time it encounters an obstacle. The flag consists of a
    MOOS variable value (\mvar{CPA} here), and a value representing
    the closest point of approach (CPA) on each obstacle
    encounter. The success of this mission depends on the posted CPA
    value never posting a value below a certain threshold, which would
    indicate a collision.}
  \label{fig_v2_alpha}
  \urlvideox{http://vimeo.com/538471620}{0:31}
\end{figure}

\noindent
This mission could be graded in a few ways: (a) a single pass/fail
result with a failure assigned if the CPA of any encounter is less
than N meters, (b) multiple pass/fail results, one for each encounter,
(c) a single grade based on the smallest CPA across all encounters, or
(d) multiple grades, one for each encounter. Furthermore, the grades
could be in raw units such as CPA in meters, or the grade could be
normalized to a range such as 0-100 with the test-designer choosing
which CPA ranges map to which grades.

\vspace{0.1in}
\noindent
A single pass/fail result is useful for triggering an alert to the
overall testing system. Think of a dashboard that shows daily testing
results and the user is looking to confirm all tests have passed. This
check can be accomplished by \app{pMissionEval} using the same
lead-check-post pattern as the \mis{Alpha} mission. The Waypoint behavior
posts to the variable \mvar{TOTAL\_WPTS} incrementing each time it
hits a waypoint. The AvoidObstacle behavior publishes to the variable
\mvar{CPA\_EVER}, holding the lowest CPA value for any encounter with
an obstacle. The lead-check-post configuraton for \app{pMissionEval} is:

\prevblue
\begin{verbatim}
  lead_condition = TOTAL_WPTS &gt; 10             // lead
  pass_condition = CPA_EVER &gt; 6                // check
  pass_flag      = MISSION_SUCCESS = true      // post
  fail_flag      = MISSION_SUCCESS = false     // post
\end{verbatim}
\postverb

\vspace{0.1in}
\noindent
On the other hand, a non-Boolean grading system is useful for
evaluating a trend, perhaps due to a bug, perhaps due to a change in
mission configuration perameters. It is also possible, that multiple
grades may be produced from a single mission. In the \mis{Alpha
  Obstacle} mission, there is only a single grade, but imagine the
path length of vehicle traversal were also being graded, shorting
being better. We would expect that changing the obstacle avoidance CPA
tolerance parameters would produce an improvement in one grade while
degrading the other. Automating successive tests with variations on
the parameter could help us better understand the tradeoff.

\vspace{0.1in}
\noindent
If the Waypoint behavior is augmented to post an \mvar{ODOMETRY}
value, a grading can be achieved with \app{pMissionEval} with the same
lead-check-post structure, with the grade being (a) the total odometry
distance and (b) the smallest CPA for all encounters:

\prevblue
\begin{verbatim}
  lead_condition = TOTAL_WPTS &gt; 10               // lead
  result_flag    = FINAL_CPA_EVER = $(CPA_EVER)  // post
  result_flag    = FINAL_ODOMETRY = $(ODOMETRY)  // post
\end{verbatim}
\postverb

\vspace{0.1in}
\noindent
In this case the lead-check-post structure has no pass/fail check
component. And since there is no notion of pass/fail, the post is not
based on a \pvar{pass\_flag} or \pvar{fail\_flag}, but the more neutral
\pvar{result\_flag}.

%------------------------------------------------------------------
\subsection*{{\em Alpha Obstacle} Mission Summary}

\begin{packed_itemize}
\item {\bf Evaluation Type}: This mission tests the operation of multiple
  components working together, in this case the Waypoint and AvoidObstacle
  behaviors compromising on a trajectory to satisfy both behaviors. 
  The evaluation type is {\em performance}.
\item {\bf Mission Structure}: This mission is comprised of a single
  vehicle running as a single MOOS community of apps, using a single
  MOOSDB. The mission structure is {\em singular}.
\item {\bf Machine Structure}: All apps in this mission are running on
  a single machine. Although the MOOS middleware supports distribution
  of singular missions across multiple machines, this is rare. The
  machine structure for this mission is {\em single machine}.
\item {\bf Evaluation Focus}: Since there is only one vehicle, the
  evaluation focus is on {\em ownship}.
\end{packed_itemize}

\end{document}



</pre></body></html>