<html><head><meta name="color-scheme" content="light dark"></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">\documentclass[onecolumn,letterpaper,11pt]{article}

%===================================================================
\input{common_latex/common_latex_packages}
\input{common_latex/common_latex_definitions}
\input{common_latex/common_latex_memo_format}
%===================================================================

\begin{document}

\begin{labcover}
\labtitle{Example Mission Berta Collision Avoid}
\vspace{0.1in}
\labdate{January 2022}
\vspace{0.1in}
\labauth{Michael Benjamin, mikerb@mit.edu}
\labauth{Department of Mechanical Engineering}
\labauth{MIT, Cambridge MA 02139}
%\labtoc{small}
\end{labcover}

\wikidoc{file=Memo.VVMissionBertaCollAvd}
\wikidoc{auth=mikerb@mit.edu}
\wikidoc{toc=false}

%==================================================================
\section{The Berta Collision Avoidance Mission - Verifying Collision Avoidance}

In the {\em Berta Collision Avoidance} mission, a single vehicle is
traversing an east-west course while some number of vehicles (1-7) are
launched and will traverse a north-south pattern, crossing the
east-west vehicle. This mission is substantially more complex than the
previous missions primarily because the the simulation involves
simulation of several distinct vehicles, each with their own autonomy.
This test environment uses the {\em uField Toolbox} described in
Section \ref{sec_ufield}. The mission, when executed, looks something
like that shown in Figure \ref{fig_v3_berta}.

\begin{figure}[H]
  \centering
  \includegraphics[width=0.95\textwidth]{figures/v3_berta_colavd_final.png}
  \caption{{\em The Berta Collision Avoidance Mission}: A vehicle
    repeatedly traverses an east-west course while crossing several
    lanes of north-south traffic. Although the north-south vehicle
    also have collision avoidance enabled between each other, the
    primary metric gathered is the collision avoidance performance of
    the east-west vehicle.}
  \label{fig_v3_berta}
  \urlvideox{http://vimeo.com/539424983}{2:42}
\end{figure}

\vspace{0.1in}
\noindent
Even with no randomization on the mission parameters or start
conditions, this mission will quickly evolve to a unique set of
vehicle encounters. A seven vehicle mission would involve about 60
unique applications sharing information. The timing is affected in
subtle ways by the host operating system handling the application
scheduling and inter-process communication. This of course also
happens in the real world, with effects of the environment and sensor
noise, so it is a welcome aspect of the simulation.  But it does mean
that, unlike the previous two mission, a failure event will be hard to
recreate on a subsequent run. The collection of detailed log files for
all vehicles is critical for debugging a failure. Bug fixes will be
hard to confirm since the exact conditions leading up to and at the
time of failure event will be hard to reproduce.

\vspace{0.1in}
\noindent
Although this mission can be run, at low time warp, on a sufficiently
powerful single computer, the launch script and mission files are
structured to also allow each vehicle to be launched on separate
computers on the same network.

\vspace{0.1in}
\noindent
As with the \mis{Alpha Obstacle} mission, this mission can be graded
on a pass/fail basis or on a performance basis. Its also worth noting
that the \mis{Alpha Obstacle} mission relied on the AvoidObstacle
behavior to report the CPA with each obstacle encounter. On some level
this is less than desirable to rely on the very module being tested,
to produce the data upon which it is graded. In the uField mission
structure, there is a separate community of software, separate from
each of the vehicles. This community is primarily used for
coodination, but also supports the creation of evaluation software to
be run, distinct from the vehicles, to provide a grade or pass/fail
result.

\vspace{0.1in}
\noindent
In the {\em Berta Collision} mission, an app called
\app{uFldCollisionDetect}, that monitors the positions of all vehicles
and assesses each encounter within a certain range. Each encounter can
categorized as simply an {\em encounter}, a {\em near miss}, or {\em
  collision}. The ranges for each category are configuration
parameters for \app{uFldCollisionDetect} and selected by the mission
designer responsible for establishing quality control thresholds.


\vspace{0.1in}
\noindent
The table below conveys the type of data that may be collected in this
mission where there are five contacts, {\em ben, cal, deb, eve,
  fin}. The only vehicle graded is the single east-west vehicle, {\em
  abe}. The total encounters, near misses and collisions in the table
below are all with respect to {\em abe}. In other words, encounters
between each of the contacts are not represented. Conceivably in other
missions all interactions could be part of the grading process too.

\vspace{0.1in}
\noindent
The following table shows an example of the types of data that may be
collected in a mission where ownship has four contacts in the
vicinitiy with many encounters. The set of data leaves many options
for how to grade the performance of ownship.

\vspace{0.1in}

\small
\begin{longtable}{|l|c|c|c|c|c|} \hline
  \bsvar{Contact Name} &amp; \bsvar{Encounters} &amp; \bsvar{Near Misses} &amp; \bsvar{Collisions} &amp; \bsvar{Min CPA} &amp; \bsvar{Avg CPA} \\ \hline
  ben          &amp; 128        &amp; 6           &amp; 1          &amp; 4.5     &amp; 11.2  \\ \hline
  cal          &amp; 68         &amp; 3           &amp; 1          &amp; 5.5     &amp; 14.0  \\ \hline
  deb          &amp; 74         &amp; 0           &amp; 0          &amp; 6.2     &amp; 9.8   \\ \hline
  eve          &amp; 108        &amp; 6           &amp; 2          &amp; 5.9     &amp; 9.3   \\ \hline
  fin          &amp; 60         &amp; 3           &amp; 0          &amp; 7.4     &amp; 12.2  \\ \hline \hline
  all          &amp; 438        &amp; 18          &amp; 4          &amp; 4.5     &amp; 9.56  \\ \hline
\end{longtable}
\normalsize

\vspace{0.1in}
\noindent
Given the above data, there are a number of options for grading this
mission. It could be argued that a single pass/fail result should be
used with a fail occuring if there are any collisions. This is fair,
and there may be such a mission structure and evaluation type as part
of an acceptance test, since acceptance tests tend to be pass/fail in
nature.

\vspace{0.1in}
\noindent
A non pass/fail grade may also be appropriate, even in an acceptance
test. Acceptance criteria may allow for a certain non-zero failure
rate. This mission has many configuration options for things like
vehicle speed, vehicle dynamics, sensor range, and the priority of the
primary mission (in this case, efficient east-west transits). An acceptance
test may have a zero-tolerance for collisions with a baseline configuration,
but a separate tolerance for other conditions, such as high-speed contacts
with lower sensor performance.

\vspace{0.1in}
\noindent
As with the first two missions, this mission will use the lead-check-post
structure of \app{pMissionEval}:

\prevblue
\begin{verbatim}
  lead_condition = ENCOUNTERS &gt; 400
  pass_condition = (COLLISIONS = 0) and (NEAR_MISSES &lt; 20)
  pass_flag      = MISSION_SUCCESS = true
  fail_flag      = MISSION_SUCCESS = false
\end{verbatim}
\postverb

%------------------------------------------------------------------
\subsection*{{\em Berta Collision Avoid} Mission Summary}

\begin{packed_itemize}
\item {\bf Evaluation Type}: This mission tests the operation of
  multiple components working together, in this case the Loiter and
  COLREGS collision avoidance behaviors compromising on a trajectory
  to satisfy both behaviors.  The evaluation type is {\em
    performance}.
\item {\bf Mission Structure}: This mission is comprised of multiple
  vehicles each comprised of single MOOS community of apps,
  coordinated with a shoreside MOOS community. The mission structure
  is {\em uField}.  
\item {\bf Machine Structure}: All apps in this mission are running on
  a single machine. Although the MOOS middleware supports distribution
  of singular missions across multiple machines, this is rare. The
  machine structure for this mission is {\em single machine}.
\item {\bf Evaluation Focus}: Since there is only one vehicle graded,
  the evaluation focus is on {\em ownship}.
\end{packed_itemize}

\end{document}



</pre></body></html>