<html><head><meta name="color-scheme" content="light dark"></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">\documentclass[onecolumn,letterpaper,11pt]{article}

%===================================================================
\input{common_latex/common_latex_packages}
\input{common_latex/common_latex_definitions}
\input{common_latex/common_latex_help_format}
%===================================================================

\begin{document}

\begin{labcover}
\labtitle{uDialogManager v5.0: User Interaction}
\vspace{0.1in}
\labdate{Spring 2019}
\vspace{0.1in}
\labauth{Michael Novitzky, novitzky@mit.edu}
\labauth{Department of Mechanical Engineering}
\labauth{MIT, Cambridge MA 02139}
\labtoc{small}
\end{labcover}

\wikidoc{file=PavLab.UDialogMgr}
\wikidoc{bugz=uDialogMgr}
\wikidoc{auth=novitzky@mit.edu}

%=================================================================
\section{uDialogManager v5.0: User Interaction }
%=================================================================
\label{sec_udialog_5_0}
\index{uDialogManager v 5.0}

The \app{uDialogManager} application is a module for interfacing with
a user.  It controls the user experience by confirming what the user
has asked for which helps reduce error.  Furthermore,
\app{uDialogManager} is responsible for interacting with other
applications such as triggering events or relaying information.  The
typical setup is the use of \app{uSpeechRec} for speech recognition
and \app{iSay} or text output for user feedback.

%\noindent
Key parts needed for \app{uDialogManager} are the possible syntax and
vocabulary combinations produced by speech recognition which are found
in the vocabulary file and grammar files for the Julius Speech
Recognition Engine.  These files and combinations are described in
\ref{sec_udialog_sentences}.

%====================================================================
\section{Using uDialogManager v5.0}
%======================================================================

Typical use of \app{uDialogManager} has it situated in a community in
which a human will interact with it using speech.  In addition to
\app{uDialogManager} another application in the same community must
interpret the user's speech such as \app{uSpeechRec} and have a method
of communicating back either through \app{iSay} or terminal output
such as through appcasting.

%======================================================================
\subsection{Typical Module Topology}

The typical module topology is shown in Figure \ref{fig_ud_topology} below. The
\app{uDialogManager} is situated in a community in which speech will be used as
a form of interaction. It is typically run alongside \app{uSpeechRec} and
\app{iSay} for an interactive experience. The \app{uDialogManager} application
subscribes to \mvar{SPEECH\_RECOGNITION\_SENTENCE} and
\mvar{SPEECH\_RECOGNITION\_SCORE} and minimally publishes
\mvar{SPEECH\_COMMANDED}, \mvar{SAY\_MOOS}. As discussed further in the
document, other variables and their values can be triggered by a speech
recognition sentence. The \mvar{SPEECH\_COMMANDED} variable is used as a logging
tool for speech commands that have been acknowledged by the user. The
\mvar{SAY\_MOOS} variable is published as a means to use \app{iSay} to give the
user auditory feedback.


\vspace{0.1in}
\begin{figure}[H]
  \centering
  \includegraphics[width=0.7\textwidth]{figures/aq_topology.png}
  \caption{{\bf Typical uDialogManager Topology:} This module runs in
    any community in which one would like to use Speech Recognition.
    It is typically used with the applications \app{uSpeechRec} and
    \app{iSay} for an interactive experience. It subscribes to
    \mvar{SPEECH\_RECOGNITION\_SENTENCE} and \mvar{SPEECH\_RECOGNITION\_SCORE} which comes from
    \app{uSpeechRec} and publishes \mvar{SAY\_MOOS} for audio output
    and user specified variables based on speech sentences.}
\label{fig_ud_topology}
\end{figure}

%--------------------------------------------------------------------
\subsection{The States of uDialogManager}
\label{sec_udialog_states}
\app{uDialogManager} can be in one of several states: \var{Waiting for
  Command}, \var{Command Received}, \var{Waiting for Acknowledgement},
and \var{Acknowledgement Received}. These states are common among
dialog managers as they control the flow of interaction.

%======================================================================
\subsection{Available Sentences}
\label{sec_udialog_sentences}
The available sentences that can be acted upon by \app{uDialogManager}
are deliverd by \app{uSpeechRec}. The \app{uSpeechRec} uses a grammar
and vocabulary file to determine the sentence strucuture and
vocabulary available to the Julius Speech Recognition system.  At the
moment these are the available sentences:

\vspace{0.15in}
\begin{fileverb} 
NAME COMMAND
ACK
\end{fileverb}
\vspace{0.15in}

\noindent These are the available vocabulary words:

\vspace{0.15in}
\begin{fileverb} 
NAME: Arnold, Betty, Charlie, Davis, Evan, Gus
COMMAND: RETURN, FOLLOW, STATION
ACK: Yes, No
\end{fileverb}
\vspace{0.15in}

%======================================================================
\subsection{Sentence Action}
\label{sec_udialog_sentence_action}

A major feature of \app{uDialogManager\_5\_0} is that a user can
specify the variable-value pairs that trigger based on an acknowledged
speech recognition sentence.  Remember that only sentences provided
through \mvar{SPEECH\_RECOGNITION\_SENTENCE} or
\mvar{SPEECH\_RECOGNITION\_SCORE} are acted upon.

\vspace{0.1in}
\noindent
Let's go through an example from a .moos file as follows:

\vspace{0.15in}
\begin{fileverb} 
	 sentence = Arnold_Deploy : DEPLOY = true
\end{fileverb}
\vspace{0.15in}

\noindent We see that a sentence action is defined with the word
\var{sentence} first followed by an equal sign to inidcate the speech
recognition sentence that will trigger the following var-value
pairs. Notice that the words in the speech recogntion sentence are
separated by an underscore instead of spaces.  The end of the speech
recognition sentence is inidicated by a semi-colon ':'.  Following the
semi-colon is the variable name \mvar{DEPLOY} followed by an equals
sign and the value \mvar{true}.  In this example, once the user
acknowledges the speech recognition sentence \mvar{ARNOLD DEPLOY},
\app{uDialogManager\_5\_0} will publish \mvar{DEPLOY}=true.

Now let's look at an example that published multiple variable-value
pairs. In the following example, multiple variable-value pairs are
published once triggered by a speech recognition sentence.

\vspace{0.15in}
\begin{fileverb} 
 	 sentence = Arnold_Deploy : DEPLOY = true + MOOS_MANUAL_OVERRIDE = false //
                                   + RETURN = false
\end{fileverb}
\vspace{0.15in}

\noindent In the example above the variable-value pairs are separated
by a plus '+' sign. In this case, \mvar{DEPLOY} is assigned 'true',
\mvar{MOOS\_MANUAL\_OVERRIDE} is assigned 'false', and \mvar{RETURN}
will be assigned 'false' when the speech recognition sentence
\mvar{ARNOLD DEPLOY} is acknowledged.


In the final example we will introduce the method in which we send
variables to another community and can encapsulate a string with
quotations.  In order to send variables to another community we
publish to \mvar{NODE\_MESSAGE\_LOCAL} which bridges to the shoreside
community through \app{uFldNodeBroker}.  This variable is then parsed
on the shoreside community and sent to the proper MOOS community based
on the \mvar{dest\_node} community name. Once at the target MOOS
community it is posted locally the the \mvar{var\_name} with the value
indicated by \mvar{string\_val}.

\vspace{0.15in}
\begin{fileverb} 
  sentence = Arnold_Follow : NODE_MESSAGE_LOCAL = //
             "src_node=mokai,dest_node=betty,var_name=TRAIL,string_val=true"
\end{fileverb}
\vspace{0.15in}

\noindent
We can see that the \mvar{NODE\_MESSAGE\_LOCAL} takes as input a
string with many variable-value pairs within. To make this possible we
will surround the string with quotes ``.

%======================================================================
\subsection{Sentence Acknowledgement}

A major feature with \app{uDialogManager\_5\_0} is the
ability to define how a sentence is acknowledged before the
variable-value pairs are posted to the MOOSDB. The three options for
acknowledgement are 1) the default of replying Yes/No for
Confirmation/Decline 2) user specified Confirmation/Decline words or
3) no confirmation in which once a sentence is recognized, the
variable-value pairs are posted immediately.  Let us look at a simple
default example below:

\vspace{0.15in}
\begin{fileverb} 
  sentence = Arnold_Follow : FOLLOW_ARNOLD = true
\end{fileverb}
\vspace{0.15in}

\noindent
In this example the sentence "Arnold Follow" has the variable-value
pair of \mvar{FOLLOW\_ARNOLD} set to \var{true}. By default,
\app{uDialogManager\_5\_0} will ask the user "Did you mean, Arnold
Follow?"  At this point, the user can confirm the speech command by
replying "Yes" or decline the speech command by replying "No."
Responding with anything other than "Yes" or "No" will result in
\app{uDialogManager\_5\_0} responding with "Command Canceled" "Wrong
Ack".

A user can specify different words for confirming or declining a
command using the following format:

\vspace{0.15in}
\begin{fileverb} 
  sentence = Arnold_Follow { CONFIRM=Verify | DECLINE=Ignore } : FOLLOW_ARNOLD = true
\end{fileverb}
\vspace{0.15in}

\noindent
In the above example, the user has specified acknowledgement options
between the curly braces, '{' and '}'.  The user has specified the
word \var{Verify} for confirming a command and the word \var{Ignore}
for declining a command.  In this example, the user says "Arnold
Follow" and the \app{uDialogManager\_5\_0} will prompt the user with
"Did you mean, Arnold Follow?."  The user can confirm with replying
"Verify" or decline the command with replying "Ignore."  If the
user replies with anything other than Verify or Ignore, the
\app{uDialogManager\_5\_0} will respond with "Command Canceled, Wrong
Ack."


When the user wants a command to simply trigger posting variable-value
pairs without acknowledgement they can specify this with a NOCONFIRM option:

\vspace{0.15in}
\begin{fileverb} 
  sentence = Arnold_Follow { NOCONFIRM }: FOLLOW_ARNOLD = true
\end{fileverb}
\vspace{0.15in}

\noindent As in the previous example, the acknowledgement options are
between the curly braces, '{' and '}'.  In this case, the user wants
the variable-value pairs to post to the MOOSDB as soon as
\app{uSpeechRec} recognizes the sentence "Arnold Follow."  Users are
cautioned when using this option as false positives in speech
recognition can lead to unwanted behavior.



%======================================================================
\subsection{Vehicle Nicknames}
\label{sec_udialog_nick_name}

Specifying nicknames between what a vehicle is called and its lab
(community) name is no longer an option (since v2.0).  Instead, the
destination of a \mvar{NODE\_MESSAGE\_LOCAL} can have the vehicle's name while
the speech sentence can have the name verbalized by the user.

%======================================================================
\subsection{Using Wave Files Instead of TTS}
\label{sec_udialog_wave_files}

\app{uDialogManager} can either send feedback using text-to-speech (TTS) or replaying
pre recorded wave files. Specifying the \mvar{use\_wav\_files} to either 'YES' or
'NO' sets this option.  If set to yes and the wave files do not exist, then
\app{iSay} will post an error.

%======================================================================
\subsection{Rejecting Sentences with Word Confidence Scores}
\label{sec_udialog_confidence_scores}

Originally, \app{uDialogManager} accepted whatever the most likely sentence that the
Julius Speech Recognition Engine delivered. However, this method still had a
high error rate. Specifying the \mvar{confidence\_thresh} parameter (range of
(0.0,1]) in the .moos
file switches \app{uDialogManager} into using a threshold on the word confidence
scores produced by the Julius Speech Recognition Engine. In general, a value of
0.7 works well. It is up to the user to experiment to determine which word
confidence rejection threshold works best for their scenario.

When \app{uDialogManager} rejects a sentence because of low word confidence
scores, it will say the phrase ``SAY AGAIN'' to indicate your last sentence was rejected.

%======================================================================
\section{Configuration Parameters of uDialogManager}
%======================================================================
\index{uDialogManager!Configuration Parameters}
\index{Configuration Parameters!uDialogManager}

The following parameter is defined for \app{uDialogManager\_5\_0}. A more
detailed description is provided in other parts of this
section. Parameters having default values are indicated so.


\vspace{0.15in}
\begin{listing}{Configuration Parameters for uDialogManager\_5\_0}
  \label{lst_uDialogManager_params}
  \renewcommand{\arraystretch}{1.1}
  
  \begin{longtable}{rp{0.85\textwidth}} 
  \pvar{confidence\_thresh}: &amp; Can reject sentences based on word confidence
  scores. Threshold is in range (0,1.0].  If not specified, automatically
  accepts the most likely sentence without consideration of word confidence scores.  Section
    \ref{sec_udialog_confidence_scores}. \\

    \pvar{use\_wav\_files}: &amp; indicate whether to use local text-to-speech (TTS)
    or pre-recorded wave files. Options are yes or no.  Section
    \ref{sec_udialog_wave_files}. \\   

    \pvar{sentence}: &amp; Assignment of an incoming speech recognition
    sentence to trigger a set of variable value pairs.  Section
    \ref{sec_udialog_nick_name}.    
  \end{longtable}
\end{listing}

%======================================================================
\subsection{An Example MOOS Configuration Block}

To see an example MOOS configuration block, enter the following from
the command-line:

\vspace{0.1in}
\begin{consoleverb}
 $ uDialogManager --example or -e
\end{consoleverb}
\vspace{0.05in}

\noindent
This will show the output shown in Listing \ref{lst_uDialogManager_econfig} below.

\vspace{0.25in}
\begin{listing}{Example configuration of the uDialogManager application.}
  \label{lst_uDialogManager_econfig}
  
\footnotesize
\begin{verbatim}
   1  =============================================================== 
   2  uDialogManager Example MOOS Configuration                     
   3  =============================================================== 
   4                                                                  
   5  ProcessConfig = uDialogManager                                
   6  {                                                               
   7    AppTick   = 4                                                 
   8    CommsTick = 4                                                 
   9
  10    //can reject sentences based on word confidence scores        
  11    //threshold value range (0,-1.0]                              
  12    //if not specified, reverts to accepting most likely sentence    
  13    //without considering word confidence 
  14    confidence_thresh = 0.7                                       
  15  
  16    //indicate whether to use local text-to-speech (TTS) or       
  17    //pre-recorded wave files. Options are yes or no              
  18    Use_Wav_Files=Yes                             
  19
  20    //list of vars and values to publish given speech sentence
  21    //var-value pairs are ''+' separated
  22    sentence = Arnold_Deploy : DEPLOY = true + MOOS_MANUAL_OVERRIDE = false + RETURN = false
  23
  24    //quotes around a string for a value can be used
  25    sentence = Arnold_Follow : NODE_MESSAGE_LOCAL = 
  26                               "src_node=mokai,dest_node=betty,var_name=TRAIL,string_val=true"
  27
  28    //sentence ack options can be specified after a sentence inside of curly braces '{' and '}'
  29    //in order to skip acknowledgment and variable-value pairs post immediately
  30    sentence = grab { NOCONFIRM } : FLAG_GRAB_REQUEST = ``vname = $(VNAME)''
  31    
  32    //the user can specify words for confirming or declining commands within the curly braces
  33    sentence = tag { CONFIRM=verify | DECLINE=ignore }  : TAG_REQUEST = ``vname = $(VNAME)''
  34  }                                                          
\end{verbatim}
\normalsize
\end{listing}
 
%======================================================================
\section{Publications and Subscriptions for uDialogManager v5.0}
%======================================================================
\index{uDialogManager!Publications and Subscriptions}
\index{Publications and Subscriptions!uDialogManager}
\label{sec_uDialogManager_interface}

The interface for \app{uDialogManager}, in terms of publications and
subscriptions, is described below. This same information may also be
obtained from the terminal with:

\vspace{0.1in}
\begin{consoleverb}
 $ uDialogManager --interface or -i
\end{consoleverb}
\vspace{0.1in}

\subsection{Variables Published by uDialogManager}
\label{sec_uDialogManager_pubs}

\begin{packed_itemize}[2]

\item \mvar{APPCAST}: Contains an appcast report identical to the
  terminal output. Appcasts are posted only after an appcast request
  is received from an appcast viewing utility.

\item \mvar{SPEECH\_COMMANDED}: A sentence that has been acknowledged by a user.

\item \mvar{DIALOG\_ERROR}: posted when a sentence is rejected using word
  confidence score threshold.

\item \mvar{SAY\_MOOS}: Either a wave file to be played for the user or a 
  sentence to be uttered by \app{iSay}.
  
\end{packed_itemize}

%======================================================================
\subsection{Variables Subscribed for by uDialogManager}
\label{sec_uDialogManager_subs}

The \app{uDialogManager} application will subscribe for the following
four MOOS variables:

\begin{packed_itemize}
\item \mvar{APPCAST\_REQ}: A request to generate and post a new
  apppcast report, with reporting criteria, and expiration. 
\item \mvar{SPEECH\_RECOGNITION\_SENTENCE}: The most likely sentence recognized by 
the Julius Speech Recognition Engine.
\item \mvar{SPEECH\_RECOGNITION\_SCORE}: Includes the most likely sentence and
  word confidence scores produced by the Julius Speech Recognition Engine. 
\end{packed_itemize}

%======================================================================
\subsection{Command Line Usage of uDialogManager}
%======================================================================
\label{sec_uDialogManager_cmdline}
\index{Command Line Usage!uDialogManager}
\index{uDialogManager!Command Line Usage}

The \app{uDialogManager\_5\_0} application is typically launched as a part
of a batch of processes by \app{pAntler}, but may also be launched from the
command line by the user.  To see command-line options enter the
following from the command-line:

\vspace{0.1in}
\begin{consoleverb}
 $ uDialogManager --help or -h
\end{consoleverb}
\vspace{0.1in}

\noindent
This will show the output shown in Listing \ref{lst_uDialogManager_usage} below.

\vspace{0.25in}
\begin{listing}{Command line usage for uDialogManager}
\label{lst_uDialogManager_usage}
\footnotesize
\begin{verbatim}
   1  ==========================================================
   2  Usage: uDialogManager file.moos [OPTIONS]               
   3  ==========================================================
   4                                                            
   5  Options:                                                  
   6    --alias=&lt;ProcessName&gt;                                
   7        Launch uFldHazardMetric with the given process name. 
   8    --example, -e                                           
   9        Display example MOOS configuration block.           
  10    --help, -h                                              
  11        Display this help message.                          
  12    --interface, -i                                         
  13        Display MOOS publications and subscriptions.        
  14    --version,-v                                            
  15        Display release version of uDialogManager.        
\end{verbatim}
\normalsize
\end{listing}

%======================================================================
\section{Terminal and AppCast Output}
%======================================================================
\label{sec_uDialogManager_appcast}

\vspace{0.1in}
\begin{listing}{Example uDialogManager console output}
\label{lst_uDialogManager_appcast}
\footnotesize
\begin{verbatim}
   1  ===================================================================
   2  uDialogManager_3_0 mokai                                       0/0(655)
   3  ===================================================================
   4
   5  Sentence Action: ARNOLD_DEPLOY : DEPLOY=true + MOOS_MANUAL_OVERRIDE=false + RETURN=false
   6
   7  Sentence Action: ARNOLD FOLLOW : NODE_MESSAGE_LOCAL=src_node=mokai,
   8                                   dest_node=betty,var_name=RETURN,string_val=true
   9
  10    sentence = grab { NOCONFIRM } : FLAG_GRAB_REQUEST = "vname = $(VNAME)"
  11    
  12    sentence = tag { CONFIRM=verify | DECLINE=ignore }  : TAG_REQUEST = "vname = $(VNAME)"
  13
  14
  15  CURRENT STATE:
  16  Ready for Command.
  17  
  18  CONVERSATIONS:
  19
  20  DM: Command Sent
  21  User: Yes
  22  DM: Did you mean arnold follow
  23  User: Arnold Follow
  24
\end{verbatim}
\normalsize
\end{listing}

\noindent 
Line 5 shows a sentence action that was defined in the .moos file.  In
this case, the speech recognition sentence of ARNOLD DEPLOY triggers
the three variables \mvar{DEPLOY}, \mvar{MOOS\_MANUAL\_OVERRIDE}, and
\mvar{RETURN} to be published with the values true, false, and false,
respectively. Line 7 shows another sentence action that was defined
but in this case the value for \mvar{NODE\_MESSAGE\_LOCAL} is a
string.  As described above, \mvar{NODE\_MESSAGE\_LOCAL} allows a
local variable to be published that then contains a var/value pair to
be sent to another community.  Line 10 shows a sentence that skips the
acknowledgement step with the NOCONFIRM option.  Line 12 shows a
sentence that has changed the default words of Yes and No to Confirm
and Decline a command to Verify and Ignore.

\vspace{0.1in}
\noindent
Line 15 displays the \var{CURRENT STATE} of \app{uDialogManager},
which can be in one of several states such as \var{Ready for Command}
or \var{Waiting for an ACK}.  Line 20 begins the display of the last
10 sentences between the user and the \app{uDialogManager} with the
most recent sentence at the top. In this example, the user started by
saying "Arnold Follow" which is displayed at Line 23.  The
\app{uDialogManager} responds with "Did you mean arnold follow"
which is shown in Line 22.  Lines 21 and 20 demonstrate that the user
acknowledged with a "Yes" and then \app{uDialogManager} responded
with "Command Sent." If there is a configuration or runtime warning
then Line 4 would be replaced with \var{.moos} file issues.

\end{document}

</pre></body></html>