moos-ivp.org IvP ManPages 2.680 LAMSS PavLab Herons NOWISTHETIMEFORALLGOODMENTOCOMETOTHE NOWISTHETIMEO NOW Edit

3 uDialogManager v2.0: User Interaction

Maintained by: novitzky@mit.edu

3.1 Overview
3.2 Using uDialogManager v2.0
     3.2.1 The States of uDialogManager
     3.2.2 Available Sentences
     3.2.3 Sentence Action
     3.2.4 Vehicle Nicknames
3.3 Configuration Parameters of uDialogManager
3.4 Publications and Subscriptions for uDialogManager v2.0
     3.4.1 Variables Published by uDialogManager
     3.4.2 Variables Subscribed for by uDialogManager
3.5 Terminal and AppCast Output

3.1 Overview [top]

The uDialogManager application is a module for interfacing with a user. It controls the user experience by confirming what the user has asked for which helps reduce error. Furthermore, uDialogManager is responsible for interacting with other applications such as triggering events or relaying information. The typical setup is the use of uSpeechRec for speech recognition and iSay or text output for user feedback.

Key parts needed for uDialogManager are the possible syntax and vocabulary combinations produced by speech recognition which are found in the vocabulary file and grammar files for the Julius Speech Recognition Engine. These files and combinations are described in 1.2.2 .

3.2 Using uDialogManager v2.0 [top]

Typical use of uDialogManager has it situated in a community in which a human will interact with it using speech. In addition to uDialogManager another application in the same community must interpret the user's speech such as uSpeechRec and have a method of communicating back either through iSay or terminal output such as through appcasting.

Typical Module Topology [top]

The typical module topology is shown in Figure 3.1 below. The uDialogManager is situated in a community in which speech will be used as a form of interaction. It is typically run alongside uSpeechRec and iSay for an interactive experience. The uDialogManager application subscribes to SPEECH_RECOGNITION_SENTENCE and minimally publishes SPEECH_COMMANDED, SAY_MOOS. As discussed further in the document, other variables and their values can be triggered by a speech recognition sentence. The SPEECH_COMMANDED variable is used as a logging tool for speech commands that have been acknowledged by the user. The SAY_MOOS variable is published as a means to use iSay to give the user auditory feedback.

Figure 3.1: Typical uDialogManager Topology: This module runs in any community in which one would like to use Speech Recognition. It is typically used with the applications uSpeechRec and iSay for an interactive experience. It subscribes to SPEECH_RECOGNITION_SENTENCE which comes from uSpeechRec and publishes SAY_MOOS for audio output and user specified variables based on speech sentences.

3.2.1 The States of uDialogManager [top]

uDialogManager can be in one of several states: Waiting for Command, Command Received, Waiting for Acknowledgement, and Acknowledgement Received. These states are common among dialog managers as they control the flow of interaction.

3.2.2 Available Sentences [top]

The available sentences that can be acted upon by uDialogManager are deliverd by uSpeechRec. The uSpeechRec uses a grammar and vocabulary file to determine the sentence strucuture and vocabulary available to the Julius Speech Recognition system. At the moment these are the available sentences:

 NAME COMMAND
 ACK

These are the available vocabulary words:

 NAME: Arnold, Betty, Charlie, Davis, Evan, Gus
 COMMAND: RETURN, FOLLOW, STATION
 ACK: Yes or No

3.2.3 Sentence Action [top]

A major feature of uDialogManager_2_0 is that a user can specify the variable-value pairs that trigger based on an acknowledged speech recognition sentence. Remember that only sentences provided through SPEECH_RECOGNITION_SENTENCE are acted upon.

Let's go through an example from a .moos file:

 	 sentence = Arnold_Deploy : DEPLOY = true

We see that a sentence action is defined with the word sentence first followed by an equal sign to inidcate the speech recognition sentence that will trigger the following var-value pairs. Notice that the words in the speech recogntion sentence are separated by an underscore instead of spaces. The end of the speech recognition sentence is inidicated by a semi-colon ':'. Following the semi-colon is the variable name DEPLOY followed by an equals sign and the value true. In this example, once the user acknowledges the speech recognition sentecne ARNOLD DEPLOY, uDialogManager_2_0 will publish DEPLOY=true.

Now let's look at an example that published multiple variable-value pairs. In the following example, multiple variable-value pairs are published once triggered by a speech recognition sentence.

  	 sentence = Arnold_Deploy : DEPLOY = true + MOOS_MANUAL_OVERRIDE = false //
                                    + RETURN = false

In the example above the variable-value pairs are separated by a plus '+' sign. In this case, DEPLOY is assigned 'true', MOOS_MANUAL_OVERRIDE is assigned 'false', and RETURN will be assigned 'false' when the speech recognition sentence ARNOLD DEPLOY is acknowledged.

In the final example we will introduce the method in which we send variables to another community and can encapsulate a string with quotations. In order to send variables to another community we publish to NODE_MESSAGE_LOCAL which bridges to the shoreside community through uFldNodeBroker. This variable is then parsed on the shoreside community and sent to the proper MOOS community based on the dest_node community name. Once at the target MOOS community it is posted locally the the var_name with the value indicated by string_val.

   sentence = Arnold_Follow : NODE_MESSAGE_LOCAL = //
              "src_node=mokai,dest_node=betty,var_name=TRAIL,string_val=true"

We can see that the NODE_MESSAGE_LOCAL takes as input a string with many variable-value pairs within. To make this possible we will surround the string with quoates ``.

3.2.4 Vehicle Nicknames [top]

Specifying nicknames between what a vehicle is called and its lab (community) name is no longer an option in v2.0. Instead, the destination of a NODE_MESSAGE_LOCAL can have the vehicle's name while the speech sentence can have the name verbalized by the user.

3.3 Configuration Parameters of uDialogManager [top]

The following parameter is defined for uDialogManager_2_0. A more detailed description is provided in other parts of this section. Parameters having default values are indicated so.

{Listing 3.1: Configuration Parameters for uDialogManager_2_0.}

sentence: Assignment of an incoming speech recognition sentence to trigger a set of variable value pairs. Section 3.2.5 .

An Example MOOS Configuration Block [top]

To see an example MOOS configuration block, enter the following from the command-line:

  $ uDialogManager --example or -e

This will show the output shown in Listing 2 below.

{Listing 2 - Example configuration of the uDialogManager application.}

    1  =============================================================== 
    2  uDialogManager Example MOOS Configuration                     
    3  =============================================================== 
    4                                                                  
    5  ProcessConfig = uDialogManager                                
    6  {                                                               
    7    AppTick   = 4                                                 
    8    CommsTick = 4                                                 
    9                          
   10    //list of vars and values to publish given speech sentence
   11    //var-value pairs are '+' separated
   12    sentence = Arnold_Deploy : DEPLOY = true + MOOS_MANUAL_OVERRIDE = false + RETURN = false
   13
   14    //quotes around a string for a value can be used
   15    sentence = Arnold_Follow : NODE_MESSAGE_LOCAL = 
   16                               "src_node=mokai,dest_node=betty,var_name=TRAIL,string_val=true"                                
   17  }

3.4 Publications and Subscriptions for uDialogManager v2.0 [top]

The interface for uDialogManager, in terms of publications and subscriptions, is described below. This same information may also be obtained from the terminal with:

  $ uDialogManager --interface or -i

3.4.1 Variables Published by uDialogManager [top]

APPCAST: Contains an appcast report identical to the terminal output. Appcasts are posted only after an appcast request is received from an appcast viewing utility.

SPEECH_COMMANDED: A sentence that has been acknowledged by a user.

SAY_MOOS: Either a wave file to be played for the user or a sentence to be uttered by iSay.

3.4.2 Variables Subscribed for by uDialogManager [top]

The uDialogManager application will subscribe for the following four MOOS variables:

APPCAST_REQ: A request to generate and post a new apppcast report, with reporting criteria, and expiration.
SPEECH_RECOGNITION_SENTENCE: The most likely sentence recognized by the Julius Speech Recognition Engine.

Command Line Usage of uDialogManager [top]

The uDialogManager_2_0 application is typically launched as a part of a batch of processes by pAntler, but may also be launched from the command line by the user. To see command-line options enter the following from the command-line:

  $ uDialogManager --help or -h

This will show the output shown in Listing 3 below.

{Listing 3 - Command line usage for uDialogManager.}

    1  ==========================================================
    2  Usage: uDialogManager file.moos [OPTIONS]               
    3  ==========================================================
    4                                                            
    5  Options:                                                  
    6    --alias=<ProcessName>                                
    7        Launch uFldHazardMetric with the given process name. 
    8    --example, -e                                           
    9        Display example MOOS configuration block.           
   10    --help, -h                                              
   11        Display this help message.                          
   12    --interface, -i                                         
   13        Display MOOS publications and subscriptions.        
   14    --version,-v                                            
   15        Display release version of uDialogManager.

3.5 Terminal and AppCast Output [top]

{Listing 4 - Example uDialogManager console output.}

    1  ===================================================================
    2  uDialogManager_2_0 mokai                                       0/0(655)
    3  ===================================================================
    4
    5  Sentence Action: ARNOLD_DEPLOY : DEPLOY=true + MOOS_MANUAL_OVERRIDE=false + RETURN=false
    6
    7  Sentence Action: ARNOLD FOLLOW : NODE_MESSAGE_LOCAL=src_node=mokai,
    8                                   dest_node=betty,var_name=RETURN,string_val=true
    9
   10  CURRENT STATE:
   11  Ready for Command.
   12  
   13  CONVERSATIONS:
   14
   15  DM: Command Sent
   16  User: Yes
   17  DM: Did you mean arnold follow
   18  User: Arnold Follow
   19

Line 5 shows a sentence action that was defined in the .moos file. In this case, the speech recognition sentence of ARNOLD DEPLOY triggers the three variables DEPLOY, MOOS_MANUAL_OVERRIDE, and RETURN to be published with the values true, false, and false, respectively. Line 7 shows another sentence action that was defined but in this case the value for NODE_MESSAGE_LOCAL is a string. As described above, NODE_MESSAGE_LOCAL allows a local variable to be published that then contains a var/value pair to be sent to another community. Line 10 displays the CURRENT STATE of uDialogManager, which can be in one of several states such as Ready for Command or Waiting for an ACK. Line 13 begins the display of the last 10 sentences between the user and the uDialogManager with the most recent sentence at the top. In this example, the user started by saying ``Arnold Follow which is displayed at Line 18. The uDialogManager responds with ``Did you mean arnold follow which is shown in Line 17. Lines 16 and 15 demonstrate that the user acknowledged with a ``Yes and then uDialogManager responded with ``Command Sent. If there is a configuration or runtime warning then Line 4 would be replaced with .moos file issues.

Page built from LaTeX source using the texwiki program, developed at MIT