33   Mission S11: The Kilo Mission


33.1 Overview of the Kilo Mission Components and Topics
33.2 Launching the Kilo Mission
33.3 Topic #1: The Use of a Standby Helm
33.4 Topic #2: The TestFailure Behavior
33.5 Topic #3: Scoping the Helm State(s) at Runtime
33.6 Suggestions for Tinkering


The purpose of the Kilo mission is to illustrate the use of the standby helm, and the use of the TestFailure behavior. The standby helm is described in Section 5.7 and consists of an otherwise normally configured helm with the additional standby parameter invoked. This helm will wait in standby mode until the heartbeat of another (primary) helm ceases to be posted to the MOOSDB for some period of time. In this example mission, a primary and standby helm are configured with the primary helm executing a simple mission similar to the Alpha mission, but also using an instance of the TestFailure behavior. This behavior will be configured to trigger either a crash or hang of the primary helm to demonstrate the manner in which a standby helm will step in to take over with its own mission. The standby mission in this case is simply a behavior to return the vehicle to its launch position.

33.1   Overview of the Kilo Mission Components and Topics    [top]


Behaviors:Waypoint, TestFailure
MOOS Apps:pHelmIvP, pLogger, uSimMarine, pMarinePID, pNodeReporter, pMarineViewer
Primary Topics:(1) The use of a standby helm.
(2) The TestFailure behavior.
(3) Scoping the primary and standby helm states at runtime.
Side Topics:(1) uHelmScope may be used to monitor the relationship between the shadow helm and primary helm.

33.2   Launching the Kilo Mission    [top]


The Kilo mission may be launched from the command line:

  $ cd moos-ivp/missions/s11_kilo/
  $ ./launch.sh --warp=10           (or more simply, ./launch.sh 10)

This should bring up a pMarineViewer window like that shown in Figure 33.4, with a single vehicle, "kilo", initially in the "PARK" mode. After hitting the DEPLOY button in the lower right corner, the vehicle enters the "SURVEYING" mode and begins to proceed along the waypoints as shown.

33.3   Topic #1: The Use of a Standby Helm    [top]


The vehicle in this example is configured with both a primary helm and a secondary helm. Configuring a mission with two helm instances is straight forward. The primary helm is configured as done normally, as in lines 3-11 in Listing 33.1 below. The standby helm is configure identically but with one additional line, as in line 23 in Listing 33.1 below. This line denotes the helm it to be a standby helm, and indicates the time threshold used in determining when to take over from a primary helm.

Listing 33.1 - Configuration of the primary and secondary helm in the Kilo mission.

   1  //----------------------------- pHelmIvP config block
   2  
   3  ProcessConfig = pHelmIvP
   4  {
   5    AppTick    = 4
   6    CommsTick  = 4
   7  
   8    behaviors  = kilo.bhv
   9    domain     = course:0:359:360
  10    domain     = speed:0:4:21
  11  }
  12  
  13  //----------------------------- pHelmIvP_Standby config block
  14  
  15  ProcessConfig = pHelmIvP_Standby
  16  {
  17    AppTick    = 4
  18    CommsTick  = 4
  19  
  20    behaviors  = kilo_standby.bhv
  21    domain     = course:0:359:360
  22    domain     = speed:0:4:21
  23    standby    = 2
  24  }

Both helms are instances of the pHelmIvP application, but one of them is named pHelmIvP_Standby. The name of the application is not in any way used to put this helm instance into a standby role. It is chosen simply to be different from the primary helm, since the MOOSDB requires all connected applications to have unique names, and to be clear when or if debugging the mission.

    The shadow helm is launched by pAntler with the name pHelmIvP_Standby. This is done by giving it the alternative name using the pAntler ExtraProcessParams parameter as shown on lines 15 and 18 below.

Listing 33.2 - Configuration of Antler.

   1  //------------------------------------------
   2  // Antler configuration  block
   3  
   4  ProcessConfig = ANTLER
   5  {
   6    MSBetweenLaunches = 200
   7  
   8    Run = MOOSDB            @ NewConsole = false
   9    Run = pLogger           @ NewConsole = false
  10    Run = uSimMarine        @ NewConsole = false
  11    Run = pNodeReporter     @ NewConsole = false
  12    Run = pMarinePID        @ NewConsole = false
  13    Run = pMarineViewer     @ NewConsole = false
  14    Run = pHelmIvP          @ NewConsole = true
  15    Run = pHelmIvP          @ NewConsole = true, ExtraProcessParams=HParams
  16    Run = uProcessWatch     @ NewConsole = false
  17  
  18    HParams=--alias=pHelmIvP_Standby
  19  }

If the shadow helm is launched independently, not with pAntler, it may be launched with:

  $ pHelmIvP kilo.moos --alias=pHelmIvP_Standby

33.4   Topic #2: The TestFailure Behavior    [top]


The Kilo mission uses the TestFailure behavior described in Section 22 to artificially generate a failure of the primary helm. In the Kilo mission it is configured to produce a "hung" helm with the following configuration:

Listing 33.3 - Configuration of the TestFailure Behavior.

   1  //----------------------------------------------
   2  Behavior = BHV_TestFailure
   3  {
   4    name                = test_failure
   5    condition           = DEPLOY=true
   6    duration            = 120
   7    duration_idle_decay = false
   8  
   9          failure_type  = hang,3
  10  }

    The first four configuration parameters are defined for all IvP behaviors. A run condition of DEPLOY=true keeps this behavior idle until the mission is launched. Once it is launched, a countdown of 120 seconds begins until the behavior will hang. The duration_idle_decay=false setting ensures that the countdown doesn't begin until the behavior is in the running state. In this case the behavior will fail by hanging for three seconds. This will cause the primary helm to also appear to hang for three seconds, noted by a three second gap between heartbeats, defined by posting to the MOOS variable IVPHELM_STATE. Since the standby helm is configured to wait no longer than two seconds (see Listing 33.1), the standby helm will promptly take over control from the primary helm. The sequence of events in taking over control is described in more detail next.

33.5   Topic #3: Scoping the Helm State(s) at Runtime    [top]


The relationship between the primary and standby helm may be monitored by scoping on the variable IVPHELM_STATE during the course of the mission. Figure 33.1 below shows the situation shortly after the vehicle is deployed using the uXMS scoping tool.

Figure 33.1: Helm State in the Kilo Mission: Both the standby and primary helms are operating normally, posting a helm state message about four times per second.

    Initially the standby helm is showing a helm state of "STANDBY", and the primary helm is showing a helm state of "DRIVE". Since both helms are operating at the same frequency, they mostly alternate between postings as shown above.

    The events shown in Figure 33.2 show that during the 9 identical postings ending at time 329.86, the standby helm is the only helm emitting a heartbeat. During this period the primary helm has either crashed or hung. Finally the secondary helm takes over and posts a helm state of "DRIVE+" for 107 consecutive heartbeats (iterations). Recall the "+" is to further distinquish that this helm was originally configured as a standby helm.

Figure 33.2: Helm State in Kilo Mission after Standby Helm Takes Over Hung Primary Helm: The standby helm has detected a delay in the primary helm heartbeat and takes over about two seconds later. Note the number in parentheses is the number of identical postings with the timestamp showing the time of the last of the identical postings.

Eventually it turns out that the original primary helm did not crash after all but was just temporarily hung. After emerging from its hung state, upon reading mail in the next loop, it realizes the standby helm has taken over and immediately concedes control and posts the "DISABLED" helm state, and will never again post. One may wonder why the primary helm, when it finally woke up, did not first mistakely post "DRIVE" before conceding control - after all it needs to first read its mail to learn that another helm is in control. The primary helm in fact did post "DRIVE" and "DISABLED" on two consecutive iterations. The first posting occurred (time 327.81) prior to querying the behaviors for input, and the second posting (time 356.70) occurred on the next iteration of upon reading mail.

    In the case where the standby helm takes over for a crashed helm, the sequence of events, viewed from the perspective of posts to IVPHELM_STATE, is similar as shown in Figure 33.3. When things are going fine the standby helm and primary helm alternately post helm states of STANDBY and DRIVE respectively as the first several posts below shoe. At some point the primary helm crashes and the only posts are made by the standby helm. In the example here, there are 10 IVPHELM_STATE="STANDBY" posts made while the standby helm is getting closer to triggering a take-over. Finally, upon takeover, it posts a helm state of "DRIVE+" thereafter. The primary helm is never heard from again.

Figure 33.3: Helm State in Kilo Mission after Standby Helm Takes Over Crashed Primary Helm: The standby helm has detected a delay in the primary helm heartbeat and takes over about two seconds later. Note the number in parentheses is the number of identical postings with the timestamp showing the time of the last of the identical postings.

33.6   Suggestions for Tinkering    [top]


  • Analyze a helm takeover from the log files. Run the Kilo mission again and, as opposed to monitoring things with a live scope as in Figures 33.1 - 33.3, let the mission play out. Change directories to the folder where the .alog file was created. Take a look at the IVPHELM_STATE postings using the aloggrep tool. Confirm that the sequence of events described in Figures 33.1 - 33.3 are consistent with entries of the log file. Hint: This can be achieved without mission configuration or code modifacations.
  • Crashing with helm with a mouse click. Rather than waiting for the TestFailure behavior to run down its clock to failure, configure it to fail immediately upon entering the run state (duration=0). But also set an additional run-condition, e.g., FAIL=true and configure the pMarineViewer to make this posting upon a mouse-click. Test the modified Kilo mission by launching it, and clicking the mouse when ready to see the failure. This can be achieved solely through mission configuration modification, without code modification.
  • Configure a standby helm that finishes a task. Create a primary and shadow helm where the primary helm is surveying a set of waypoints. The shadow helm should be able to survey the remaining points when/if the primary helm is taken over. Hint: this will likely require code generation or modification. Extra bonus if achievable solely through mission modification.

Figure 33.4: The Kilo Mission (1): The vehicle "kilo" traverses the waypoints using a primary helm, launched alongside a standby helm. The TestFailure behavior is a ticking bomb that will hang the primary helm momentarily.

Figure 33.5: The Kilo Mission (2): The TestFailure behavior has hung the primary helm. The standby helm has detected the absence of a heartbeat (the IVPHELM_STATE variable), and take over with a simple return mission.


Page built from LaTeX source using the texwiki program.