<html><head><meta name="color-scheme" content="light dark"></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">\appendix
\newcommand{\ivp}{\mbox{{I}\hspace{-.008in}{v}\hspace{-.011in}{P} }}
\newcommand{\ivpp}{{I}\hspace{-.008in}{v}\hspace{-.011in}{P}}
\newcommand{\ripal}{\texttt{RIPAL}}
\newcommand{\ipal}{\texttt{IPAL}}
\input{sec_appendix1}

\section{Notes on Empirical Results}
\label{appx}

Ideally, it would be best to test \ivp solution methods on a set of
problems that are close in all aspects of difficulty to the problems
expected to find in practice. This has two drawbacks. First is that
while a method may work well for one application, it may be difficult
to judge how it would scale to other domains with different problem
characteristics. Second, it would require that the application be
explained to some extent to justify why certain sets of objective
functions are typical, which would significantly widen the scope of
this paper. 

Instead, the empirical results in this paper are based on a certain
class of functions called {\em ring functions}, described next. This
class is chosen because a) certain properties, such as non-linearity
and non-convexity, will pose at least as great, or greater
difficulties as those one might find in the ``real'' target
applications, and b) they are easy to generate by using random number
generators to set parameters, resulting in unique functions with
significantly different properties in terms of the amount, degree, and
location of local and global extrema, and c) one can crank up the
parameters, such as dimension, number of objective functions, and
pieces etc., to find trends in algorithm performance on types of 
problems not yet encountered in practice.

\subsection{Ring Functions}
\label{ring_funcs}

A {\em ring function} is built by placing a $n-$dimensional ring
somewhere in $n-$space, by choosing both its center and radius, and
then associating values to all points in the domain based on the
distance to the ring. Simple examples are depicted below in Figure
\ref{ringfiga}(a) and (b). More than one ring can be used as shown on
the right in Figure \ref{ringfiga}(c) and (d).

\begin{figure}[ht]
\begin{minipage}[b]{.99\textwidth}
    \centering \includegraphics[height=1.4in]{./localfigs/RingFig1245.eps}
\end{minipage}%
\caption{Example ring functions with one (a), (b), two (c) and 
         five (d) rings.}
  \label{ringfiga}
\end{figure}

\noindent
With just one ring, this represents a non-linear, non-convex
function. As more rings are added, the number of local optima also
tends to rise. The following definition identifies the parameters of
interest for generating sets of ring functions. A {\em ring function}
is a function with the following form:

$$f(x,y) = f_1(x,y) + \ldots + f_p(x,y),$$
\medskip

\newcommand{\range}{\mbox{\footnotesize{\tt{range}}}}
\newcommand{\tdis} {\mbox{\footnotesize{\tt{max-dist}}}}
\newcommand{\base} {\mbox{\footnotesize{\tt{base}}}}
\newcommand{\rad}  {\mbox{\footnotesize{\tt{rad}}}}
\newcommand{\rexp} {\mbox{\footnotesize{\tt{exp}}}}

\noindent
where each of the $p$ functions $f_i(x,y)$ correspond to a particular
ring given by:
\medskip


$$f_i(x,y) = ((1 - \frac {|\sqrt{(x-h)^2 + (y-k)^2} - \rad|} 
	                 {\tdis})^{\rexp} * \range) + \base,$$
\medskip

\noindent
where each function $f_i(x, y)$ is equal to $g(r(x, y))$ given by: 

$$r(x, y) = | \sqrt{(x-h)^2 + (y-k)^2} - \rad |,$$

\noindent
and

$$g(x) = ((1 - \frac{x}{\tdis})^{\rexp} * \range) + \base.$$
\medskip

\noindent 
The function $r(x, y)$ indicates a circle (ring) with radius \rad, and
returns the shortest distance of a point, $(x, y)$ to the ring.  The
function $g(x)$ takes this distance and produces the desired value
based on the following intuition.  The center of each ring is set to
be somewhere in the universe given by the Cartesian product of each
variable's domain. The value of {\tdis} is the maximum distance in
this universe, i.e., the length from corner to opposite corner of the
universe. The value of $\frac{r(x,y)}{\tdis}$ is therefore always in
the range $[0, 1]$.  Subtracting $\frac{r(x,y)}{\tdis}$ from $1$ and
raising it to the exponent {\rexp} still leaves us with a value in the
range $[0, 1]$. Multiplying this by {\range} and adding it to \base,
ensures that each function, $f_i(x,y)$, is guaranteed to range over
$[\base, \; \base+\range]$, and thus $f(x,y)$ has the range
$[p(\base), p(\base+\range)]$. The actual range of the function may be
quite smaller and is unknown based solely on the parameters.

By controlling the parameters, $p$, \rad, \rexp, \base, and \range, as
well as the number of variables and their domains, we can get
functions with rather predictable properties.  For example, by using a
large number of rings with a small radius and large {\rexp} factor,
instances such as shown below in Figure \ref{ringfigb}(a) are typical.
\begin{figure}[ht]
  \begin{minipage}[b]{.99\textwidth}
    \centering \includegraphics[height=1.8in]{./localfigs/RingFig679.eps}
  \end{minipage}
  \caption{Three types of ring functions based on varying parameter inputs.}
  \label{ringfigb}
\end{figure}
By then increasing the values to the radius parameter, instances such
as that in Figure \ref{ringfigb}(b) are common. And finally by
lowering the ring parameter, $p$, and the {\rexp} parameter, instances
like that in Figure \ref{ringfigb}(c) are typical. By first setting
the above parameters, and then letting the ring positions vary
randomly, significantly different looking functions can be generated
within a class of functions with certain expected properties.

In all experiments reported in this paper, the domain of each variable
ranged from 0 to 850, for a total of 851 possible values for each
variable. Each ring function was randomly generated with the following
values for the ring function parameters. The parameters {\tt base} and 
{\tt range} were set to $-100$ and $200$ respectively. The value of {\tt rad}
was randomly chosen between $20$ and $100$, and the value of {\tt exp}
was randomly chosen between $1$ and $20$.

%----------------------------------------------------------------------------------
\newpage
\subsection{Trial 1: Comparing \ripal{} version-1 vs. version-2}
\label{appx_trial1}

\noindent
Trial 1 contains a pair of trials, 1A and 1B. 
See the paper for the plotted data shown in these two tables. 
As with all trials reported
in this paper, time is reported in seconds, and the superscript for
each table entry contains 4 comma-separated values: the min-value, the
max-value, the standard deviation, and the number of test problems
contributing to that particular entry.

the plots 
comparing \ripal{} version 1 and version 2.

\input{../experiments/inc_trial1A}
\input{../experiments/inc_trial1B}

\begin{table}[H]
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableOneA
  \caption{Two objective functions}
\label{table1a}
  \end{minipage}%
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableOneB
  \caption{Three objective functions.}
  \label{table1b}
  \end{minipage}%
\end{table}


%----------------------------------------------------------------------------------
\newpage
\subsection{Trial 2: Comparing \ripal{} version-2 vs. version-3}
\label{appx_trial2}

\noindent
Trial 2 contains a pair of trials, 2A and 2B. 
See the paper for the plotted data shown in these two tables. 
As with all trials reported in this paper, time
is reported in seconds, and the superscript for each table entry
contains 4 comma-separated values: the min-value, the max-value, the
standard deviation, and the number of test problems contributing to
that particular entry.


\input{../experiments/inc_trial2A}
\input{../experiments/inc_trial2B}

\begin{table}[H]
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableTwoA
  \caption{Two objective functions}
  \label{table2a}
  \end{minipage}%
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableTwoB
  \caption{Three objective functions.}
  \label{table2b}
  \end{minipage}%
\end{table}


%----------------------------------------------------------------------------------
\newpage
\subsection{Trial 3: Comparing \ripal{} version-3 vs. version-4}
\label{appx_trial3}

\noindent
Trial 3 contains three pairs of trials, 3A and 3B, 3C and 3D, 3E and
3F.  See the paper for the plotted data shown in each pair of tables.  As
with all trials reported in this paper, time is reported in seconds,
and the superscript for each table entry contains 4 comma-separated
values: the min-value, the max-value, the standard deviation, and the
number of test problems contributing to that particular entry.


\input{../experiments/inc_trial3A}
\input{../experiments/inc_trial3B}

\begin{table}[H]
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableThreeA
  \caption{Two dimensions, 10 objective functions.}
  \label{table3a}
  \end{minipage}%
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableThreeB
  \caption{Four dimensions, 10 objective functions.}
  \label{table3b}
  \end{minipage}%
\end{table}

\input{../experiments/inc_trial3C}
\input{../experiments/inc_trial3D}

\begin{table}[H]
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableThreeC
  \caption{Two dimensions, 20 objective functions.}
  \label{table3c}
  \end{minipage}%
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableThreeD
  \caption{Four dimensions, 20 objective functions.}
  \label{table3d}
  \end{minipage}
\end{table}
    
\input{../experiments/inc_trial3E}
\input{../experiments/inc_trial3F}

\begin{table}[H]
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableThreeE
  \caption{Two dimensions, 4000 pieces.}
  \label{table3e}
  \end{minipage}%
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableThreeF
  \caption{Four dimensions, 4000 pieces.}
  \label{table3f}
  \end{minipage}
\end{table}


%----------------------------------------------------------------------------------
\newpage
\subsection{Trial 4A and 4B: \ripal{} With and Without an Initial Solution }
\label{appx_trial4}

\noindent
Trial 4 contains six pairs of trials, 4A and 4B, 4C and 4D, 4E and 4F.
See the paper for the plotted data shown in each pair of tables.  As
with all trials reported in this paper, time is reported in seconds,
and the superscript for each table entry contains 4 comma-separated
values: the min-value, the max-value, the standard deviation, and the
number of test problems contributing to that particular entry.


\input{../experiments/inc_trial4A}
\input{../experiments/inc_trial4B}

\begin{table}[H]
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableFourA
  \caption{Two dimensions, 10 objective functions.}
  \label{table4a}
  \end{minipage}%
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableFourB
  \caption{Four dimensions, 10 objective functions.}
  \label{table4b}
  \end{minipage}%
\end{table}

\input{../experiments/inc_trial4C}
\input{../experiments/inc_trial4D}

\begin{table}[H]
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableFourC
  \caption{Two dimensions, 4000 pieces.}
  \label{table4c}
  \end{minipage}%
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableFourD
  \caption{Four dimensions, 4000 pieces.}
  \label{table4D}
  \end{minipage}%
\end{table}


\input{../experiments/inc_trial4E}
\input{../experiments/inc_trial4F}

\begin{table}[H]
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableFourE
  \caption{Two dimensions, 4000 pieces.}
  \label{table4e}
  \end{minipage}%
  \begin{minipage}[b]{0.50\textwidth}
    \centering \TableFourF
  \caption{Four dimensions, 4000 pieces.}
  \label{table4f}
  \end{minipage}%
\end{table}

</pre></body></html>