Computing Pi using Exclusive Tasks

Introduction

To introduce exclusive tasks we will create an application to compute Pi. We will use the Monte Carlo algorithm to generate input values. The values are generated by creating a square then placing a circle within the square. Point selection in the square is calculated using a random number generator. The total number points within the circle is multiplied by 4 and then divided by the total number of points in the square. The result is an approximation of Pi.

You may create your own fabric application and client application, or you may use the files and projects provided in tutorial_samples.zip and tutorial_samples.tgz archives. These files and projects are not included in the Fabric Software Development Kit (SDK) installation, but will be available for later versions. The fabric APIs are not dependent on a particular build tool or development environment (IDE), so you may use your preferred build tools. For your convenience, we have included project files for several platforms. These can be seen in the table to the right.

In this tutorial we will

describe an exclusive task
compare exclusive tasks to other types of fabric tasks
explain an exclusive task's life cycle
create an exclusive task
configure an exclusive task
create a simple client
submit requests to the fabric

Projects Provided For Tutorial Samples

Language	Platform	Project Type
Java 1.5	All	Eclipse
Java 1.5	All	Ant scripts
.NET 2.0	Windows	Visual Studio 2005
'C'	Windows	Visual Studio 2005
'C'	Linux	Make scripts

Unknown macro: {switcher}

What our computing Pi application will do

client submits a request to the fabric to execute application's process flow
the process flow calls the Monte Carlo Pi task
the Monte Carlo Pi task calculates data points and returns the results to the process flow
the process flow returns the request to the client
the client receives the calculated data points in the completed request
client computes Pi using calculated data points and prints the computed Pi to the console
To learn all the steps for building, packaging, and deploying your own fabric application and client,
you should reference the Hello World with Process Flows and Tasks tutorial

Monte Carlo Pi Fabric Application

Exclusive task

An exclusive task is any task that you want to restrict the number of instances running in the fabric per worker or per CPU. This allows you to control the number of instances of a specific task running in the fabric. For example, you may have a task that very resource hungry (CPU, memory, or disk intensive for example) and want to restrict how many instances of this code can run at a given time on individual fabric workers. Another example would be if your task code or some underlying library is not thread-safe and you want to ensure that only single copies of the task are not run concurrently. Classifying the task as exclusive ensures only N instances of that task runs per worker or per CPU.

How do I decide which type of task to choose for my task code?

The following are some general guidelines:

if your task code and its underlying libraries is thread safe, short running, and not resource hungry (CPU, memory, or disk intensive for example) then configure your code as an unlimited task
if your task code needs to use a restricted resource, for example you have a limited number of licensed database connections, use a limited task
if your task code or its underlying libraries are not thread safe, or your task code will be long running (more than a couple of seconds or more), or is resource hungry (CPU, memory, or disk intensive for example), then configure your code as an exclusive task
if you need to run code in the fabric that does not need to service fabric requests from a client, but perhaps needs to monitor a socket, receive work from a fabric accessible memory (FAM) queue, or poll some outside service or database, then configure and model your code to run as a background task.

The fabric is capable of running multiple, concurrent instances of a task on a worker or CPU. However, this is not always desirable. Appistry EAF allows you to configure a task by classifying it as one of four types and adding parameters based on the task type. The table below provides a basic overview of the four task types. For in depth explanations, refer to the appropriate task's tutorial.

Task Type	Characteristics	Configuration	Tutorial
Unlimited	No restrictions Multiple instances may run concurrently on multiple workers in the fabric Multiple requests for the task are serviced concurrently depending on available memory and CPU	Task XML file	Hello World with Process Flows and tasks
Limited	Only a specified number of instances may run concurrently in the fabric (for example, a limited task may be configured to run on 2 out of 10 workers) A worker may run multiple instances of the task Multiple requests for the task are serviced concurrently depending on configuration, available memory, and CPU	Task XML file
Exclusive	Only a specified number of instances may run concurrently per worker or per CPU (for example, an exclusive task will run on every worker but be restricted to only one instance at a time based on worker or CPU) The fabric performs load control to balance requests for exclusive tasks across the fabric When multiple requests for an exclusive task arrive on a single worker, the requests are executed in a gated fashion. For example, if exclusive tasks are configured to allow two concurrent per worker, then two requests are serviced at a time while other requests wait their turn. Good for long running tasks (several seconds or more) or CPU intensive	Task XML file and fabric.cfg	Computing Pi using Exclusive Tasks tutorial
Background	Runs constantly waiting for work (like a unix daemon or Windows service) Background tasks are not called from a process flow as part of a client request, but rather get "work" as defined by the task implementer. For example, the background task might monitor a queue in fabric accessible memory (FAM), read from a socket. or poll a service external to the fabric. Since it is not part of a fabric request, background tasks do not have the same inherent reliability as the other task types	Task XML file

Exclusive task configuration

A specific task is classified as exclusive in its task XML definition file.

However, exclusive task settings are defined fabric wide. Exclusive task configuration consist of two settings in the fabric.cfg file: exclusive-task-type and exclusive-task-count. The exclusive-task-type specifies whether the number instances running is based on worker or CPU. This parameter has two possible values: per-machine and per-cpu. The per-machine value indicates a worker can run the configured number of exclusive tasks. The per-cpu value indicates a worker can run the configured number of exclusive tasks on each CPU. The exclusive-task-count parameter specifies the maximum number of exclusive tasks that may run concurrently on a worker or CPU.

The exclusive-task-type and exclusive-task-count are fabric-wide settings. All exclusive tasks running in the fabric use the same exclusive-task-type and exclusive-task-count settings. The fabric does not support individually configured exclusive tasks.

For example, if you have a fabric of 50 single CPU workers and 50 dual CPU workers and the exclusive task type is per-machine and the exclusive task count is 2, the maximum number of exclusive tasks that may run at the same time is 200, 2 per worker. If the exclusive task type is per-cpu and the exclusive task count is 2, the maximum number of exclusive tasks that may run at the same time is 300, 2 per each single CPU worker and 4 per each dual CPU worker.

Modify the fabric.cfg file

To ensure you are modifying a current version of the fabric.cfg file, obtain a copy directly from the fabric using Fabric Control.

To retrieve a copy of the fabric.cfg file currently in the fabric, run the following command with the correct fabric administrative user, password, and fabric address. Fabric Control stores a copy of the fine in the current working directory.
```
fabric_ctl -d 239.255.0.1:30000 -u fabric-admin/fabric-admin get fabric.cfg
```

Open the fabric.cfg file in a text editor. Below is an example this file.

version=1.0
exclusive-task-type=per-machine
exclusive-task-count=1
concurrent-shutdown-threshold=2
auto-start-worker=true
split-region-threshold=20
max-request-handler-count=1000
min-request-handler-count=100
recovery-store-purge-interval=24
recovery-store-request-ttl=24
mcast-ttl=1
request-handler-percentage=100
request-service-ssl-enabled=false
request-service-port=16000
request-service-ssl-port=16001
admin-server-port=0
#message-logger-group=239.255.0.1
#message-logger-port=4000
action-on-reboot=just-reboot
exclusive-task-count=1
exclusive-task-type=per-machine
guaranteed-non-fam-memory=0

If the exclusive-task-type parameter is not in the file, add it to the bottom of the list with a value of per-machine or per-cpu. For example, exclusive-task-type=per-machine.
If the exclusive-task-count parameter is not in the file, add it to the bottom of the list with a numerical value. For example, exclusive-task-count=1.

Increase the version number.

Version numbers
All version numbers are right justified and zero filled. When comparing version 1.2 with 1.19, the application fabric actually compares 1.02 with 1.19. Version 1.19 is considered newer than version 1.2. Also, when comparing version 1.0.0 with 1.0 and 1, 1.0.0 is considered newer than version 1.0 and 1.0 is considered newer than version 1.

Save the file and close the text editor.
To deploy the modified fabric.cfg file, run the following command with the correct fabric administrative user, password, and fabric address.
```
fabric_ctl -d 239.255.0.1:30000 -u fabric-admin/fabric-admin deploy fabric.cfg
```

Monte Carlo Pi task

Creating the code for an exclusive task is the same as creating the code in the Hello World tutorial.

MonteCarloPi.java (in tutorial_samples/java/monte_carlo_pi/com/appistry/samples/pi/)

0001. package com.appistry.samples.pi;
0002. 
0003. import java.util.Random;
0004. import com.appistry.task.Annotations.TaskParameter;
0005. import com.appistry.task.Annotations.TaskReturnValue;
0006. 
0007. public class MonteCarloPi {
0008. 
0009.     // OUTPUT: map method return value into "computed-points"
0010.     // key in current fabric request object	
0011.     // INPUT: on execution, map value for "total-points-to-compute"
0012.     // from current fabric request object into method parameter
0013.     @TaskReturnValue(name="computed-points")
0014.     public long computePoints(@TaskParameter("total-points-to-compute") long totalPoints) {
0015. 		
0016.         double x, y;
0017.         int pointsInCircle = 0;
0018.         // one could insert a better random number generator here....
0019.         Random generator = new Random( (long)(System.currentTimeMillis()) );
0020. 		
0021.         // compute points using monte carlo method by keeping track of
0022.         // points falling in the "circle" (i.e. computed as <= 1)
0023.         for (int i=0; i < totalPoints; i++) {
0024.             x = generator.nextDouble();
0025.             y = generator.nextDouble();
0026.             if ( Math.sqrt( x * x + y * y ) <= 1 )
0027.                 pointsInCircle++;
0028.         }
0029. 	
0030.         // return count of points to client for Pi computation
0031.         return pointsInCircle;
0032.     }
0033. }

MonteCarloPi.cs (in tutorial_samples/dotnet/monte_carlo_pi/monte_carlo_pi_app/)

0001. using System;
0002. using Appistry.Task;
0003. 
0004. namespace Appistry.Samples.Pi
0005. {
0006.     public class MonteCarloPi
0007.     {
0008.         // OUTPUT: map method return value into "computed-points"
0009.         // key in current fabric request object	
0010.         // INPUT: on execution, map value for "total-points-to-compute"
0011.         // from current fabric request object into method parameter
0012.         [return: TaskReturnValue("computed-points")]
0013.         public long computePoints([TaskParameter("total-points-to-compute")] long totalPoints)
0014.         {
0015.             double x, y;
0016.             int pointsInCircle = 0;
0017.             // one could insert a better random number generator here....
0018.             Random generator = new Random((int)DateTime.Now.Ticks);
0019. 
0020.             // compute points using monte carlo method by keeping track of
0021.             // points falling in the "circle" (i.e. computed as <= 1)
0022.             for (int i = 0; i < totalPoints; i++)
0023.             {
0024.                 x = generator.NextDouble();
0025.                 y = generator.NextDouble();
0026.                 if (Math.Sqrt(x * x + y * y) <= 1)
0027.                     pointsInCircle++;
0028.             }
0029. 
0030.             // return count of points to client for Pi computation
0031.             return pointsInCircle;
0032.         }
0033.     }
0034. }

monte_carlo_pi.cpp (in tutorial_samples/cpp/monte_carlo_pi/monte_carlo_pi_app/)

0001. #include "fabric_client.h"
0002. #include <cstdlib> 
0003. #include <ctime>
0004. #include <cmath>
0005. 
0006. #ifdef _WIN32
0007. #include <windows.h>
0008. #endif
0009. 
0010. extern "C" void computePoints(fab_req* request)
0011. {
0012.     // use fab_uint64 to be compatible with Java and Dotnet clients using primitive long
0013.     fab_uint64 totalPoints = fab_req_get_int64(request, "total-points-to-compute");
0014. 
0015.     double x, y;
0016.     long pointsInCircle = 0;
0017.     // one could insert a better random number generator here....
0018.     srand((unsigned)time(0)); 
0019. 
0020.     // compute points using monte carlo method by keeping track of
0021.     // points falling in the "circle" (i.e. computed as <= 1)
0022.     for (int i = 0; i < totalPoints; i++)
0023.     {
0024.         x = rand()/(RAND_MAX + 1.0);
0025.         y = rand()/(RAND_MAX + 1.0);
0026.         if (sqrt(x * x + y * y) <= 1)
0027.             pointsInCircle++;
0028.     }
0029. 
0030.     fab_req_set_int64(request, "computed-points", pointsInCircle);
0031. }
0032.

monte_carlo_pi.def (Windows only) (in tutorial_samples/cpp/monte_carlo_pi/monte_carlo_pi_app/)

1. EXPORTS
2. 	computePoints

Monte Carlo Pi Task XML Definition

The Monte Carlo Pi Task XML Definition file is almost identical to the Hello World Task XML Definition file. The main difference is that the Task element has two additional attributes: type and busy-wait-timeout.

Attribute	Description
type	The type attribute identifies the task's classification. The possible values are unlimited, limited, exclusive, and background.
busy-wait-timeout	The busy-wait-timeout attribute indicates the amount of time, in milliseconds, the fabric waits for an available worker when workers running exclusive tasks are busy.

For this XML file, add the type attribute with a value of exclusive to the Task element as illustrated below.

Read more about task XML definitions under Task Module Definition XML.

Monte Carlo Pi Process Flow and Application XML Definitions

The Monte Carlo Pi Process Flow XML Definition and Monte Carlo Pi Application XML Definition files are similar to Hello World Process Flow XML Definition and Hello World Application XML Definition files. They are illustrated below.

Java and .NET process flow definition file

Read more about process flows under Process Flow Definition XML.

monte_carlo_pi_java_flow (in tutorial_samples/<java|dotnet|cpp>/monte_carlo_pi/>/monte_carlo_pi/)

1. <?xml version="1.0"?>
2. <!DOCTYPE process SYSTEM "ProcessDefinition.dtd">
3. <process name="monte_carlo_pi_flow" default_retries="0" default_timeout="30">
4.     <state name="start">
5.         <task name="computeMonteCarloPiPoints"/>
6.         <arc task-status="complete" next-state="finish"/>
7.     </state>
8. </process>

Java and .NET application definition file

Read more about application definitions under Fabric Application Definition XML.

Building, packaging and deploying the Monte Carlo Pi Fabric Application

Now we must build, package, and deploy our fabric application. Follow the instructions in the information box below, and then return to this tutorial.

The Building, Packaging, and Deploying a Fabric Application tutorial gives directions for building, packaging and deploying fabric applications.

Simple Monte Carlo Pi Fabric Client

We have provided a simple, blocking client for testing your Monte Carlo Pi fabric application. This client is very similar to the Hello World Fabric Client.

"Okay, where's the real Monte Carlo Pi fabric client?"
To really make use of our new exclusive task, we need a client that can push enough requests into the fabric to drive up utilization on our workers. Our simple Monte Carlo Pi client listed here won't do that. To really put the Monte Carlo Pi fabric application through it's paces, we have provided the [Multi-threaded, Asynchronous Client Tutorial] (coming soon!). Test your deployed Monte Carlo Pi fabric application using the simple client here, and then go over and put your fabric through its paces in the [Multi-threaded, Asynchronous Client Tutorial] (coming soon!).

The expected output from our client's call to the fabric application should be similar to that below (the values will vary).

Pi computed using Monte Carlo method
Total points: 6000000
Valid points: 4712959
PI: 3.1419726666666667

MonteCarloPiClient.java (in tutorial_samples/java/monte_carlo_pi/)

00001. package com.appistry.samples.pi;
00002. 
00003. import com.appistry.fabric.Encryption;
00004. import com.appistry.fabric.Fabric;
00005. import com.appistry.fabric.FabricRequest;
00006. 
00007. public class MonteCarloPiClient {
00008. 
00009.     public static double computePiFromPoints(long computedPoints, long totalPoints) {
00010.         return (4.0 * computedPoints / totalPoints);
00011.     }
00012. 	
00013.     public static void reportResults(long computedPoints, long totalPoints, double pi) {
00014.         System.out.println("Pi computed using Monte Carlo method");
00015.         System.out.println("Total points: " + totalPoints);
00016.         System.out.println("Valid points: " + computedPoints);
00017.         System.out.println("PI: " + pi);
00018.     }
00019. 	
00020.     public static void main(String[] args) {
00021.         // construct a fabric API object instance specifying fabric
00022.         // address values from fabric.cfg, Mcast TTL and encryption type
00023.         // Mcast TTL must be 1 or greater if the fabric client is running
00024.         // on a box separate from the fabric workers.
00025.         Fabric fabric = new Fabric("239.255.0.1", 31000, 1, Encryption.NONE);
00026. 
00027.         // construct a fabric request object instance specifying
00028.         // the fabric application and process flow to run
00029.         FabricRequest request = new FabricRequest("monte_carlo_pi_java_app", "monte_carlo_pi_flow");
00030.     
00031.         // we need to specify the total number of monte carlo points for 
00032.         // a worker to compute for this request. we set this on the request.
00033.         long totalPoints = 2000000;
00034.         request.put("total-points-to-compute", totalPoints);
00035. 
00036.         // submit the request to the fabric
00037.         fabric.execute(request);
00038. 
00039.         // compute the value of pi from the worker-generated monte carlo points
00040.         // we do the final computation here because a full client would submit
00041.         // to and collect results from multiple workers before computing pi.
00042.         long computedPoints = (Long) request.get("computed-points");
00043.         double pi = computePiFromPoints(computedPoints, totalPoints);
00044.         reportResults(computedPoints, totalPoints, pi);
00045.     }
00046. }
00047.

MonteCarloPiClient.cs (in tutorial_samples/dotnet/monte_carlo_pi/monte_carlo_pi_client/)

000001. using System;
000002. using Appistry.FabricAPI;
000003. 
000004. namespace Appistry.Samples.HelloWorld
000005. {
000006.     public class MonteCarloPiClient
000007.     {
000008.         public static double computePiFromPoints(long computedPoints, long totalPoints)
000009.         {
000010.             return (4.0 * computedPoints / totalPoints);
000011.         }
000012. 
000013.         public static void reportResults(long computedPoints, long totalPoints, double pi)
000014.         {
000015.             Console.WriteLine("Pi computed using Monte Carlo method");
000016.             Console.WriteLine("Total points: " + totalPoints);
000017.             Console.WriteLine("Valid points: " + computedPoints);
000018.             Console.WriteLine("PI: " + pi);
000019.         }
000020. 
000021.         static void Main(string[] args)
000022.         {
000023.             try
000024.             {
000025.                 // construct a fabric API object instance specifying fabric
000026.                 // address values from fabric.cfg, Mcast TTL and encryption type
000027.                 // Mcast TTL must be 1 or greater if the fabric client is running
000028.                 // on a box separate from the fabric workers.
000029.                 Fabric fabric = new Fabric("239.255.0.1", 31000, 1, Encryption.NONE);
000030. 
000031.                 // construct a fabric request object instance specifying
000032.                 // the fabric application and process flow to run
000033.                 FabricRequest request = new FabricRequest("monte_carlo_pi_dotnet_app", "monte_carlo_pi_flow");
000034. 
000035.                 // we need to specify the total number of monte carlo points for 
000036.                 // a worker to compute for this request. we set this on the request.
000037.                 long totalPoints = 20000000;
000038.                 request["total-points-to-compute"] = totalPoints;
000039. 
000040.                 // submit the request to the fabric
000041.                 fabric.Execute(request);
000042. 
000043.                 // compute the value of pi from the worker-generated monte carlo points
000044.                 // we do the final computation here because a full client would submit
000045.                 // to and collect results from multiple workers before computing pi.
000046.                 long computedPoints = (long)request["computed-points"];
000047.                 double pi = computePiFromPoints(computedPoints, totalPoints);
000048.                 reportResults(computedPoints, totalPoints, pi);
000049.             }
000050.             catch (Exception e)
000051.             {
000052.                 Console.WriteLine(e.ToString());
000053.             }
000054.         }
000055.     }
000056. }

monte_carlo_pi_client.cpp (in tutorial_samples/cpp/monte_carlo_pi/monte_carlo_pi_client/)

0000001. #include "fabric_client.h"
0000002. 
0000003. #include <iostream>
0000004. using namespace std;
0000005. 
0000006. double computePiFromPoints(fab_int64 computedPoints, fab_int64 totalPoints)
0000007. {
0000008.   return (4.0 * computedPoints / totalPoints);
0000009. }
0000010. 
0000011. void reportResults(fab_int64 computedPoints, fab_int64 totalPoints, double pi)
0000012. {
0000013.   cout << "Pi computed using Monte Carlo method" << endl;
0000014.   cout << "Total points: " << totalPoints << endl;
0000015.   cout << "Valid points: " << computedPoints << endl;
0000016.   cout << "PI: " << pi << endl;
0000017. }
0000018. 
0000019. int main(int argc, char* argv[])
0000020. {
0000021.   // construct a fabric properties object to specify our Mcast TTL.
0000022.   // Mcast TTL must be 1 or greater if the fabric client is running
0000023.   // on a box separate from the fabric workers.
0000024.   fab_properties* properties = fab_properties_new();
0000025.   fab_properties_set_mcast_ttl(properties, 1);
0000026. 
0000027.   // construct a fabric API object instance specifying fabric
0000028.   // address values from fabric.cfg. 
0000029.   fabric* fabric = fab_new("239.255.0.1", 31000, properties);
0000030. 
0000031.   // construct a fabric request object instance specifying
0000032.   // the fabric application and process flow to run
0000033.   fab_req* request = fab_req_new("monte_carlo_pi_cpp_app", "monte_carlo_pi_flow");
0000034. 
0000035.   // we need to specify the total number of monte carlo points for 
0000036.   // a worker to compute for this request. we set this on the request.
0000037.   fab_int64 totalPoints = 20000000;
0000038.   fab_req_set_int64(request, "total-points-to-compute", totalPoints);
0000039. 
0000040.   // submit the request to the fabric
0000041.   int rc = fab_execute(fabric, request);
0000042. 
0000043.   if ( rc == FAB_SUCCESS )
0000044.   {
0000045.     // compute the value of pi from the worker-generated monte carlo points
0000046.     // we do the final computation here because a full client would submit
0000047.     // to and collect results from multiple workers before computing pi.
0000048.     fab_int64 computedPoints = fab_req_get_int64(request,"computed-points");
0000049.     double pi = computePiFromPoints(computedPoints, totalPoints);
0000050.     reportResults(computedPoints, totalPoints, pi);
0000051.   }
0000052.   else
0000053.   {
0000054.     cout << "Fabric Error Code: " << rc << " Message: " << fab_req_error_message(request) << endl;
0000055.   }
0000056. 
0000057.   // free up our request and fabric API resources
0000058.   fab_req_free(request);
0000059.   fab_free(fabric);
0000060. 
0000061.   return rc;
0000062. }
0000063.

Running the Monte Carlo Pi Fabric Client and Application

Before running the Monte Carlo Pi client, verify the following.

The fabric is running. If the fabric is not running, refer to Installing the Fabric for instructions.
The Log Monitor is running in a separate window or terminal. To start the Log Monitor run the following command. The MCAST address must match the fabric-address set in the addr.cfg file.
```
log_monitor 239.255.0.1:4000
_
```

Run the client

Run the Monte Carlo Pi client. The following output appears:

Mixing it up
It is entirely possible to call the one language version of the Monte Carlo Pi fabric application using a different language version of the client. For example, you can use the Java client to call the 'C' version of the fabric application. Just change the Java client to use the "monte_carlo_pi_cpp_app" fabric application instead of the "monte_carlo_pi_java_app" version.

This is made possible by the fact that our application only uses primitive data types. If you have a fabric client and application that passes objects (for example Java client passing Java objects to/from a Java-based fabric application), then this would, of course, not work.

When using Java and Ant
You can issue the command "ant run" to execute the client.