The Fabric Accessible Memory is fabric-based storage for applications to maintain and share data for inter-fabric requests. The FAM is dependable as it adheres to fabric principles.

FAM storage model

The FAM is a data store for your application's data attributes. The fundamental unit of data within the FAM is a single name/value pair. You may read and write any name/value pair to the FAM, and depend of the FAM for reliability and integrity. You may store any of the base data types within the FAM, for example, strings, integers (32-bit and 64-bit), float, double, and binary. Data stored in the FAM can be accessed by any worker in any request. This makes the FAM an ideal place to hold the following types of data.

• Data elements crossing multiple FAM requests. Applications that use multiple requests to perform an operation, for example, shopping carts, can store intermediate data values with in the FAM so to prevent excessive data coping between the fabric and user interface.

• Application level data attributes. Any application wide data, for example, status flags and counters, can be stored in the FAM for shared access within the fabric.

FAM space

A space is a collection of keys distributed across one or more workers. All keys written to the FAM are placed into a space. Keys must have unique identifiers within a space. Key identifiers may be repeated across spaces.

The number of workers storing a space is dependent on the parity scheme you choose when creating the space. When creating, you must also choose the space size. The maximum size of a space is limited to the amount of available RAM on any single worker within the fabric. The workers within the space are responsible for watching each other's health and healing when workers are lost.

Parity scheme

A parity scheme (n,k) defines how the FAM distributes data across multiple workers, where n is the number of workers that must survive a failure to successfully recover the data and k is the total number of workers storing the data. FAM supports the (1,1), (2,1), and (2,1) with quorumless read parity schemes.

In the (1,1) parity scheme, only one worker stores the data. If that worker fails, the data is lost. This scheme is most useful for developing fabric applications on one worker and fabric applications that use the FAM as a data cache that can easily be recreated if the data is lost. The (1,1) parity scheme requires one worker.

In the (2,1) parity scheme, the data is mirrored on two workers. If one worker fails, the fabric automatically copies the data to a second worker. If one worker fails and the second worker fails before the copying process is complete, the data is lost. In this parity scheme, reads are processed only if the FAM space consists of two workers. The (2,1) with quorumless read parity scheme is the same as the (2,1) parity scheme except that reads are processed even if the FAM space consists of only one worker. Data reads are slightly faster in this parity scheme. Both (2,1) parity schemes require three workers.

Choosing a parity scheme

The parity scheme for a FAM space depends on your needs. If the speed of writing and retrieving data is more important than ensuring data integrity and preventing data loss, then a less secure parity scheme is appropriate. If the speed of writing and retrieving data is just as important as ensuring data integrity and preventing data loss, then a more secure parity scheme is appropriate.

Managing spaces

The ability to manage spaces includes create space, delete space, and query space. The create space function requires you to specify the space name, size, and parity scheme for a space. The fabric locates workers with appropriate resources and allocates the space. The delete space function removes the space from the fabric and all data elements with the space. The query space function returns the list of existing spaces within the fabric.

To set up the fabric with the appropriate space structure for your application, create a process flow within your application that calls upon the FAM API to create and initialize the space. Appistry highly recommends restricting this process flow to fabric administrative users and only calling it after the application is initially deployed to the fabric.

Note. DO NOT include a create space function in an onStartup task method. This can result in multiple executions of the space creation, one for each worker in the fabric.

Deleting a space is similar to creating a space. Create a process flow to clean up your FAM usage and call the process flow when the application is about to be removed from the fabric.

Note. Do not create and/or delete spaces on a split fabric. Calling create on each side of a split fabric can result in multiple spaces with the same name when the fabric rejoins. In the event that your joined fabric has multiple, same-named spaces, you must delete all same-named spaces and recreate them. In the event of any error during any of these activities, you will be notified that the action did not occur.

Data integrity

All interactions with the FAM are performed in the context of a session. The session provides ACID level properties to actions performed within the body of a session.

• FAM updates within a session are all or nothing. All the updates succeed or none of the updates succeed.
• The update occurs according to the application logic.
• FAM updates are isolated between sessions. Only committed data is available to other sessions. Inter-session updates are only available to the current session.
• The fabric backs up intermediate state information to multiple workers. This data is only made live when the session is closed.

The application developer can create a session in two ways:

• Task defined sessions - Fabric automatically creates a session whenever the developer accesses the FAM within the body of a task running within a process flow. Upon completion of that task, the session is committed. In the event of application level errors, you may explicitly call rollback ending the session.
• Developer defined sessions - You are able to use the FAM API to create and close single sessions.

FAM queues

The FAM provides a shared queue construct for applications that need transaction ordering and/or load sharing. Applications can create a first-in first-out (FIFO) queue stored in the fabric memory. Any application in the fabric can push a new element to the back of the queue. Any request in the fabric can wait on the queue for work and process it accordingly.

The FAM queue provides a excellent data structure for producer-consumer style applications where incoming load would otherwise overwhelm the fabric. You may create process flows that preprocess incoming data and place the data on the queue. Other background tasks and process flows can then wait on the queue for work to process and drain the queue in a orderly manner.

Unlike other FAM data types, the queue is a fully distributed data structure. Multiple requests are able to push to the queue simultaneously. Multiple requests are able to attempt to pull from the queue, although only one request can own the front of the queue at a time.

Pushing and popping from the queue adheres to the same session rules as other data types. All queue access must happen within the scope of a session. At the end of a session, all work is committed or rolled back. Only committed pushes may be pulled from consumer oriented requests.

The basic queue operations include push, pop, unpop, and delete.

• push adds a new data element to the end of the queue
• pop retrieves and removes the front element from the queue
• unpop replaces items on a queue in the order in which they were popped
• delete removes the entire queue from the FAM

FAM queues do not need to be created. Calling push or pop activates the appropriate actions in the FAM.

The pop operation has an associated timeout. The pop function either retrieves the first data element on the queue or times out because no elements are on the queue.

With these operations, you may prepare a large data set for processing by placing data elements on the queue. For example, in a batch oriented application, you may place individual file names for processing on the queue. These elements can be retrieved and processed using a combination of background tasks and process flows. The fabric executes as many background tasks as you specify. Each of these tasks can execute a process flow, through the client API, that pulls an element off the queue for processing. This prevents overwhelming the fabric while effectively maintaining the queue.

Rolling updates

The fabric attempts to respect existing spaces during shutdowns and fabric updates. Fabric updates result in the graceful restart of every worker in an organized manner.

During application, system, and image updates, the fabric attempts to protect FAM spaces by only permitting one member of each space to restart at a time. During the restart, (2,1) spaces continue to provide data access even as one of the workers in the space is being updated. For (1,1) spaces, data is lost during the roll. This is another reason why (1,1) spaces should only be used for development purposes or for temporary data caches that are easily repaired by the application.

If the administrator attempts to shutdown or change the address of single workers within the fabric, the fabric preserves all spaces on those workers by allowing the data to move to other workers. If the fabric is at FAM capacity, the Fabric Control utility returns an error indicating the worker cannot be moved or shutdown because the spaces in the FAM could not be moved. If this occurs, the administrator should add new workers.

If the administrator issues a shutdown all command through Fabric Control, the fabric immediately terminates the FAM and shutdowns all workers. In this case, all data within the FAM is erased.