Data Synchronization Overview: The Exchange Data Model

When developing mobile software solutions, one of the primary considerations is the most efficient way in which to synchronize production data.

While all client-server systems must account for this, mobile software development presents its own set of challenges, which stem from the almost universal truth that, at some point, mobile users will need to work in a disconnected environment.

Users will not always be connected, and in many cases will spend most of their day without network connectivity. Therefore, when users do synchronize their clients, information must be resolved concerning what data a user needs. It can be extremely inefficient to attempt to retrieve all production data during synchronization. Most production applications contain large amounts of data stored on the client devices and attempting to retrieve everything during synchronization can result in long delays during the synchronization process. This is unnecessary in most environments, as much of the production data stored on the client is likely to still be current and accurate as compared to the data in the back end system.

An Agentry application project accounts for this within its structure. The architecture of all synchronization components allows the developer to be far more selective about what data needs to be retrieved during the synchronization process. The method recommended by Syclo is called the Exchange Data Model.

Exchange data is the term used to refer to information about what production data the client has and when it was last retrieved, as well as the data contained in the back end system and when it was last modified. With this information available the developer can implement synchronization processes that only retrieve information that has been modified since the last time a client synchronized with the Server. Any unchanged data is not retrieved. This model will result in quicker and more efficient synchronization for users, as well as reducing the amount of resources needed by the system as a whole during the synchronization process. All data definition types and their related synchronization components allow for and are intended to be used in an exchange data model.

The use of the exchange data model requires certain information be available during synchronization. This information can include:
  • The date and time when an object, complex table record, or data table was last downloaded to the Client.
  • The date and time when the data in the back end system was added or last modified.

For the date and time of data retrieval on the clients, the synchronization processing within Agentry provides the ability to retrieve, store and access the client-side information about when data was last retrieved. Objects, complex tables and data tables all have the ability to store what is called the “last update” value that represents the date and time data was retrieved.

For date and time values related to changes made on the back end system, mechanisms must exist or be added for the mobile application to track changes to data that occur in between users’ synchronizations.

During synchronization the exchange data about which data has been retrieved, changed, and added, and when those events occurred is put to use according to the following general process. Note that this applies only to downstream synchronization:
  1. The Agentry Client sends the information to the Agentry Server about what data it currently contains and when it was retrieved. This includes unique identifiers and the last update values for each data instance.
  2. The Agentry Server processes the client-side exchange information according to the synchronization definitions. This can involve adding the data to back end objects created specifically for the mobile data synchronization, or by using existing back end objects that suit these purposes.
  3. The Agentry Server next processes the synchronization definitions that determine what has changed in the back end system since the date and time for the client-side data. Comparisons are made between the client’s date and time values and the date and time values in the back end system that reflect when the back end data was last affected. Items with date and time values more recent than matching items on the client, or items added to the back end not currently residing on the client, are flagged for retrieval. Alternately, and depending on the synchronization methods specific to the type of back end system, the comparison and retrieval may be accomplished at the same time. This data is retrieved using synchronization definitions and returned to the Agentry Server.
  4. Data that should be removed from the client is determined separately from data that should replace or be added to the client. Definition types within the Agentry architecture exist to specifically look for and return items that should be removed from the client.
  5. The Agentry Server builds object instances, or complex table and data table records based on data returned to it. These instances are then sent to the client to be stored in their respective structures.
  6. Data to be removed from the client is denoted via it’s unique identifiers. The ID’s are sent by the Agentry Server to the client. The messaging sent includes instruction to the client to remove the denoted item from its respective data structure. This includes deleting object instances or removing complex table records. Note that individual records cannot be deleted from data tables, for reasons explained in the discussions specific to this definition type.

The specific methods and mechanisms for accomplishing the above tasks will differ as a result of a combination of different factors that include the type of production data being synchronized (objects, complex tables, or data tables), the type of back end system in use, the capabilities of a specific back end system, the specific needs of a given application, and the specific needs of an implementation of a given application.

Regardless of the technical details of how the above steps are accomplished, the following summary of the exchange data model holds true. Begin by determining what the client has and when it received it. Next use this information to determine what is different in the back end system and when it was changed. Finally, retrieve only the data which is different on the back end. Any other data in the back end system can be ignored as it has not been modified and therefore is accurate and still current on the client.