Using Caché Objects
Using the Object Synchronization Feature
This appendix describes the object synchronization feature, which you can use to synchronize specific tables in databases that are on occasionally connected
systems. This appendix includes the following sections:
When viewing this book online, use the preface
of this book to quickly find other topics.
Introduction to Object Synchronization
Object synchronization is a set of tools available with Caché objects that allows application developers to set up a mechanism to synchronize databases on occasionally connected
systems. By this process, each database updates its objects. Object synchronization offers complementary functionality to Caché system tools that provide high availability and shadowing. Object synchronization is not designed to provide support for real-time updates; rather, it is most useful for a system that needs updates at discrete intervals.
For example, a typical object synchronization application would be in an environment where there is a master copy of a database on a central server and secondary copies on client machines. Consider the case of a sales database, where each sales representative has a copy of the database on a laptop computer. When Mary, a sales representative, is off site, she makes updates to her copy of the database. When she connects her machine to the network, the central and remote copies of the database are synchronized. This can occur hourly, daily, or at any interval.
Object synchronization between two databases involves updating each of them with data from the other. However, Caché does not support bidirectional synchronization as such. Rather, updates from one database are posted to the other; then updates are posted in the opposite direction. For a typical application, if there is a main database and one or more local databases (as in the previous sales database example), it is recommended that updates are from the local to the main database first, and then from the main database to the local one.
For object synchronization, the idea of client and server is by convention only. For any two databases, you can perform bidirectional updates; if there are more than two databases, you can choose what scheme you use to update all of them (such as local databases synchronizing with a main database independently).
This section addresses the following topics:
To ensure that updates work properly, each object in a database should be uniquely distinguishable. To provide this functionality, Caché gives each individual object instance a GUID
a globally unique ID. The GUID makes each object universally unique.
The GUID is optionally created, based on the value of the GUIDENABLED
parameter. If GUIDENABLED
has a value of 1, then a GUID is assigned to each new object instance.
Consider the following example. Two databases are synchronized and each has the same set of objects in it. After synchronization, each database has a new object added to it. If the two objects share a common GUID, object synchronization considers them the same object in two different states; if each has its own GUID, object synchronization considers them to be different objects.
Each update from one database to another is sent as a set of transactions. This ensures that all interdependent objects are updated together. The content of each transaction depends on the contents of the journal for the source
database. The update can include one or more transactions, up to all transactions that have occurred since the last synchronization.
Resolution of the following conditions is the responsibility of the application:
If two instances that share a unique key have different GUIDs. This requires determining if the two records describe a single object or two unique objects.
If two changes require reconciliation. This requires determining if the two changes were to a common property or to non-intersecting sets of properties.
The SyncSet and SyncTime Objects
When two databases are to be synchronized, each has transactions in it that the other lacks. This is illustrated in the following diagram:
Two Unsynchronized Databases
Here, database A and database B have been synchronized at transaction 536 for database A and transaction 112 for database B. The subsequent transactions for each database need to be updated from each to the other. To do this, Caché uses what is called a SyncSet
object. This object contains a list of transactions that are used to update a database. For example, when synchronizing database B with database A, the default contents of the SyncSet
object are transactions 547, 555, 562, and 569. Analogously, when synchronizing database A with database B, the default contents of the SyncSet
object are transactions 117, 124, 130, and 136. (The transactions do not use a continuous set of numbers, because each transaction encapsulates multiple inserts, updates, and deletes which themselves use the intermediate numbers.)
Each database holds a record of its synchronization history with the other. This record is called a SyncTime
table. For database, its contents are of the form:
Database Namespace Last Transaction Sent Last Transaction Received
B User 536 112
The numbers associated with each transaction do not provide any form of a time stamp. Rather, they indicate the sequence of filing for transactions within an individual database.
Once database B has been synchronized with database A, the two databases might appear as follows:
Two Databases, Where One Has Been Synchronized with the Other
Because the transactions are being added to database B, they result in new transaction numbers in that database.
Analogously, the synchronization of database B with database A results in 117, 124, 130, and 136 being added to database A (and receiving new transaction numbers there):
Two Synchronized Databases
Note that the transactions from database B that have come from database A (140 through 162) are not
updated back to database A. This is because the update from B to A uses a special feature that is part of the synchronization functionality. It works as follows:
Each transaction in a database is labeled with what can be called a database of origin.
In this example, transaction 140 in database B would be marked as originating in database A, while its transaction 136 would be marked as originating in itself (database B).
method, which bundles a set of transactions for synchronization, allows you to exclude transactions that originate in a particular database. Hence, when updating from B to A, AddTransactions()
excludes all transactions that originate in database A because those have already been added to the transaction list for database B.
This functionality prevents creating infinite loops in which two databases continually update each other with the same set of transactions.
Modifying the Classes to Support Synchronization
Object synchronization requires that the sites have data with matching sets of GUIDs. If you are starting with an already existing database that does not yet have GUIDs assigned for its records, you need to assign a GUID to each instance (record) in the database, and then make sure there are matching copies of the database on each site. In detail, the process is:
For each class being synchronized, set the value of the OBJJOURNAL
parameter to 1.
Parameter OBJJOURNAL = 1;
This activates the logging of filing operations (that is, insert, update, or delete) within each transaction; this information is stored in the ^OBJ.JournalT
global. An OBJJOURNAL
value of 1 specifies that the property values that are changed in filing operations are stored in the system journal file; during synchronization, data that needs to be synchronized is retrieved from that file.
can also have a value of 2, though the possible use of this value is restricted to special cases. It is never
for classes using the default storage mechanism (%CacheStorage
). A value of 2 specifies that property values that are changed in filing operations are stored in the ^OBJ.Journal
global; during synchronization, data that needs to be synchronized is retrieved from that global. Also, storing information in the global increases the size of the database very quickly.
Optionally also set the value of the JOURNALSTREAM
parameter to 1.
Parameter JOURNALSTREAM = 1;
By default, object synchronization does not support synchronization of file streams. The JOURNALSTREAM
parameter controls whether or not streams are journaled when OBJJOURNAL
is true and OBJJOURNAL
is true, then streams are journaled. Object synchronization tools will process journaled streams when the referencing object is processed.
For each class being synchronized, set the value of its GUIDENABLED
parameter to 1; this tells Caché to allow the class to be stored with GUIDs.
Parameter GUIDENABLED = 1;
Note that if this value is not set, the synchronization does not work properly. Also, you must set GUIDENABLED
for serial classes, but not for embedded objects.
For each class being synchronized, give each object instance its own GUID by running the AssignGUID()
Set Status = ##class(%Library.GUID).AssignGUID(className,displayOutput)
is an integer where zero specifies that no output is displayed and a nonzero value specifies that output is displayed.
Put a copy of the database on each site.
Performing the Synchronization
This section describes how to perform the synchronization. The database providing the updates is known as the source database
; and the database receiving the updates is the target database
. To perform the actual synchronization, the process is:
Each time you wish to synchronize the two databases, go to the instance with the source database. On the source database, create a new SyncSet using the %New()
method of the %SYNC.SyncSet
Set SrcSyncSet = ##class(%SYNC.SyncSet).%New("unique_value")
The integer argument to %New()
, should be an easily identified, unique value. This ensures that each addition to the transaction log on each site can be differentiated from the others.
specifies a namespace within a database whose transactions are not included in the SyncSet.
This method collects the synchronization data and puts it in a global, ready for export.
Or, to synchronize all transactions since the last synchronization, omit the first and second arguments:
This gets all transactions, beginning with the first unsynchronized transaction to the most recent transaction. The method uses information in the SyncTime
table to determine the values.
Set ExcludedDB = $ListBuild(GUID,namespace)
Call the ErrCount()
method to determine how many errors were encountered. If there have been errors, the SyncSet.Errors query provides more detailed information.
Export the data to a local file using the ExportFile()
is the file to which the transactions are being exported; it is a name with a relative or absolute path.
specifies whether or not the method writes output to the current device. Specify d
for output or -d
for no output.
is a boolean value that specifies whether or not the SyncTime
table is updated (where the default is 1, meaning True). It may be helpful to explicitly set this to 0 at this point, and then set it to 1 after the source receives assurance that the target has indeed received the data and performed the synchronization.
Move the exported file from the source machine to the target machine.
Create a SyncSet
object on the target machine using the SyncSet.%New()
method. Use the same value for the argument of %New()
as on the source machine this is what identifies the source of the synchronized transactions.
Read the SyncSet
object into the Caché instance on the target machine using the Import()
Set Status = TargetSyncSet.Import(file,lastSync,maxTS,displaymode,errorlog,diag)
is the file containing the data for import.
is the last synchronized transaction number (default from synctime table).
specifies whether or not the method writes output to the current device. Specify d
for output or -d
for no output.
provides a repository for any error information (and is called by reference to provide information for the application).
provides more detailed diagnostic information about what is happening when importing
This method puts data into the target database. It behaves as follows:
If the method detects that the object has been modified on both the source and target databases since the last synchronization, it invokes the %ResolveConcurrencyConflict()
callback method; like other callback methods, the content of %ResolveConcurrencyConflict()
is user-supplied. (Note that this can occur if either the two changes both modified a common property or the two changes each modified non-intersecting sets of properties.) If the %ResolveConcurrencyConflict()
method is not implemented, then the conflict remains unresolved.
If, after the Import()
method executes, there are unsuccessfully resolved conflicts, these remain in the SyncSet
object as unresolved items. Be sure to take the appropriate action regarding the remaining conflicts; this may involve resolution, leaving the items in an unresolved state, and so on.
method returns a status value but that status value simply indicates that the method completed without encountering an error that prevented the SyncSet
from being processed. It does not indicate that every object in the SyncSet
was processed successfully without encountering any errors. For information on synchronization error reporting, see Import()
in the class reference for %SYNC.SyncSet
Once the first database updates the second database, perform the same process in the other direction so that the second database can update the first one.
Translating Between GUIDs and OIDs
To determine the OID of an object from its GUID or vice versa, there are two methods available:
is a class method of the %GUID
class that takes a GUID of an object instance and returns the OID associated with that instance.
is a class method of the %Persistent
class that takes an OID of an object instance and returns the GUID associated with that instance; the method can only be run if the GUIDENABLED
parameter is TRUE for the corresponding class. This method dispatches polymorphically and determines the most specific type class if the OID does not contain that information. If the instance has no GUID, the method returns an empty string.
Manually Updating a SyncTime Table
To perform a manual update on the SyncTime table for a database, invoke the SetlTrn()
method, which sets the last transaction number:
Set Status=##class(%SYNC.SyncTime).SetlTrn(syncSYSID, syncNSID, ltrn)
method sets the highest transaction number synced in on the target system instead of the default behavior (which is to set the highest transaction number exported from the source system). Either approach is fine and is a choice available during application development.