Release and Upgrade Information for Caché 2015.2

DeepSee Improvements

This release provides additional improvements in DeepSee capabilities, namely:

New controls provided by the dashboard editor include:

• Title bar: color, opacity, hide/show

• Tool bar: color, opacity, hide/show

• Widget borders: hide/show

• Legend: color, opacity

• Dashboard background: image, color, opacity

Prior to this release, listings had to be defined as part of a cube. If you wanted to allow someone to create new listings, you had to give them access to edit the cube definition, which would allow them to change anything in the cube definition.

In this release, you can allow someone to create new listings without giving them access to cube definitions. Listings can now be part of a new entity called a Listing Group. A Listing Group is a class that contains 1 or more listings. Listing Groups can be edited with Studio or the new Listing Group Manager that is available in the DeepSee Tools menu. Access to the Listing Group Manager is controlled via security resources. Details on Listing Groups are available in the DeepSee material.

This release provides a new feature that allows you to upgrade a cube with minimal impact on current users.

Changes to a cube definition require the cube to be rebuilt. Prior to this release, the first step of rebuilding a cube was to always delete the existing cube. So while the cube was rebuilding, queries on the cube were prohibited. When the rebuild completed, you could again query the cube. Depending on the size of the cube and when the rebuild happened, the cube may have been unavailable longer than is acceptable to users.

This release provides a new optional feature for defining cube versions. With cube versions, the existing cube is not deleted at the start of the rebuild. While the new (pending) version of a cube is being built, the current (older) version of the cube is still available for queries. After the new version of the cube finishes building, you run a method that makes the pending cube the current cube and deletes the older cube.

Support For QR Codes

QR (Quick Response) codes have become a popular mechanism for presenting information in an easily-consumed format. They are used for URLs, business cards, setup information, and other short segments. With this release of Caché, application developers can now use the methods of the %SYS.QRCodeOpens in a new tab class to generate their own QR codes. InterSystems itself uses QR codes in the new two-factor, time-based, one-time password (TOTP) implementation.

New iKnow Language Models For iKnow: Russian And Ukrainian

iKnow now includes language models for Russian and Ukrainian, adding to the languages already supported: English, Spanish, Portuguese, German, Dutch, and French. It now also supports stemming, allowing you to work with stems as a level of abstraction above entities in languages with a full case system like Russian and Ukrainian.

Support For Composite Search Strings In iFind

When using iFind for full-text search, you can now submit composite search strings, using parentheses and Boolean operators (AND, OR, and NOT) to express complex criteria in a single search string. For example, passing the string “(long strings) AND (memory OR disk*) AND NOT {error reference}” to the search_index() method in SQL will yield all records that contain the word sequence “long strings” as well as the word “memory” or a word starting with “disk”, excluding the records that contain the entity “error reference”.

Other Improvements In iKnow And iFind

In this release, iKnow and iFind also include many other improvements and simplifications for working with unstructured data, including the following new features:

• iFind now supports stemming and decompounding for all index levels.

• You can now define and query iFind Basic indices for any language that uses spaces as word separators if no iKnow language model is specified or available.

• Both iKnow and iFind leverage an improved algorithm for calculating the semantic dominance of entities in a record and will consume less disk space for storing these metrics.

• You can now customize attribute detection (negation, sentiment) by supplying marker terms the engine should include when identifying the presence and scope of these semantic attributes.

Support For Islamic Calendar Dates

In this release, Caché adds support for dates expressed according to the Islamic calendarOpens in a new tab (Hijri). Dates may be specified in accord with either the calculated or observational systems; both are supported. Users can determine and store the observational result to determine the length of a given month. In addition to providing this feature to application developers, DeepSee also takes advantage of this new capability.

Database Caching Internal Optimizations

This release expands the scalability of database references on large multi-core systems by substantially reducing runtime contention in the management of the database cache.

Dynamic Allocation Of Global Vectors

This release substantially improves the performance of global references for processes that repeatedly access more than 32 different globals. The performance improvement comes from an increase in the number of global vectors that a process allocates. Each global vector contains information about a different global that the process accessed recently, and this information is used to optimize its next reference to that global. Previously, processes were limited to 32 global vectors. Now, processes allocate global vectors dynamically as needed, up to a maximum of 1000. For more detail, see the Upgrade Checklist.

ECP Concurrency Improvements

This release expands the scalability of large multi-core ECP data servers. Changes made to the server reduce the runtime contention associated with managing the database cache on behalf of ECP clients.

Use Power 8 Encryption Hardware

When running on the IBM AIX® platform, Caché checks for the presence of encryption hardware. If present, it performs encryption using the new Power 8 In-Core option to maximum performance when processing encrypted data. If the hardware is not present, Caché performs the encryption in software as it does on prior releases.

Fast Verification Of Recent Database Blocks After Host Failure

This release introduces additional verification within Caché startup on Windows and UNIX® to check for physical database inconsistencies on recovery after a host failure. The verification runs for a limited amount of time so as not to impact availability; it does not change normal operating procedures. If inconsistencies are detected, it is typically evidence of a storage device problem, and startup waits for user intervention in order to protect the system from running in an uncertain state. See “WIJ Block Comparison” in the Caché Data Integrity Guide for more information.

Mirroring Improvements

This release of Caché has improved mirroring support and capability, namely:

• Quicker failover

  The time for automatic mirror failover to complete has been improved. The improvements are most substantial in the event where the primary host crashes or becomes network-isolated. When that happens, the failover time is reduced by approximately 15 seconds. In many environments, this represents the largest portion of the failover time and reduces the time from detection of an outage to the databases being available on the new primary to several seconds.

• Simpler authorization of new members when using SSL/TLS

  Async Mirror Members using SSL no longer require manually entering the Distinguished Name to authorize their membership. Instead, they are now added to the mirror in the same way that a second failover member is added: the administrator approves the new async member from the primary.

Non-Root Installations

Starting with this release, the installer for Caché and Ensemble can be run without requiring root or administrator privileges. In this case, the installation belongs to the installing user who has full access to and control of the instance. The registry entries for this instance also belong to this user account only; it is not shared with other users or the root user. Please refer to the Installation Guide for more detailed instructions and information.

New Startup Default And Journal Database Encryption Key Management

The Caché database encryption facility has two settings that define default key usage behavior:

1. the database encryption key to be used when creating new encrypted databases (including CACHETEMP), and

2. the key for new encrypted journal files.

The initial values for these settings for a Caché instance are determined the first time database encryption keys are activated, and are derived from material in the encryption key file used. They can be changed at any time using the management portal or ^EncryptionKey. The current values are preserved across Caché shutdowns and restarts.

Two-Factor Time-Based One-Time Password Authentication

Beginning with this release, Caché supports Two-factor Time-based One-time Password authentication (TOTP). Please refer to Configuring for Two-factor Authentication for more information.

Users who have scripts which create or modify users using ^SECURITY or the Security.UsersOpens in a new tab class may need to modify the scripts to add the new fields needed by this capability.

Remote System ID Is Now ECP System ID

In the Management Portal and the routine, ^JRNDUMP, the field that was formerly labeled “Remote System ID” has been changed to be“ECP System ID”. The value in that field is now the ECP System ID instead of the Remote System ID. The two values differ only in the top few bits, which are zeros in the ECP System ID but typically non-zero in the Remote System ID. Also ECPSystemID is now present as a new property of a %SYS.Journal.RecordOpens in a new tab object and a new column in the List query of the class.

System Monitor Changes

System Monitor extends the Health Monitor sensor object to %SYS via 2 new classes: %SYS.Monitor.Sensor() and %SYS.Monitor.SensorItem(). These classes provide alerting not only on Sensors, but can also turn on alerting and set alert values for individual sensor items. For example, DiskPercentFull (critical = 99%, warn = 90%) is the default for all databases. But this can be modified or turned off on an individual database such as /mgr/cachelib.

Also, Notifications and SensorReadings are no longer stored in the multidimensional items SensorReadings and Notifications. Instead the interfaces provided must be used to access notifications and sensors: %SYS.Monitor.AbstractComponent.GetNotification() and %SYS.Monitor.AbstractSubscriber.GetNextSensor(), respectively.

Mirroring

• More Stringent Checks For Mirror Member Names

  The following characters (and 2–character combinations) are no longer valid for mirror member names. Their presence will cause a validation failure when the configuration class is saved and an error will be returned to the user to avoid future problems:

  #    ;    //    /*    =    ^    ~    "    <space>    <tab> 
  

  Important:

  Systems that used one of the prohibited special characters in a mirror member name will need to change the member names before upgrading or the upgrade will fail because the .cpf file will fail validation.

• Newly Created Mirrors Use a Longer Quality of Service Timeout

  In this release, the default Quality of Service for newly created mirrors is now 8 seconds. The previous default of 2 seconds was too short to account for certain events at the host or network level that could cause a system to become temporarily unresponsive; this resulted in alerts and unwanted failovers. Existing mirrors that are upgraded will remain unchanged. On some hardware configurations, particularly in virtual environments, the previous default of 2 seconds was too short to account for the timing of events at the host or network level; this caused the system to be unresponsive resulting in alerts and unwanted failovers. Refer to Configuring the Quality of Service (QoS) Timeout Setting for more detail.

• Mirror Journal File Purge Policy

  There are two changes to the journal file purge policy that are designed to allow more flexibility while retaining journal files needed by the mirror.

  1. In prior versions, the backup failover member would not purge any journal file that the primary was retaining. Now, the number of days to retain journal files is honored independently for each member. In this way, members can retain differing amounts of journal history, as long as those journal files are not needed by other members of the mirror.

  2. Also in prior versions, all async mirror members, including disaster recovery (DR) members were allowed to purge journal files as soon as they had been applied to the databases. For DR members, this could cause problems when other members tried to connect to a DR member after it had been promoted. Now, DR members follow the same mirror journal purge rules as failover members.

  System administrators should review their configuration to ensure that DR members have sufficient space to retain the configured amount of journal files. A DR member must be configured with enough journal space to act as primary in the event of a disaster. While this requirement is not new in this version, DR members with insufficient space would have gone unnoticed in previous versions until the member was promoted.

• Enforce Restriction For Changing Reporting To DR Async Member Type

  Caché now places restrictions on member type conversion between DR, Read-Only Reporting, and Read-Write Reporting async members. The DR can be changed to Reporting async member without any restriction, and there is also no restriction between Read-Only and Read-Write Reporting members. However, a Reporting type cannot be changed to DR type when there is any mirrored DB has FAILOVERDB attribute cleared. Caché also does not allow Reporting async member to become DR, if the ISCAgent is not running.

• Mirror Names May Not Differ Only In Case

  Caché no longer allows users to create mirrored databases with a name that differs from existing mirrored databases only in the case of the name. For example, if there is an existing mirrored database with a name of “Test”, then Caché will prevent the creation of a database with a name of “TEST”. Beginning with this release, mirror names will be converted to uppercase.

SHA256 Added As Default Signature Hash

The DefaultSignatureHash property has been added to the System.Security class to allow specifying the default hashing algorithm to be used. The default is SHA256. The XML Signature is enhanced to use the new System.Security property to determine the default hashing algorithm if none is explicitly specified. The Management Portal was enhanced also to allow DefaultSignatureHash to be displayed and modified on the “System-wide Security Parameters” page. And, the default for AlgorithmSuite in the SOAP Configuration wizard is changed to Basic128Sha256.

If the receiver of the signature which is created with the new default of SHA256 does not support SHA256, then the customer will need to either explicitly set the DigestMethod in his code or change DefaultSignatureHash to be “SHA1”. The actual XML Signature is self-describing so that if the receiver supports “SHA256” (which all standard toolkits do), then code will continue to work and be more secure.

Error Text Differences

The text of error messages describing the failure of attempts to access a subscripted variable now provide more detailed information. For example, in previous releases the error would look something like this:

    <SUBSCRIPT>TEST1+5^TEST1 *a(1,"hello","")    

when the third subscript was empty. In this release, the error text will look like this:

    <SUBSCRIPT>TEST1+11^TEST1 *a() Subscript 3 is "" 

Other variants of the message are:

    <SUBSCRIPT>TEST1+65^TEST1 *a() Subscript 5 > 511 chars
    <SUBSCRIPT>TEST1+24^TEST1 *a() Encoded subscript 1 > 511 bytes 

These new messages provide more information about which subscript in the array is failing and the reason for the failure.

Shadowing DejrnCOS Setting Eliminated

In prior versions, an undocumented setting allowed shadowing to use an alternate (older) implementation of dejournaling referred to as “DejrnCOS”. In this release, the setting is removed and no longer available. If shadowing is started with this setting present, it is removed and converted to the standard implementation of dejournaling. Shadows converted in this way will be set to run with zero prefetcher jobs; this most closely matches the runtime characteristics of “DejrnCOS”.

Use Of $CHAR(1) As Monthlist Delimiter Invalid

As a result of changes made to accommodate the Hijri (Islamic) calendar capability, the use of $CHAR(1) as the delimiter in the monthlist passed to $ZDATE, $ZDATEH, $ZDATETIME, and $ZDATETIMEH is now prohibited. Applications using $CHAR(1) to separate the month names must choose a new character as the delimiter.

Application Licensing Entries Updated

Prior to this version, the $System.License methods TakeApplicationLicense() and ReturnApplicationLicense() assumed that only CSP server processes would take an application license for a CSP session. This assumption is invalid. The class and the underlying methods have been corrected. A description of the argument list and behavior is in the online documentation of class %SYSTEM.LicenseOpens in a new tab.

Error Trap Code Runs Using Current Roles

$ZTRAP * (or $ETRAP *) indicates that the code in the error trap should be run at the same level as the code that caused the error. In previous versions, when control was dispatched to the error trap code, $ROLES was reset to the state it was in when the process began. This means that if application roles were assigned, and the application set an error trap expecting it to have the application roles, that expectation was not met and the error trap might not function properly.

Beginning with this release, the error trap will be run with the roles that were in effect for the execution level at which the error trap was set.

Upgrades Overwrite Manually Updated Zen Mojo

Upgrading a Caché instance may downgrade the installed Zen Mojo version if it was updated manually since the last time the instance was installed or upgraded. Consider the following sequence of events:

• Caché 2015.1 ships with Zen Mojo version <N> preinstalled.

• Zen Mojo version <N+1> is released.

• Caché 2015.1.1 is released and it contains Zen Mojo version <N+1>.

• Zen Mojo version <N+2> is released. The user manually updates the instance of Caché 2015.1 with Zen Mojo version <N+2>.

• Later, that instance is updated to Caché 2015.1.1. The installer updates Zen Mojo back to version <N+1>.

In this case, the user must manually re-install Zen Mojo version <N+2>.

Relationship Storage Moved To Objects

The temporary structures that keep track of relationships among objects (for example, indexes for parent-child tables) are no longer held in process-private globals as they were in prior versions. With this release, they are now allocated as part of the parent and child objects. This increases speed of access at the cost of a small increase in the space taken up inside a partition. Systems whose memory is managed tightly may need to increase the memory allocation for the partition.

Increase In Global Vector Cache Size

Each process caches information about the globals that it uses in order to optimize performance. This cache is referred to as global vectors, and is stored in process memory; it is counted against the memory usage of the process. In previous versions, the number of global vectors per process was limited to 32. With this release, the limit has been raised to 1000. Moreover, it is now dynamically allocated, expanding as different globals are used. (A global vector is used for each global having a different global name, or stored in a different physical database.) This change substantially improves performance when a process repeatedly accesses more than 32 different globals.

Application processes that reference many different globals will now consume more physical memory than in previous versions. The memory allocated to each global vector entry is approximately 750 bytes. Processes that reference the largest number of unique global names can allocate as much as 1000 global vectors, or 750KB. If your application uses many different globals, and is configured with a highly restrictive setting for Maximum Per-Process Memory or $ZSTORAGE, you may need to increase that setting to avoid <STORE> errors. See the article on Caché Process Memory for additional information.

As part of this change, the vectors configuration parameter is now ignored and will be removed in a future version.

TLS v1.1 and v1.2 are Disabled by Default

In Caché 2015.2, you can explicitly enable TLS v1.0, v1.1, and/or v1.2 on the SSL/TLS configuration page. In Caché 2015.1, you could only enable TLS v1.0 on the SSL/TLS configuration page, but TLS v1.1 and v1.2 were enabled by default and could be used.

If you upgrade from Caché 2015.1 to Cache 2015.2 and have existing SSL/TLS configurations, TLS v1.1 and v1.2 will initially be disabled for those configurations. If you want these configurations to use TLS v1.1 and/or v1.2, you must go to the SSL/TLS Configuration page and enable TLS v1.1 and/or TLS v1.2.

To enable TLS v1.1 and/or TLS v1.2 for SSL/TLS configurations:

1. Select System Administration > Security > SSL/TLS Configurations.

2. Select the Edit link for each SSL/TTL Configuration requiring update.

3. Select the TLSv1.1 and/or TLSv1.2 check boxes and select Save.

Windows

• Installation Permission Change

  Caché installation on Windows adds an access control entry (ACE) to the installation directory that permits full access to all authenticated users. This ACE can be displayed with the ICACLS command. whose output looks like this:

  
  C:\intersystems\latest NT AUTHORITY\Authenticated Users: (F)
  NT AUTHORITY\Authenticated Users:(OI)(CI)(IO)(DE,Rc,WDAC,WO,GW,
      GE,GA,RD,WD,AD,REA,WEA,X,DC,RA,WA)
  
  

  Installation will now create a Caché instance group named Cache_Instance_<InstanceName> and add an ACE to the installation directory granting the instance group full access to the install directory. The installation process will also make the Caché service account a member of this group, granting it full control over the install directory.

  When a Caché service account is assigned or changed with

  cinstall setserviceusername <InstanceName> <username> <password>
  

  

  the new service account will be made a member of the instance group, creating the group if necessary, and adding the ACE to the installation directory if not already present. It is therefore necessary to use cinstall setserviceusername to change the Caché service user account rather than the Windows control panel applet.

  It is now possible to impose stricter limits on the access of Windows users to the Caché installation directory by removing the ACE granting access to all authenticated users. The following command can be used to remove the ACE. %CACHE_INSTALL_DIRECTORY% represents the path where the Cache instance is installed, for example “C:\InterSystems\Cache”.

  icacls %CACHE_INSTALL_DIRECTORY% /remove "NT AUTHORITY\Authenticated Users"
  

  

  If the ACE granting “NT AUTHORITY\Authenticated Users” access to the instance directories is removed, users of the local terminal connection to Caché on Windows must be granted access to the installation directory by being made members of the Caché instance group or must be administrators to guarantee that they have proper access to the files in and under the instance's installation directory. The local terminal connection can be recognized by the appearance of TRM:PID (InstanceName) in the title bar.

  Important:

  The modifications to the installation directory tree described here should not be applied to instances installed under earlier versions of Cache unless upgraded to a version with this change present.

• Installation To Global Assembly Cache

  As of this release, the installer will no longer place CacheProvider assemblies into the Global Assembly Cache (GAC) by default. To have assemblies installed into the GAC, one must set property ISCINSTALLINTOGAC to 1, either by passing a command line argument “ISCINSTALLINTOGAC=1”or by editing the MSI file in Orca or other similar tool.

• Changes To The Behavior Of $NOW On Multi-Core Windows Systems

  Prior to this release, Caché attempted to keep the value of $NOW consistent on systems with multiple cores. Under some circumstances, this could lead to more than one CPU attempting simultaneously to reset the time and result in $NOW generating a <FUNCTION> error.

  In this release, InterSystems has modified the code that adjusts the microsecond clock on Windows systems so that it will no longer make adjustments visible to all processes running on a Caché instance. Instead, each Caché Windows process will make its adjustments local to the individual process. A consequence of this is that the value returned from $NOW(...) on differing Caché processes can disagree with each other. While his may affect some applications that do detailed timing studies, Caché will no longer generate <FUNCTION> errors when calling $NOW(...).

UNIX®

• Recommendation to Set TZ Environment Variable for Linux Platforms

  On Linux platforms, InterSystems recommends that you set the TZ environment variable, which indicates the time zone. If this variable is not set, there can be significant degradation of the performance of Caché time-related functions. For details on setting this variable, see the man page for tzset.

• ABRT and Caché (Red Hat Linux Installation)

  Starting with RHEL 6.0, Red Hat introduced a utility called ABRT (Automatic Bug Reporting Tool) which is intended to intercept program crashes and store the relevant information (including a core dump, if one exists) in a centralized location for further analysis. Under normal circumstances, it does not interact with Caché and one can simply ignore it. However, in the following two situations, it may be desirable to change the default configuration for the needs of the site administrator.

  • A crash in Caché kernel — In the event of a Caché malfunction that causes a process to crash, it is useful to contact InterSystems and, if possible, send the core dump file to be analyzed. Depending on Caché installation options, the user running the process at the time of the crash and permissions of the current directory, a core dump may be created or not. If a core file is not created but the user would like to have one to be analyzed, please see the recommendations below.

  • A crash in the cstat utility — The cstat utility (ccontrol stat) accesses a number of tables in shared memory that can be rapidly changing. Depending on the system load and other circumstances, an inconsistent memory snapshot may cause cstat to crash. This type or crash is harmless and cstat has code to prevent a core dump from being created. However, ABRT may still record the event and copy a core file to its centralized database. To preserve disk space, the site administrator might want to prevent these crashes from being recorded.

  This following provides some basic information about ABRT.

  • There are two major ABRT versions, 1.x and 2.x. To determine which you are running, use the 'abrt-cli' command:

    $ abrt-cli --version
    

    

  • When a crash happens, check the ABRT directory to see if the event was recorded. To do so, use the applicable version-specific command:

    ABRT 1.x	$ abrt-cli --list
    ABRT 2.x	$ abrt-cli list

  • Each event (crash) gets its own directory under the base ABRT directory.

    ABRT 1.x	/var/spool/abrt
    ABRT 2.x	/var/tmp/abrt

  • ABRT sends lots of messages to /var/log/messages, so it might be useful to inspect this file (usually the very end) when investigating what happened to a certain crash.

  • The online ABRT documentation is available at https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/ch-abrt.htmlOpens in a new tab

  • If ABRT is somehow interfering with Caché and it is not needed for any other reason, the easiest way to deal with the situation is by disabling it. To do so, use the following command:

    # service abrtd stop
    

    

    If running ABRT 2.x (RHEL 6.2 or later) you also need to disable this related service:

    # service abrt-ccpp stop
    

    

    Besides, it is necessary to change the "core pattern" that tells the system the format of the core file name. Normally one uses "core.<pid>" and this is obtained as follows:

    # cat >/proc/sys/kernel/core_pattern 
      core.%p 
      <CTRL+D>
    

    

    To permanently disable ABRT (persistent over reboots), also execute the following:

    ABRT 1.x and ABRT 2.x	# chkconfig abrtd off
    ABRT 2.x	# chkconfig abrt-ccpp off

  • To instruct ABRT to save a Caché core dump, do all of the following instructions that apply:

    • Edit the configuration file:

      ABRT 1.x	/etc/abrt/abrt.conf
      ABRT 2.x	/etc/abrt/abrt-action-save-package-data.conf

    • If the Caché installation was done via cinstall script (most common), change the following parameter from 'no' to 'yes': ProcessUnpackaged = yes

    • If the Caché installation was done via RPM, change the following parameter from 'yes' to 'no': OpenGPGCheck = no

  • To instruct ABRT not to save a cstat core dump. If you enable ABRT to process unpackaged executables, core dumps will be created not only for the Caché executable but also for cstat. If you do not want the core dumps for cstat, do both of the following to exclude them:

    • Edit the configuration file (see notes above)

    • Update the BlackListedPaths parameter so that it includes the full path to cstat:

      BlackListedPaths = [retain existing list], <installation dir>/bin/cstat
      

      

  • To get the status of the ABRT daemon:

    $ service abrtd status
    

    

  • To restart ABRT:

    # service abrtd restart
    

    

• Asynchronous And Direct I/O For Database Writes On UNIX® Platforms

  Caché has been enhanced to use Asynchronous I/O for writes to database files (the files named CACHE.DAT) on AIX, HP-UX, Solaris, and Mac OS X. This is coupled with automatic use of direct I/O (or concurrent I/O as appropriate) instead of buffered I/O on a subset of these platforms. These changes are designed to optimize the disk I/O characteristics for database files, allowing higher throughput on busy systems at peak load. Of course, synchronization still occurs at key points to guarantee data integrity.

  The use of direct I/O (or concurrent I/O) instead of buffered I/O for database files may require attention in certain configurations. The following describes the change on each UNIX platform along with any considerations that apply:

  • AIX

    In addition to now utilizing asynchronous writes, Caché now uses concurrent I/O automatically for database files (concurrent I/O is the recommended form of direct I/O on AIX). Many AIX sites already configure databases to use concurrent I/O via the cio mount option. Sites that do not (thus were using buffered I/O in previous versions) should note the following:

    • Check that the sufficient database cache is configured. File system buffering may have allowed Caché to perform acceptably with an under-configured database cache in previous versions.

    • In unusual configurations, where an external command is used to read a database file while Caché is running, the external command may fail because the file is opened for concurrent I/O by Caché. An example is performing an External Backup using the cp command instead of a more sophisticated backup or snapshot utility. Mounting the file system with the cio option resolves this by forcing all programs to open the files with concurrent I/O.

  • HP-UX

    Caché continues to use buffered I/O for database files because HP-UX does not provide program-level control of concurrent I/O (the recommended form of direct I/O on HP-UX). Concurrent I/O can be enabled by mounting the OnlineJFS or VxFS filesystem with the cio mount option. Caché uses asynchronous I/O in this version to provide better write characteristics than previous versions, particularly when concurrent I/O is used (even though concurrent I/O is not required).

  • Solaris

    Caché now uses direct I/O automatically for database files located on UFS filesystems; this applies to NFS as well although NFS is not a recommended filesystem on Solaris. If using UFS (or NFS), check that sufficient database cache is configured. Filesystem buffering may have allowed Caché to perform acceptably with an under-configured database cache in previous versions. Direct I/O is not relevant to the ZFS filesystem.

    In all cases, Caché now uses asynchronous I/O to write to the database files.

  • Linux

    Caché continues to use buffered I/O for database files and does not use asynchronous I/O. This is unchanged from previous versions.

  • Mac OS X

    Caché uses buffered I/O, but now uses asynchronous writes for database files. No further considerations apply.

• Caché 2015.2 Upgrade Fails on AIX without Additional Libraries

  You must install additional C/C++ runtime libraries on AIX before installing or upgrading to Caché 2015.2. If these files are not present, the installer will not complete the upgrade.

  Caché 2015.2 AIX is compiled with version 13. According to IBM Support documentation, programs compiled with version 13 and that use encryption features require 3 runtime filesets from runtime package, IBM_XL_CPP_RUNTIME_V13.1.0.0_AIX.tar.Z, on machines where Caché is installed:

  • xlC.aix61.rte 13.1

  • xlC.rte 13.1

  • xlC.msg.en_US.rte 13.1

  Complete details on the package are available at IBM XL C/C++ Runtime for AIX 13.1Opens in a new tab.

Mac OS X

Programs that use the callin feature and incorporate cache.o must be linked by a C++-aware linker, such as via the C++ compiler, or by manually adding “-stdlib=libstdc++” to the makefile.

Diagnose Invalid Default Arguments

Previously, if the default value for a formal argument was too long or was not a valid expression, the ObjectScript compiler may not have identified this situation. Instead, a <LIST> error would be thrown when the entry point was called.

Now, the compiler will give a compile error on the entry point. If the entry point is called, it will throw <PARAMETER> or <SYNTAX> depending on how it is called.

Class Deletions

The following classes were present in the previous version and have been removed in this release

• %CSP.UI.Portal.Dialog — MirrorAuthorizedAsync

• %CSP.UI.Portal.Mirror.Dialog — AddFailover

• %Collection.MV — ArrayOfObjCN, ListOfObjCN

• %SYS.Monitor — AbstractAsyncSensor

• %iFind — EntityI

• %iFind.Stemmer — AbstractStemmer

• %iKnow.Objects — Utils

• SYS.Monitor — CSPGateway, TestAsync

• SYS.Monitor.Health — AbstractCallback

Class Component Deletions

The following class components have been moved or removed in this version from the class where they were previously found.

Method Return Changes

The following methods have different return values in this version of Caché:

• %Compiler.UDL.TextServices — GetTextAsFile

• %Library.RegisteredObject — %ConstructCloneInit

• %Library.RoutineMgr — getGlobalList

• %SYSTEM.Process — StoreErrorReason

• %UnitTest.Manager — LogStateEnd

• %iKnow.Filters.Filter — GetNextSrcId

• Config.CommonMultipleMethods — Create

• SYS.Database — Copy

• SYS.Shadowing.Shadow — GetLatency, LatencyGet

Method Signature Changes

The following methods have different signatures in this version of Caché:

Read-Ahead Added To TCP/IP For ADO.Net

In previous releases, Caché processed xDBC messages synchronously for many messages. This meant that when the client was processing the server was idle; and when the server was processing the client was idle. This was evident when benchmarking xDBC with the client and server on the same machine. Even though there were two processes, the application would never consume more than 100% of the CPU for the sum of both processes.

In this version, after the client reads the header of a server message “A”, it immediately requests the next server message to be sent (“B”). The client continues to process server message “A” as directed by the application, while the server prepares the next portion of the result set. If the application continues to process the result set, at some point it will reach the end of buffer “A” and will read the header of message “B”. The process will then repeat with message “B” and the client will request “C” …

Although the processing of the data will still logically occur in the same order, this new behavior may uncover timing issues due to the read ahead nature.

Improve Throughput In TCP/IP Data Transfer

In previous versions, when transferring lists of data to or from an instance, Caché used extensive locking to properly manage shared values used to interate over the lists. These locks slowed performance because of the number of accesses sometimes made to data items. In this version, changes to the way data is accessed eliminate the need to lock reads (gets), and writes are protected by channeling them through a single thread. This removes most locking from putting and getting data from the server via TCP/IP. While this improves performance, new obscured timing issues may be uncovered in applications.

Differences In The Transmission Of Double Values

This release contains minor changes to the format of $DOUBLE values when transmitted to clients running on earlier releases. The changes suppress insignificant digits; values transmitted by the server as “0.93” in previous releases are now sent as “.93”. $DOUBLE values may also differ in the least significant digits from prior releases because of changes in rounding algorithms.

Applications that depend on the exact characters or values being identical to those previously transmitted may need to be changed.

SQL Statistics Logging Changes

The error logging for SQLStats has changed. In previous releases, every error encountered while logging SQLStats would be stored in ^%sqlcq($Namespace, "PTools", "Error", $JOB, counter). Storing the error for every process in the global led to the global becoming quite large. The logging error trap has been changed; beginning with this release it only stores the last error for every job: ^%sqlcq($Namespace, "PTools", "Error", $JOB).

Also when you purge SQLStats for a namespace, as in

Do ##class(%SYS.PTools.SQLStats).Purge("SAMPLES")

the errors will also be purged. If you only purge statistics for a specified routine, the errors will not be purged.

Note:

Code that loops over the SQLStats errors will need to be changed; the structure has one less subscript level now. Previously, it was

^%sqlcq($Namespace, "PTools", "Error", $JOB, counter)

and now it is

^%sqlcq($Namespace, "PTools", "Error", $JOB)

Invalid Dates And Times Now Return Zero From CAST

In prior releases, an attempt to CAST a invalid value to a date or time would result in a “”. Beginning with this release, a

CAST( x AS DATE )

or

CAST( x AS TIME )

when x is a %String, %TimeStamp, or %FileMan.TimeStamp value that is not a valid value will return a zero. Applications that depend on a return value of null must be changed.

Length Of The Empty String Corrected

The SQL empty string has a length of 0 and is now reported as such by $LENGTH. Applications expecting SQL $LENGTH('') to return 1 need to be adjusted to recognize that 0 is now be returned.

Change Default For Gathering Statistics

In prior releases, the default value of $SYSTEM.SQL.SetSQLStatus was 1 which causes generated SQL code to check if the query should log any additional data about its execution. This slows down execution speed. In this release, the default has been changed to 0 so queries will not generate unneeded checks if they should log any additional data.

If a situation arises where this logging is required, the user can modify the system wide value, or the process specific value and then either purge the query they are interested in or recompile the routine/class which contains the query and then obtain the logging information as before.

Changes To SQL Plan Optimizer

In previous releases, when attempting to choose the best query plan, the optimizer could face a choice between two plans that seemed equally fast. In this release, the calculation of “cost” for plans has been refined and choices that formerly seemed equivalent now differ. In this case, the choice the optimizer makes could, in fact, be worse for some types of queries depending on the metadata present about the tables involved.

SQLStats Generation Now On By Default For Level 2

This release restores the default behavior to generate SQLStats code in routines. The default is to only generate level 2 SQLStats call in the generated code. This is one line in the query OPEN and one in the query CLOSE. SQLStats still has a Level 3 option that gathers info at every Module level. This option is generally only used once a problem query has been identified and further research is need.

Also, in the past, switching from SQLStats = 2 to SQLStats = 3 did not require a recompile of the SQL. With this release, a recompile of SQL is needed when you change SQLStats to level 3.

$CHAR(152) Mapping In CP1251

As a convenience to users employing Cyrillic scripts, the undefined code point 152 in Windows Code Page 1251 (CP1251) now maps to itself when being translated to Unicode. Conversely, when U+0098 (152) is converted from Unicode to CP1251 the result is also 152. This change is convenient to Russian users, for example, because it allows the round trip of $C(152) when translating between Unicode and CP1251.

Changes To System Locales Prohibited

The Modify() method in the Config.NLS classes will now return an error if you try to edit a system locale or table. The method continues to work as before for custom locales and tables. The system locales are supplied by InterSystems and their contents is assumed; so they should not be edited by customers.

Non-Relevant Text Now Stored In EntOccId

In this release, non-relevant sentence parts will now be stored in the same data structures as entity occurrences. This enables more transparent SQL access to all constituent parts of a sentence and will more easily accommodate future extensions (new entity types, annotating non-entity sentence parts, ...).

Applications relying on the unwritten (and never documented!) rule that entity occurrence IDs only represented entities will now need to take into account the role of the corresponding entries to ensure they're looking at entities rather than non-relevants.

New Dominance Algorithm Implemented

This release introduces refinements to the definition of dominance and a new algorithm for calculating these entity relevance metrics in both iKnow and iFind. In addition, some of the underlying data structures were rationalized, which may mean up to 15% decrease in storage consumed by fully-generated iKnow domains. This change also introduces new queries for the DominanceAPI and ProximityAPI that support filtering.

New "local dominance" values for entities are calculated based entirely on information from within the document. These values range from 0 to 1 (then multiplied by 1M and rounded for internal efficiencies) and are therefore comparable between documents.

For iFind users, only the actual dominance values will change.

For iKnow users, there are a few more changes accompanying the new algorithm:

• The dominance profile is generated based on a new formula selecting a relevant fraction of the upper quartile (top 25%), always including the highest values.

• Dominance values for CRCs and paths are still generated (as a weighted average of the entities they consist of), but should primarily be used to compare relevance with other CRCs or paths respectively.

• The default values for queries retrieving dominance profiles are therefore updated to retrieve concept profiles rather than aggregate ones.

• The DominanceAPI has a new entry point in the form of a straightforward GetTop() method (supporting filtering). The main entry point for the ProximityAPI is now GetProfile() or GetProfileById().

Customized Sentiment And Negation Attributes In The User Dictionary

This release changes the user-visible part of the mechanism to customize sentiment and negation detection through a User Dictionary from the implementation in its trial exposure. By using the three new methods – AddNegationTerm(), AddPositiveSentimentTerm(), and AddNegativeSentimentTerm() – of the %iKnow.UserDictionaryOpens in a new tab class, specific words can be annotated as indicating negation or sentiment. This causes the iKnow engine to treat them appropriately and expands them to the relevant parts of the sentence.

Change Default CSP Error Page

In previous releases, if a CSP application did not have any error page specified, Caché would default to %CSP.Error.cls which displayed a lot of information about the %request, %response, %session objects in order to aid in debugging the problem. In this release, if a customer has not supplied their own error page, the default error page is now %CSP.ErrorLog.cls which just displays a simple message saying that an error has occurred and logs all the information to the ^%ETN error log. Caché does this to ensure we do not leak information when an application error happens.

%CSP.ErrorLogOpens in a new tab class is the superclass of %CSP.ErrorOpens in a new tab, and it has a method, LogError, which captures the information so %ETN can log this correctly.

Base64–Encoded XML Output No Longer Includes Line Breaks by Default

In previous releases, Caché by default included line breaks when generating output for base64–encoded XML, specifically for properties of type %BinaryOpens in a new tab or %xsd.base64BinaryOpens in a new tab in XML-enabled classes. This default posed problems when working with non-Caché web services and web clients. In this release, Caché no longer includes line breaks when generating output for such properties.

To include these line breaks for SOAP web service or web client binary output, set the Base64LineBreaks property = 1 or the BASE64LINEBREAKS parameter = 1. If both the Base64LineBreaks property and the BASE64LINEBREAKS parameter are set, the property overrides the parameter.

Default Output Order of Properties Changed in Case of Multiple Inheritance

In some cases, a given class might be based on multiple XML-enabled superclasses. In previous releases, the corresponding XML schema was created with inheritance from the primary superclass, followed by properties from the second superclass, and so on. The order of properties for export and import was inconsistent with that: Caché started with the properties of the right-most superclass. As of this release, by default, the properties for export/import are also generated in left-to-right order. To obtain the behavior of previous releases, specify the XMLINHERITANCE parameter as "right" in the subclass.

CLASSNAME=1 Required For Serial Classes With Subclasses

When developing applications via the SOAP or XML schema wizards, use the CLASSNAME=1 property parameter for properties which reference a serial class that has subclasses. If CLASSNAME=1 is not added as a property parameter on the referencing property, attempting to save the object will generate an error. Without this property parameter, the generated code will no longer be able to identify subclasses of the referenced object.

Implement CORS Within REST

Beginning with this release, Caché now handles the HTTP OPTIONS verb even if the REST application requires authentication. Browers do not send authentication headers when making a pre-flight options request and thus previously there was no way of handling this when the application required a login. Now we return the Allow: header containing the list of allowed HTTP verbs and the ACCESS-CONTROL-ALLOW-ORIGIN header specifying the origin derived from the ORIGIN header of the request.

The %CSP.RESTOpens in a new tab class now contains a new callback method,, HandleCORSRequest(pUrl). An application can set the required access control headers to achieve the CORS compliance it requires. The default implementation does nothing, but example code is provided in the body of the method.

Pivot Tables Handle Calculated and Regular Measures Consistently

In previous releases, the Analyzer generated pivot tables of different forms for calculated measures than for regular measures, in one specific scenario. That difference has been corrected so that the Analyzer now treats both kinds of measures in the same way.

The specific scenario is a pivot table defined with a level or other item used in Columns and a single measure (or calculated measure) used in Measures. In this scenario, there are two possible pieces of information to include in column headers: the name of the item specified in Columns and the name of the measure. In previous releases:

• If the measure was not calculated, the pivot table did not include a header that shows the name of the measure.

• If the measure was a calculated measure, the pivot table did include a header that shows the name of the measure. This header was displayed below the header for the item specified in Columns.

In this release, by default the pivot table does not include a measure header in this scenario. The Analyzer provides a new option that you can use to control whether the measure header is shown; to access this option, click the Options icon in the Measures box. This option applies only to the scenario described here.

Restriction on Use of Measures on Multiple Axes

In previous releases, it was possible to create a pivot table that used measures on more than one axis, and the results were indeterminate, but DeepSee did not display an error. In this release, the MDX engine has been adjusted to prevent the existence of measures on more than one axis. The Analyzer, for example, no longer lets users drag and drop measures to multiple axes. If measures are defined on more than one axis, it will throw the error: “Measures cannot exist on multiple axes”.

As part of this adjustment, many of the MDX functions have been formalized internally so that, for example, they return either a scalar value or a member. The DeepSee MDX Reference provides the returned type for each function.

If a function returns a number, it is effectively a measure and is thus subject to the restriction described here. As a result of that change, some pivot tables that used to work now display the error ERROR #5001: Two measures cannot be crossjoined (2).

Note the following exceptions:

• The engine does not treat the functions %CELL and %CELLZERO as measures, because they are processed as a final step in query execution, and when used properly should not create ambiguity.

• The engine does not treat aggregating functions such as SUM as measures, unless these functions are used with their optional expression argument.

For example, the following queries are legal:

>>SELECT Measures.[Amount Sold] ON 0, AVG(Product.P1.[Product Name].MEMBERS) ON 1 FROM HOLEFOODS
                                   Revenue
AVG                               $5,199.61

>>SELECT AVG(Product.P1.[Product Name].MEMBERS,Measures.[Units Sold]) ON 1 FROM HOLEFOODS

AVG                                  997.82

In contrast:

>>SELECT Measures.[Amount Sold] ON 0, AVG(Product.P1.[Product
Name].MEMBERS,Measures.[Units Sold]) ON 1 FROM HOLEFOODS

ERROR #5001: Measures cannot exist on multiple axes
[%Prepare+174^%DeepSee.Query.query.1:SAMPLES]

Better Handling of Legend Padding on Dashboards

This release provides a different implementation for the padding of chart legends on dashboards. If you have existing dashboards that specified non-default sizes or padding for chart legends, you may need to readjust those dashboards. In particular, a chart legend that was previously visible could no longer be visible; if this occurs, use the Dashboard Editor to modify the chart legend size and padding.

Hiding Measure Captions Or Labels

The issue of how to deal with measures in pivot tables has been an ongoing one. Single measures were placed in the slicer instead of on one of the displayed axes (rows/columns). This caused problems with calculated measures since not all made sense in the slicer; so calculated measures were placed on a displayed axis. The difference then between simple and calculated measures was sometimes confusing for users. There was no way to hide the label of a calculated measure, since it would never be placed in the slicer (items in the slicer are never shown in a pivot table).

In this version, the display of measure headers/labels is now a configurable part of the pivot table. The measures dialog has been expanded to include a new setting, “Display measure headers”, which has three options: When More Than 1 Measures Are Used; Always; and Never. This setting is saved with the pivot table definition and will be active whenever it's displayed. The default is When More Than 1 Measures Are Used.

Client-Side Changes To %ZEN.Component.Group Layout Property Prohibited

To prevent execution of arbitrary class methods from Javascript, the layout property of %ZEN.Component.group and subclasses (other than %ZEN.Component.menu) are now encrypted client-side and can only be changed on the server. Changes made on the client to the layout property of Zen groups (i.e., zen('myGroup').setProperty('layout','horizontal')) will result in <ILLEGAL VALUE> errors when the component properties are next processed by the server.

Generated Javascript Documents Are Now UTF-8

In prior releases, files generated by Zen were not explicitly encoded; they assumed to be in the locale of the instance on which they were generated. This behavior could create issues if the application assumed the files were in a different encoding (such as Unicode), or used them in a different locale.

Beginning with this release, Zen generates files explicitly encoded as UTF-8. Application that assumed a specific locale must be changed to accommodate the new encoding.

jQM: Id Attribute Change

The id attribute for the layout object $panel is deprecated; it should be replaced with the attribute, key to be consistent with other layout objects. This means that code which presently reads

{type:'$panel’, id:'leftPanel', displayMode:'push', children:[]}

should be rewritten as

{type:'$panel’, key:'leftPanel', displayMode:'push', children:[]}

In this release, $panel will still work if id is specified, but InterSystems will remove support for it in some future version.