Skip to main content
Previous sectionNext section

Caché 5.1

This chapter provides the following information for Caché 5.1:

New and Enhanced Features for Caché 5.1

Welcome and thank you for using Caché!

This chapter provides an overview of the new and improved features in Caché 5.1.

See the Major New Features section for a description of the important new features and enhancements included in this release.

If you are new to Caché, you can refer to the Getting Started page which contains a variety of links to documentation organized by topic.

Upgrading and Installation

If you are upgrading existing applications and databases from prior versions, please read the Caché 5.1 Upgrade Checklist and the Upgrading chapter of the Caché Installation Guide.

For information on installing Caché, refer to the following sources:

Major New Features

Caché 5.1 introduces a significant number of new features as well as enhancements to existing features. These features are focused on:

  • Advanced Security

  • Maximizing Scalability

  • Maximizing Development Speed

  • Minimizing Support Load

The new features are summarized here. For additional details, refer to the cited chapters or guides.

A host of new capabilities have been added to give Caché the most advanced security of any mainstream database.

A new integrated system management interface, built with CSP, replaces Control Panel, Explorer, and SQL Manager. This removes the requirement for a Windows PC in order to manage Caché and, because no Caché client software is required, eliminates potential client/server version mismatch issues and simplifies management of multiple versions of Caché from a single device.

Caché system improvements include many new or enhanced classes and methods, plus major enhancements such as nested rollback and the ability to map class packages to namespaces.

  • Nested Rollback — When nested TSTARTs are used, this enhancement enables the innermost TSTART to be rolled back, without rolling back the entire open transaction.

  • Namespace Mapping for Class Packages — Namespace mapping has been extended with the ability to map class packages by name, just as routines and globals are mapped.

The ObjectScript language now provides significantly improved runtime error reporting. Many other enhancements have been introduced, including the following items:

  • New $FACTOR Function

  • New $LISTNEXT, $LISTTOSTRING, and $LISTFROMSTRING Functions

  • New $ROLES and $USERNAME Special Variables

  • New Error Trapping Syntax

  • More Efficient Code Generation

  • Pattern-Match “E” Adapted For Unicode

  • Faster MERGE Command

New Language Bindings

With this release, Caché introduces new Perl and Python bindings, as well as an improved version of the Caché ActiveX binding.

The Caché 5.1 Class Library provides many new features and major enhancements.

  • Index on Computed Fields — An index definition can now reference properties defined as CALCULATED and SQLCOMPUTED.

  • Object Synchronization — Caché can now track records of all object filing events (insert, update and delete) for journaled classes, export the journaled object data, and synchronize it with other databases. Applications with no access to the original database can then resolve references to the synchronized objects.

  • Studio Enhancements — New %Studio.Extension classes provide mechanisms for custom menus and user defined data entry. %Studio.SourceControl classes now provide enhanced source control hooks, allowing customized checkout and checkin to a source control system.

  • Performance Improvements — Significant improvements have been made to the in-memory performance of relationships.

  • Syntax for defining stream and collection properties has been improved, and enhancements have been made to the behavior of streams and collections.

Caché SQL support includes many new or enhanced features, including the following items:

  • New SQL/XML Support Functions

  • JDBC 3.0 Support

  • SAVEPOINT: New Transaction Processing Feature

  • CREATE TABLE: New IDENTITY Keyword

  • DROP VIEW: New CASCADE Keyword

  • INSERT: New DEFAULT VALUES Clause

  • New RowId Counter Validation Option

  • New Query Optimizer Plan Verification

  • Subquery Flattening

  • Enhanced Locking Behavior for Foreign Key References

  • READONLY Tables and Fields

  • Support for %%CLASSNAMEQ and %%TABLENAME

  • CREATE BITMAP INDEX Support for Oracle Import Compatibility

Caché 5.1 introduces many new options for network connectivity.

  • ECP Enhancements — A number of enhancements have been made to the Caché Enterprise Cache Protocol. It is now supported in shared disk cluster configurations with OpenVMS and Tru64 UNIX®.

  • SNMP Support — Support for the Simple Network Management Protocol (SNMP) has been added to enable monitoring of Caché by a variety of systems management tools and frameworks.

  • LDAP Client — Programmatic access to LDAP servers has been added.

  • Mac OS X Server Support — Support has been added for Mac OS X as a server plus the following client components: ODBC, JDBC, Objects, CSP Gateway for Apache.

Caché Advanced Security

With version 5.1, InterSystems introduces Caché Advanced Security. This release of Caché contains a host of new capabilities that provide the most advanced security of any mainstream database. Caché Advanced Security provides a simple, unified security architecture that offers the following advantages:

  • It offers a strong, consistent, and high-performance security infrastructure for applications.

  • It meets certification standards.

  • It makes it easy for developers to build security features into applications.

  • There is a minimal burden on performance and operations.

  • It ensures that Caché can operate effectively as part of a secure environment and that other applications and Caché can work together well.

See the Caché Security Administration Guide for detailed information on Caché Advanced Security.

Key Features

Here are a few of the more important new security features offered in Cache 5.1:

  • Kerberos based Security Infrastructure

    Two Authentication models are now available. In addition to Caché Authentication (Username/Password), Cache now provides Kerberos based Security Infrastructure. Kerberos libraries are available on all supported platforms (Windows Single Sign-on for Win32/64 platforms in an Active Directory Domain = Kerberos Realm, since Microsoft uses Kerberos at the heart of their Authentication model).

  • Security Management Interface

    The Management Portal's web-based Security Management facility allows complete access to Users, Roles, Services, Resources (including Schemas), Auditing, and all other aspects of Caché security management.

  • Security Advisor Utility

    The new Security Advisor utility makes recommendations for securing a Caché DBMS (Security settings, Applications and Auditing).

  • Authentication in ODBC/JDBC

    ODBC and JDBC drivers now offer both Caché and Kerberos Authentication. Kerberos mode provides three levels of Encryption: Clear, Integrity (Source and Content Validation), and Encrypted (complete, end-to-end AES Encryption).

  • Auditing Facilities

    Caché provides detailed auditing facilities that store audit information in a specially protected Audit Database. Auditing capabilities are available from an Automated/Management and Programmatic/API point of view.

  • Encrypted Database Management Facility

    The new Encrypted Database facility allows you to create fully encrypted (AES, up to 256 bit) CACHE.DAT files that stay Encrypted on Disk at all times. I/O is encrypted and decrypted on the fly, with minimal performance impact. The database is encrypted with a Special Key file that is stored on removable devices (like USB Flash Drives) and must be present to mount the DB for use.

Security Advisor

To assist system managers in securing a Caché system, Caché includes a Security Advisor. This is a Web page that shows current security-related system configuration information, recommends changes or areas for review, and provides links into other system management Web pages to make recommended changes.

Caché 5.1 contains the initial version of the Security Advisor. Its function and range will expand in future versions. It is accessed through the System Management Portal at Home, Security Management, Security Advisor.

InterSystems strongly recommends a review and resolution of the issues raised by the Security Advisor before allowing a secured system to attain production status.

Low-level Security Interfaces

System administrators can exercise low-level control over the security of Caché systems through two character-oriented interfaces:

  • ^SECURITY allows examination and editing of security data related to users, roles, domains, services, applications, and auditing. An overview of ^SECURITY can be found in The CHUI-Based Management Routines.

  • ^DATABASE provides low-level management capabilities related to Caché databases. An overview of ^DATABASE can be found in The CHUI-Based Management Routines.

Common Criteria Security Certification

Security certification is becoming an increasingly frequent requirement for government purchases, and is more and more requested for private sector purchases. Because of this, InterSystems has had Caché certified according to the Common Criteria standard. Specifically, effective February 15, 2007, Caché received certification according to the Common Criteria standard (EAL 3).

The Common Criteria provides a set of common security standards for a wide number of nations in North America, Europe, and the Far East. It provides an assurance scale from 1 to 4, where a product's rating indicates the rigor of testing to which it has been subjected; commercially available products are rated from 1 (least rigorous testing) to 4 (most rigorous). Caché is currently under consideration for a level-3 rating. Such a rating indicates that Caché can effectively serve as part of a highly secure operational environment.

Caché Advanced Security Concepts

Caché Advanced Security is based on authentication, authorization, and auditing:

  • Authentication ensures the verification of the identity of all users.

  • Authorization ensures that users can access the resources that they need, and no others.

  • Auditing keeps a log of predefined system and application-specific events, to provide forensic information about the database activities.

Authentication: Establishing Identity

Authentication is how you prove to Caché that you are who you say you are. Without trustworthy authentication, authorization mechanisms are moot — one user can impersonate another and then take advantage of the fraudulently obtained privileges.

The authentication mechanisms available depend on how you are accessing Caché. Caché has a number of available authentication mechanisms:

  • Kerberos — The most secure means of authentication. The Kerberos Authentication System, developed at MIT, provides mathematically proven strong authentication over an unsecured network.

  • Operating-system–based — Available for UNIX® and OpenVMS, OS-based authentication uses the operating system’s user identity to identify the user for Caché purposes.

  • Caché login — With Caché login, Caché maintains a table of hashed password values for each user account; at login, Caché confirms user identity by comparing the value in the table with a hash of the password provided by the user.

Authorization: Controlling User Access

Once a user is authenticated, the next security-related question to answer is what that person is allowed to use, view, or alter. Authorization manages the relationships of users and assets such as databases, Caché services like ODBC access, and user-created applications.

In the most basic authorization model, there are all possible assets, a list of users, and all the relationships between the first group and the second.

Auditing: Knowing What Happened

Auditing provides a verifiable and trustworthy trail of actions related to the system. Auditing serves multiple security functions:

  • It provides proof — the proverbial “paper trail” — recording of the actions of the authentication and authorization systems in Caché and its applications.

  • It provides the basis for reconstructing the sequence of events after any security-related incident.

  • Knowledge of its existence can serve as a deterrent for attackers (since they know they will reveal information about themselves during their attack).

The auditing facility automatically logs certain system events; it also allows you to enable logging for other system events, as well as site-defined application events. All audited events are placed in a tamper-resistant log file. Authorized users can then create reports based on this audit log, using tools that are part of Caché. Because the audit log can contain sensitive information (such as regarding positive or negative values for medical tests), running an audit report itself generates an entry for the audit log. The included Caché tools support report creation, archiving the audit log, and other tasks.

System Management Portal

Caché 5.1 now uses a browser-based interface, the System Management Portal, for system management. This new interface subsumes the functions previously distributed among Explorer, SQL Manager, Configuration Manager, and Control Panel functions of the Windows Caché Cube. In 5.1, these have been removed from the Cube.

An advantage of this approach is that it is no longer a requirement that Caché be installed on the system you use to manage an installation. Remote management of systems over the web, subject to access control established for the site, is now much easier. No Caché client software is required, simplifying management of multiple versions of Caché from a single device. Cross-release compatibility issues are minimized because both the data and its formatting information come directly from the system being managed.

See Using the System Management Portal for a detailed description of the new interface.

System Improvements

New Caché 5.1 system features and enhancements:

New Features:

  • Nested Rollbacks

  • Namespace Mapping for Class Packages

  • New Method $SYSTEM.Util.CleanDeadJobs()

  • New Class $SYSTEM.Monitor.Line

  • New Method $System.Device.GetNullDevice()

  • New Optional Argument for $ZF(-2)

Enhanced Features:

  • Option to Filter Records before Dejournaling on a Shadow

  • Callin Enhancements

  • 64K Routine Buffer Support

  • CVENDIAN Enhancements

Nested Rollbacks

This version of Caché introduces multiple transaction levels , which make it possible to roll back part of a transaction without losing all work completed to that point. When nested TSTARTs are used, this enhancement enables the innermost TSTART to be rolled back, without rolling back the entire open transaction. When two TSTARTs are issued without an intervening COMMIT or TROLLBACK, the transaction level ($TLEVEL) is incremented by 1 (limited to a maximum of 255). When a TCOMMIT or TROLLBACK 1 is issued, the transaction level is decremented by 1. When an unqualified TROLLBACK is issued, the transaction level is decremented to 0, and the entire transaction is rolled back.

Transaction commands now work as follows:

  • The argumentless TROLLBACK command works as usual, rolling back to the very top level transaction and closing the transaction.

  • The TROLLBACK 1 command rolls the current open transaction back one level. All the globals changed within this transaction will be restored, and $TLEVEL is decremented by 1. If there is no open transaction ($TLEVEL is zero) then no action is taken. TROLLBACK 1 won't roll back globals mapped to a remote system that doesn't support nested transactions unless $TLEVEL is 1.

  • The TCOMMIT command works as usual. In nested transactions, it decrements $TLEVEL and writes a 'PTL' (pending commit with transaction level) journal record to the journal file.

  • The TSTART command also works as usual. In nested transactions, it increments $TLEVEL and writes a 'BT'(begin transaction) record in the journal file. If the new $TLEVEL is greater than 1, it writes a 'BTL'(Begin Transaction with level) instead of 'BT'.

Caché SQL now includes standard SQL commands that take advantage of nested rollbacks (see New SAVEPOINT Features).

Namespace Mapping for Class Packages

Namespace mapping has been extended with the ability to map class packages from a database to one or more namespaces, just as routines and globals are mapped. Automatic namespace mapping is provided for system classes. All the schemas that begin with '%' from %sys are mapped to all namespaces automatically. These mappings allow the user to access SQL Table, View, Procedures, and classes across multiple namespaces. For example, assume a class %Test that has the following query:

Select field1 From %Test
Copy code to clipboard

Without mapping, attempting to inherit from this class in a user namespace would result in error: "Table %Test not found". With mapping, the class will compile successfully in any namespace.

For detailed information, see Configuring Data in the Caché System Administration Guide.

New Method $SYSTEM.Util.CleanDeadJobs()

New class method $SYSTEM.Util.CleanDeadJobs() is used to roll back a dead job's open transaction (if any) and clean up the dead job's Process Table (pidtab) slot so it can be re-used.

New Class $SYSTEM.Monitor.Line

New class $SYSTEM.Monitor.Line is a programmer API for the line-by-line monitor (^%MONLBL). It allows integration with Studio, and is also generally useful as a programmable alternative to ^%MONLBL. For details, see the Programming Interface section in the MONLBL chapter of the Caché Monitoring Guide.

New Method $System.Device.GetNullDevice()

New class method $System.Device.GetNullDevice() returns the name of the null device appropriate for the current operating system type (/dev/null for UNIX®, _NLA0 for OpenVMS, //./nul for Windows). It facilitates development of applications that reference the Null device, and provides an OS-independent method for obtaining the name of the Null Device.

New Optional Argument for $ZF(-2)

Function $ZF(-2) now has an optional fifth argument that specifies whether or not the spawned process ID should be stored in $ZCHILD. For example:

s rc=$zf(-2,"program","","",1) 
s childpid=$ZCHILD 
Copy code to clipboard

If the new argument is zero or not specified then $ZCHILD is unchanged, otherwise $ZCHILD is set to the spawned process ID when it is successfully spawned.

Option to Filter Records before Dejournaling on a Shadow

To filter journal records before they get dejournaled on a shadow, set the global node ^SYS("shdwcli",shdw_id,"filter") to the name of the filter routine (without the leading "^"). The input parameters of the filter routine are:

  • pid: process ID of the record

  • dir: SOURCE (not SHADOW) database directory

  • glo: global reference in the form of global(subscripts) (without leading "^")

  • addr: offset of the record in the journal file

  • type: type of the record: "S" = SET, "s" = BITSET, "K" = KILL, "k" = ZKILL

  • time: timestamp of the record

In compatible mode shadowing, the pid and timestamp parameters passed to the filter routine always have the value "". The filter routine should return 0 if the record should be skipped; otherwise the record will be dejournaled by the shadow. For example:

^SYS("shdwcli","MyShadow","filter")="MyShadowFilter"
MyShadowFilter(pid, dir, glo, type, addr, time) ; 
;skip X* globals 
If $EXTRACT($qs(glo,0))="X" q 0 
Set Msg = pid
Set Msg = Msg _ "," _ dir
Set Msg = Msg _ "," _ glo
Set Msg = Msg _ "," _ type
Set Msg = Msg _ "," _ addr
Set Msg = Msg _ "," _ time
Do ##class(%Library.Device).Broadcast("",Msg) 

q 1
Copy code to clipboard

Callin Enhancements

The Callin include files ccallin.h and mcallin.h have been enhanced to merge common functionality and provide greater flexibility for building user-defined C and C++ Callin modules. Defines have been added to make building user Callin modules as independent of interface details as possible. Two features control the selection of interfaces:

#define ZF_DLL

If ZF_DLL is not defined, the Callin module is built for linking with the Caché engine. If it is defined, the module is built as a dynamic shared library using Callback and invoked through the Callout facility. This is the same define employed by cdzf.h.

#define CACHE_UNICODE

If CACHE_UNICODE is not defined, string handling functions and arguments are treated as 8-bit characters. If defined, strings are treated as 16-bit Unicode. String handling functions are available with the "A" suffix, meaning 8-bit (or ASCII), the "W" suffix, meaning 16-bit Unicode (or wide), and no suffix. In the last case the function resolves to either the "A" or "W" suffix according to the definition of CACHE_UNICODE.

New functionality has been implemented to permit NLS translation using the CacheCvtInW() and CacheCvtOutW() functions for Unicode Callin to 8-bit Caché. They will now convert data within the 8-bit character set range of the Caché engine, instead of reporting an "unimplemented" error. CacheCvtInA() and CacheCvtOutA() functions for 8-bit Callin to Unicode Caché are not currently implemented.

You can further refine 8-bit argument prototypes with the new macro USE_CALLIN_CHAR, which declares them as (char *) rather than (unsigned char *).

64K Routine Buffer Support

It is now possible to run with routine sizes up to 64K, by changing the Memory/RoutineBufSize value on the Home, Configuration, Advanced Settings page of the Management Portal from 32 to 64. The default and minimum value is still 32 (32K), but now values can be specified from 33..64 (rounded to the nearest 2K increment). Routines or class descriptors greater than 32K will be stored as two global values, the first chunk in ^rOBJ(<routine name>) as currently, and the second chunk in ^rOBJ(<routine name>,0).

CVENDIAN Enhancements

The cvendian database endian conversion utility has been enhanced to allow for positive identification of the desired endian orientation, or to optionally just inform the current endian orientation with no conversion. The command syntax is:

cvendian [-option] file1 [file2 ... file8]

where option is one of the following:

  • -big — convert the database to big-endian

  • -little — convert the database to little-endian

  • -report — report the endian orientation of the database

The options may be shortened to their initial letter. If this is a conversion request, and the database already is of the specified endian orientation, a warning message is displayed and no further processing is done. Prior cvendian call formats remain supported.

Object Improvements

New Caché 5.1 object features and enhancements:

Object Enhancements

  • New Option to Index on Computed Fields

  • New Object Synchronization

  • New Studio Extension Classes and Source Control Hooks

  • New Stream Syntax

  • New %SwizzleObject Class

  • Extended POPSPEC Syntax

  • Performance Improvements for Relationships

  • Enhanced VisM OCX

New Option to Index on Computed Fields

An index definition can now reference properties defined as CALCULATED and SQLCOMPUTED. The property value calculation must be deterministic, always returning the same value for a given set of parameters. For example, it would be a mistake to use a function such as $Horolog, which returns different values depending on when it is called. Indexing on a property whose computation is nondeterministic will result in an index that is not properly maintained.

To support this option, properties defined as SQLCOMPUTED are now computed in Caché Objects. A new class method, Compute, is called by the property's Get method. The Compute method generates a return value by scanning SQLCOMPUTECODE for field references and converting those references to property or literal values. If the property also has SQLCOMPUTEONCHANGE, the Compute method is called whenever the property is changed.

New Object Synchronization

This new feature enables Caché to synchronize objects between databases. All object filing events (insert, update and delete) for journaled classes are automatically tracked. Object synchronization utilities provide methods to export the journaled object data and synchronize it with other databases. Applications with no access to the original database can then resolve references to the synchronized objects.

A new class, %SYNC.SyncSetObject, supplies methods to externalize an object and apply it to the target database. All references to persistent objects from the object being externalized are converted to GUID (Globally Unique Identifier) values. The GUID values are used to look up the corresponding object on import.

Another class, %SYNC.SyncSet, implements methods to manage the set of objects being synchronized. A 'synchronization set' is a set of externalized object values which guarantee that all object references can be resolved, either because the referenced object is in the same sync set, or because it already exists in the target database.

New Studio Extension Classes and Source Control Hooks

This release enhances the flexibility of Studio by introducing the %Studio.Extension classes, which provide mechanisms for custom menus and user defined data entry. The %Studio.SourceControl classes now provide enhanced source control hooks, allowing customized checkout and checkin to a source control system.

When the user performs an action in Studio that may require user interaction with the server (for example, attempting to edit a document that is in source control but is not checked out), Studio now calls the UserAction method.

UserAction (Type, Name, InternalName, SelectedText, .Action, .Target, .Msg)

Type options are:

  • Server defined menu item selected

  • Other Studio action

Name is the menu item name if Type is a menu item, otherwise Name indicates one of the following options:

  • User has tried to change a document that is locked in source control

  • User has created a new document

  • User has deleted a document

InternalName is the name of the document this action is concerned with.

SelectedText contains any selected text in the document that has focus.

Action returns an action that Studio should perform:

  • Do nothing (this method can still perform some action such as check an item out of source control, but Studio will not ask for user input).

  • Display the default Studio dialog with a yes/no/cancel button. The text for this dialog is provided in the Target return argument.

  • Run a CSP Template. Target is the start page name for the template. The template will be passed the current document name, any selected text, the project name, and the namespace.

  • Run an EXE on the client. Target is the name of an executable file on the client machine.

  • Insert the text in Target in the current document at the current selection point

  • Studio will open the documents listed in Target

You can define custom menus for Studio to display. Studio obtains the menus when it first connects to a namespace by running two queries, MainMenus and MenuItems. MainMenus returns the list of top level menu names. After this top level menu is selected, MenuItems is used to return the list of items on a specific menu. MainMenus can be either a regular menu or a context submenu that is added to all the context menus. The MenuItems query is passed the current document name and any selected text in case you wish to vary the menu based on these arguments.

By default, the source control class inherits these queries from %Studio.Extension.Base, where they are defined as SQL queries against prebuilt tables. To load data into these tables, define an XData block called Menu in your source control class. When the source control class is compiled, this data is loaded and used automatically. Queries defined in the source control subclass can be changed or completely customized. When data is being returned from the MenuItems query, each menu name will generate a call to an OnMenuItem method in the source control class, where you may disable/enable this menu item. This allows simple modification of the menus without having to write a custom query.

New Stream Syntax

The class hierarchy for current stream classes has been changed so that %Stream.Object is the top class. This change does not alter stream runtime behavior.

In prior versions of Caché, it was necessary to define a stream property as type = %Stream, with a collection value of binarystream or characterstream. Now a stream property is defined by specifying the actual stream class as the type, and the collection keyword values of binarystream and characterstream are no longer used. A stream class is declared with a classtype = stream. This declaration is automatic for any class that extends a new class, %Stream.Object. For backward compatibility, the classes %Library.GlobalCharacterStream, %Library.GlobalBinaryStream, %Library.FileCharacterStream, and %Library.FileBinaryStream have been converted to use the new representation, and are to be used for all existing stream data.

For more detailed information, see the Streams chapter in Using Caché Objects.

New %SwizzleObject Class

A new class, %SwizzleObject, is now the primary (and only) superclass of both %Persistent and %SerialObject. The purpose of the new class is to define the swizzling interface and implement the parts of that interface that are common to both %Persistent and %SerialObject.

See the %Library.SwizzleObject class documentation for more detailed information.

Extended POPSPEC Syntax

The syntax of POPSPEC has been extended to allow an SQL table name and an SQL column name to be specified. When they are specified, the Populate() method constructs a dynamic query to return the distinct column values from the table. The requested number of values will then be randomly selected from the distinct column values and placed in a value set. The property will then be assigned values randomly from the resulting value set.

See The Caché Data Population Utility for more detailed information.

Performance Improvements for Relationships

The in-memory performance of relationships has been significantly improved by using additional in-memory indices to keep track of oref's and oid of items already in the relationship. Previously, when a new item was inserted into the relationship (either using the Insert method, or indirectly via the Relate method) it would scan the entire relationship to avoid inserting a duplicate item. By keeping an index of the oref's and oid's in the relationship, the cost of checking for duplication items is kept very low even for large numbers of items.

Partition memory used is lower, speed is significantly faster (94x in the second insert of 1000 items) and %Save time is faster. When measured with a small number of items in the relationship, there was no measurable slowdown in performance associated with the upkeep of the additional in-memory indices.

Enhanced VisM OCX

This release contains a new version of the Caché Direct control (VISM.OCX) that features enhancements such as security upgrades, support for multithreading, and improved error handling.

Language Improvements

New Caché 5.1 ObjectScript features and enhancements:

  • Improved Runtime Error Reporting

  • New $FACTOR Function

  • New $LISTNEXT, $LISTTOSTRING, and $LISTFROMSTRING Functions

  • New $ROLES and $USERNAME Special Variables

  • New $ZUTIL(62,1) Function

  • New $ZUTIL(69) Configuration Functions

  • New $ZUTIL(158) Function

  • New $ZUTIL(186) Function

  • New $ZUTIL(193) Function

  • New Error Trapping Syntax

  • More Efficient Code Generation

  • Pattern-Match “E” Adapted For Unicode

  • Faster MERGE Command

New Language Bindings:

  • New Perl Binding

  • New Python Binding

  • New ActiveX Bindings

Improved Runtime Error Reporting

Many runtime errors now report additional information. For instance, an "<UNDEFINED>" error will now report the name of the undefined variable.

Error information is stored in the system variable $ZERROR, which now returns more information than before. For example, when a routine attempts to use a variable that has not been defined, $ZERROR now includes the name of the undefined variable. Whereas in previous versions of Caché the value of $ZERROR might look like this:

<UNDEFINED>zMethodName^Pkg.Class.1
Copy code to clipboard

in version 5.1, it looks generically like this (adding " *someinfo"):

<ERRCODE>Tag^Routine+line *someinfo
Copy code to clipboard

A consequence of this change is that error handling routines that made assumptions about the format of the string in $ZERROR may now require redesign to work as before. For further information, see the Cache Conversion Guide, and the $ZERROR special variable in the Caché ObjectScript Reference.

New $FACTOR Function

$FACTOR is a new ObjectScript function for 5.1 that converts a numeric value to a bitstring. Its primary use is for the creation of bitslice indices. For further information, see the $FACTOR function in the Caché ObjectScript Reference.

New $LISTNEXT, $LISTTOSTRING, and $LISTFROMSTRING Functions

Caché 5.1 adds three new functions for processing list structures: $ListNext, $ListToString and $ListFromString.

$ListNext(list, ptr, val) allows extremely rapid traversing of a list structure (up to 400x faster than doing a loop with $LIST).

Before the first call to $ListNext, ptr should be initialized to 0. After each call, ptr will contain the position of the next element in list (0 if the end of the list was reached), and val will contain the value of the element at that position (undefined if there was no value at that position). $ListNext will return 1 if it found another list element, or 0 if it is at the end of the list.

$ListToString(list[,delim]) takes list, and returns the elements as a string separated by delim (default ",").

$ListFromString(string[,delim]) takes string, delimited by delim (default ","), and returns the pieces as a list.

For further information, see the $LISTNEXT, $LISTTOSTRING, or $LISTFROMSTRING function in the Caché ObjectScript Reference.

New $ROLES and $USERNAME Special Variables

At Caché 5.1, the $ROLES special variable lists the security roles currently assigned to the user. The $USERNAME special variable list the user name for the current process. For further information, see the $ROLES, $USERNAME special variables in the Caché ObjectScript Reference.

New $ZUTIL(62,1) Function

The $ZUTIL(62,1) function performs syntax checking on a line of ObjectScript code. It returns the character position of the error and the text of an error message. For further information, see the $ZUTIL(62,1) function in the Caché ObjectScript Reference.

New $ZUTIL(69) System Configuration Functions

Caché 5.1 documents the following additional system-wide configuration functions: $ZUTIL(69,19), $ZUTIL(69,21), $ZUTIL(69,31), $ZUTIL(69,35), $ZUTIL(69,37), $ZUTIL(69,44), $ZUTIL(69,49), and $ZUTIL(69,60).

Caché 5.1 also supports the new $ZUTIL(69,63) and $ZUTIL(68,63) functions that control whether a lowercase “e” should be interpreted as an exponent symbol.

For further information, see the $ZUTIL(69) functions in the Caché ObjectScript Reference.

New $ZUTIL(158) Function

The $ZUTIL(158) function can be used to return the number of installed printers and the pathname of a specified printer. For further information, see the $ZUTIL(158) function in the Caché ObjectScript Reference.

New $ZUTIL(186) Function

The $ZUTIL(186) function can be used to specify the information displayed as part of the Terminal prompt. For further information, see the $ZUTIL(186) function in the Caché ObjectScript Reference.

New $ZUTIL(193) Function

The $ZUTIL(193) function inter-converts Coordinated Universal Time and local time values. For further information, see the $ZUTIL(193) function in the Caché ObjectScript Reference.

New Error Trapping Syntax

This version of Caché implements a special syntax that allows an error trap to pass control up the program stack to a previously established error trap. The syntax is ZTRAP $ZERROR. This command will pop entries off the program stack until a level is found with an error trap. Then that error trap will be executed with $ZERROR and $ECODE unchanged.

This command replaces the two commands ZQUIT 1 GOTO @$ZTRAP, which did not work in new-style procedures. This new command syntax can be used in both procedures and old-style subroutines. The old style of passing control up to a previous error trap will continue to work in old-style subroutines. If a ZQUIT command is issued in a procedure, it will now result in a <COMMAND> error.

The ZQUIT command is obsolete as of 5.1, and should not be used for new programming.

More Efficient Code Generation

The CacheBasic compiler now uses an improved algorithm that generates significantly smaller and faster code.

Pattern-Match “E” Adapted For Unicode

In prior version of Caché, the options used with the pattern-match operator(s) assumed 8–bit characters. This caused the “E” pattern (match every character) to fail when Unicode characters above $CHAR(255) were present in the string.

In Caché 5.1, the “E” pattern matches all characters.

Faster MERGE Command

The MERGE command is now much faster and more efficient when merging two local variables.

New Perl and Python Bindings

The Caché Perl and Python bindings provide a simple, direct way to manipulate Caché objects from within Perl or Python applications. They allow binding applications to establish a connection to a database on Caché, create and open objects in the database, manipulate object properties, save objects, run methods on objects, and run queries. All Caché datatypes are supported.

See Using Perl with Caché and Using Python with Caché for more detailed information.

Improved ActiveX Bindings

Caché 5.1 includes a new version of the Caché ActiveX binding, CacheActiveX.dll. Internally this new version uses the Caché C++ binding to get object-level access to a Caché server. Using this new binding provides the following benefits:

  • access to the client/server security model available within Caché 5.1 (for example, the ability to use Kerberos authentication)

  • better performance in some cases due to more sophisticated object caching.

While every attempt has been made to make this new DLL functionally compatible with the older CacheObject.dll it is not 100% binary compatible.

To preserve complete compatibility with existing applications, Caché installs two ActiveX bindings; the newer CacheActiveX.dll as well as the original CacheObject.dll. By default, existing applications will continue to use the original CacheObject.dll. If you wish to use the newer binding you have to modify your existing application to reference this new DLL and test that your application performs as expected.

SQL Improvements

New Caché 5.1 SQL features and enhancements:

New Features

  • New SQL/XML Support Functions

  • SAVEPOINT: New Transaction Processing Feature

  • CREATE TABLE: New IDENTITY Keyword

  • DROP VIEW: New CASCADE Keyword

  • INSERT: New DEFAULT VALUES Clause

  • New RowId Counter Validation Option

  • New Query Optimizer Plan Verification

SQL Enhancements

  • JDBC 3.0 Support

  • GRANT and REVOKE Command Changes

  • CREATE USER Command Changes

  • Subquery Flattening

  • Enhanced Locking Behavior for Foreign Key References

  • READONLY Tables and Fields

  • SQLCODE Changes

  • Support for %%CLASSNAMEQ and %%TABLENAME

  • CREATE BITMAP INDEX Support for Oracle Import Compatibility

  • Extended Support for Milliseconds

  • Date and Time Function Enhancements

New SQL/XML Support Functions

5.1 implements a collection of new built-in SQL functions for transforming “flat” relational queries into hierarchical XML documents. Application programs that need to generate HTML, or that need to export data in XML format, now have a general and portable interface that has wide industry support (ANSI/ISO SQL-2003 standard).

The following SQL/XML functions are available:

  • XmlElement – Creates an XML element of the form: <tagName>body</tagName>, with optional attributes. XmlElement creates one tagged element that can contain multiple concatenated values.

  • XmlAttributes – Specifies attributes for an XML element. XmlAttributes can only be used within an XmlElement function.

  • XmlConcat – Concatenates two or more XML elements.

  • XmlAgg – Aggregate function that concatenates the data values from a column.

  • XmlForest – Creates a separate XML element for each item specified. XmlForest provides a convenient shorthand for specifying multiple elements nested within another element, where element instances that are NULL are omitted.

For more detailed information see XMLELEMENT, XMLAGG, XMLCONCAT and XMLFOREST in the Caché SQL Reference.

New SAVEPOINT Features

With version 5.1, Caché introduces multiple transaction levels (see Nested Rollbacks), which make it possible to roll back part of a transaction without losing all work completed to that point. Caché SQL now offers the following standard SQL commands that take advantage of this ability:

  • SAVEPOINT <savepointName> — establishes a savepoint within a transaction.

  • ROLLBACK TO SAVEPOINT — rolls back to the most recent savepoint.

  • ROLLBACK TO SAVEPOINT <savepointName> — rolls back to the specified savepoint.

  • COMMIT – commits only the current sub-transaction when $TLEVEL > 1.

For more detailed information see SAVEPOINT in the Caché SQL Reference.

CREATE TABLE: New IDENTITY Keyword

Caché SQL now supports the ability to define a column with a system-generated numeric value in a CREATE TABLE statement. An IDENTITY column is an exact non-negative integer column whose values are system-generated, and may not be assigned by the user in either INSERT or UPDATE statements. It may, however, be viewed using SELECT *. The syntax is:

CREATE TABLE <tablename> ( 
   [ other-table-elements , ] 
   <columnname> [ <datatype> ] IDENTITY
   [ UNIQUE | NULL | NOT NULL | 
      DEFAULT [(]<default-spec>[)] | 
      [COLLATE] <sqlcollation> | 
      %DESCRIPTION <literal> 
   ] 
[ , other-table-elements ] 
) 
Copy code to clipboard

An IDENTITY column is always data type INTEGER with unique non-null values. You can specify a datatype and constraints, but these are ignored by Caché.

This syntax is consistent with Microsoft SQL Server and Sybase syntax.

For more detailed information, see CREATE TABLE in the Caché SQL Reference.

DROP VIEW: New CASCADE Keyword

Caché SQL now supports the ability to cascade the deletion of a view to also delete any view that references that view. The new keywords are CASCADE and RESTRICT. The RESTRICT keyword is the default and is the same as prior DROP VIEW behavior.

For more detailed information, see DROP VIEW in the Caché SQL Reference.

INSERT: New DEFAULT VALUES Clause

Caché SQL now supports the ability to use default field values when inserting a row into a table. The syntax is:

INSERT INTO <tablename> DEFAULT VALUES 
Copy code to clipboard

The statement will insert a single row into the table. Each field that has a default value will have the value assigned to the column. Fields without default values will be NULL for the row.

For more detailed information, see INSERT in the Caché SQL Reference.

New RowId Counter Validation Option

A new configuration option now makes it possible to validate new system-assigned ID values. The option is activated by setting ^%SYS("dbms","validate system-assigned id") to 1. Although such validation is not normally necessary, it is possible that the ID could be invalid if the user has modified the value manually, or if objects are inserted into the table without using the object or SQL filer. Other system recovery errors could also allow this condition to exist (bad recovery of a journal file, disk failure, etc.).

When this option is enabled, the table compiler will generate a uniqueness check on insert for the Id value. If validation fails, SQLCODE=-119, will be returned to the caller and a message will be written to the console log. After writing a message to the Console.log file and before returning from the filer, the user-defined routine ^%ZOIDERROR will be called. It is important to review the console log when this error is reported.

When this error is reported, it will be necessary to bring the ID counter back into sync with the data. Each failure will cause the system ID counter to be incremented, so it is possible that the problem will correct itself over time. At the point the error is reported it is not necessarily true that the counter is wrong, since the data itself may be incorrect. It is the responsibility of the user to determine how the counter became invalid.

New Query Optimizer Plan Verification

Regression tests based on TestSQLScript now have an easy way to verify query plan stability. Defining the class parameter SHOWPLAN=1 in %UnitTest.TestSQLScript will cause the query optimizer plan to be written to an output file.

JDBC 3.0 Support

Cache 5.1 supports JDK 1.4 and JDBC 3.0. All required features and most optional features are supported.

GRANT and REVOKE Command Changes

Due to the extensive improvements to Caché security at 5.1, the SQL GRANT and REVOKE commands no longer support the following syntactical forms:

  • GRANT ACCESS ON namespace

  • GRANT %THRESHOLD number

  • The %GRANT_ANY_PRIVILEGE, %CREATE_USER, %ALTER_USER, %DROP_USER, %CREATE_ROLE, %GRANT_ANY_ROLE, and %DROP_ANY_ROLE privileges

The GRANT and REVOKE command support the following additional options:

  • Granting a role to a role, creating a hierarchy of roles

  • The EXECUTE object privilege

  • The granting of object privileges to stored procedures, as well as tables and views

  • The use of the asterisk (*) to grant EXECUTE object privileges to all stored procedures

For more detailed information, see GRANT and REVOKE in the Caché SQL Reference.

CREATE USER Command Changes

At 5.1, issuing a CREATE USER does not automatically assign any roles or privileges to the user, regardless of the privileges held by the creator. Privileges and roles must be assigned to a new user using the GRANT command.

For more detailed information, see CREATE USER in the Caché SQL Reference.

Subquery Flattening

In many cases the SQL engine will now attempt to “flatten” certain types of SQL queries. That is, a query will be internally converted into an equivalent form that does not contain a subquery. In many cases, it is easier for the SQL optimizer to recognize this equivalent form, and a better execution plan is generated.

Enhanced Locking Behavior for Foreign Key References

Locking behavior during table filing has been changed in the following ways:

  • During SQL DELETE, for every foreign key reference a long-term shared lock will be acquired on the row in the referenced table. This row will be locked until the end of the transaction. This ensures that the referenced row is not changed before a potential rollback of the SQL DELETE

  • During SQL INSERT, for every foreign key reference a long term shared lock will be acquired on the referenced row in the referenced table. This row will be locked until the end of the transaction. This ensures that the referenced row is not changed between the checking of the referential integrity and the end if the INSERT's transaction.

  • During SQL UPDATE, for every foreign key reference which has a field value being updated, a long-term shared lock will be acquired on the old referenced row in the referenced table. This row will be locked until the end of the transaction. This ensures that the referenced row is not changed before a potential rollback of the SQL UPDATE.

  • During SQL UPDATE, for every foreign key reference that is being changed, a long term shared lock will be acquired on the new referenced row in the referenced table. This row will be locked until the end of the transaction. This ensures that the referenced row is not changed between the checking of the referential integrity and the end if the UPDATE's transaction.

READONLY Tables and Fields

Prior to this version of Caché, trying to INSERT, UPDATE, or DELETE into a ReadOnly table would not result in an error until the statement was executed. In this version, an SQLCODE=-115 error will be raised during compilation.

When a property is defined as ReadOnly, the field in the corresponding SQL table is also now defined as ReadOnly. READONLY fields may only be defined via an initialexpression or SQL Compute code; they may never be explicitly insert or updated via SQL statements. Any attempt to INSERT or UPDATE a value for the field (even a NULL value) will result in an SQLCODE=-138 error ("Cannot INSERT/UPDATE a value for a ReadOnly field").

SQLCODE Changes

The following SQLCODE error codes have been added for 5.1:

  • -129: This error is raised when you attempt to set a Caché Locale setting to an invalid value. See SET OPTION in the Caché SQL Reference for further details.

    SQLCODE = -129: Illegal value for SET OPTION locale property

  • -138: This error is raised when you attempt to compile an INSERT or UPDATE that references a read-only field. See INSERT in the Caché SQL Reference for further details.

    SQLCODE = -138: Cannot INSERT/UPDATE a value for a ReadOnly field

  • -142: This error is raised when the CREATE VIEW command contains a mismatch between the number of columns in the view definition and number of columns in the query. See CREATE VIEW in the Caché SQL Reference for further details.

    SQLCODE = -142: Cardinality mismatch between the View-Column-list and View Query's SELECT clause

  • -308: This error is raised when you attempt to define more than one IDENTITY field for a table. See CREATE TABLE in the Caché SQL Reference for further details.

    SQLCODE = -308 Identity column already defined for this table

  • -316: This error is raised when a Foreign key references a non-existent column.

    SQLCODE = -316 Foreign key references non-existent key/column collection

  • -321: This error is raised when you attempt to drop a view when another view references that view. See DROP VIEW in the Caché SQL Reference for further details.

    SQLCODE = -321 Cannot DROP view - One or more views reference this view

  • -356 and -357: These two errors may be raised by an attempt to use a user-defined SQL function.

    SQLCODE = -356: SQL Function (function Stored Procedure) is not defined to return a value

    SQLCODE = -357: SQL Function (function Stored Procedure) is not defined as a function procedure

  • -375: This error is raised when you attempt to roll back to a savepoint that was either never established or has already been rolled back.

    SQLCODE = -375 Cannot ROLLBACK to unestablished savepoint

  • -417: This error is raised when login fails. Usually this is due to username and password checking failure. It can also occur if the username is not privileged.

    SQLCODE = -417 Cache Security Error

  • -431: This error is raised when you attempt to pass a literal as a stored procedure parameter when the underlying argument type is an object type.

    SQLCODE = -431 Stored procedure parameter type mismatch

  • -459: This error is raised when you try to connect using Kerberos and security authentication fails. Possible reasons include: the Kerberos security executable cconnect.dll is missing or fails to load; your connection is rejected because of the Kerberos credentials you supplied.

    SQLCODE = -459 Kerberos authentication failure

The following obsolete SQLCODE values have been removed:

SQLCODE -340, -341, -342, -343, -344, -345, -346, -347

For a complete list of SQLCODE values, refer to the “SQLCODE Values and Error Messages” chapter of the Caché Error Reference.

Support for %%CLASSNAMEQ and %%TABLENAME

Caché SQL now supports {%%CLASSNAMEQ} and {%%TABLENAME} references in class definition SQL specific COS code in the following locations:

  • SQL Computed field code

  • SQL Trigger code

  • %CacheSQLStorage conditional map condition expression.

{%%CLASSNAMEQ} (not case-sensitive) will translate to the quoted string for the name of the class which projected the SQL table definition.

{%%TABLENAME} (not case-sensitive) will translate to the quoted string for the qualified name of the table

For example, assume the following trigger in the class User.Person:

Trigger AfterInsert1 [ Event = INSERT, Order = 1, Time = AFTER ] 
{ 
Set ^Audit("table",{%%TABLENAME},$j,"AFTER INSERT TRIGGER")=1 
Set ^Audit("class",{%%CLASSNAMEQ},$j,"AFTER INSERT TRIGGER")=1 
} 
Copy code to clipboard

If User.Employee extends User.Person, the following SQL trigger code will be pulled as an AFTER INSERT trigger in the SQLUSER.EMPLOYEE table:

Set ^Audit("table","SQLUser.Employee",$j,"AFTER INSERT TRIGGER")=1 
Set ^Audit("class","User.Employee",$j,"AFTER INSERT TRIGGER")=1 
Copy code to clipboard

CREATE BITMAP INDEX Support for Oracle Import Compatibility

When loading an Oracle SQL script file through $SYSTEM.SQL.DDLImport() or $SYSTEM.SQL.Oracle(), Caché SQL now recognizes the CREATE BITMAP INDEX statement.

Extended Support for Milliseconds

Caché SQL now supports fractional seconds in all date/time functions. The DATEADD, DATEDIFF, DATENAME, and DATEPART functions now support a datepart of "ms" or "milliseconds". The ODBC Scalar functions {fn TIMESTAMPADD()} and {fn TIMESTAMPDIFF()} now support the SQL_TSI_FRAC_SECOND parameter.

See DATEPART in the Caché SQL Reference for more detailed information.

Date and Time Function Enhancements

  • The SQL Scalar functions TO_DATE and TO_CHAR now accept %Library.TimeStamp logical values as input. In addition, the following format codes have been added for support of TimeStamp values:

    • HH – hour of day (1-12)

    • HH12 – hour of day (1-12)

    • HH24 – hour of day (0-23)

    • MI – minute (0-59)

    • SS – second (0-59)

    • SSSSS – seconds past midnight (0-86388)

    • AM – meridian indicator

    • PM – meridian indicator

  • There is a new configuration setting for the default format value for the TO_DATE() function. The default format is still "DD MON YYYY", but it can be changed using the following commands:

    Do $SYSTEM.SQL.SetToDateDefaultFormat(<value>)

    or

    Do SetToDateDefaultFormat^%apiSQL(<value>)

    For example:

    Do $SYSTEM.SQL.SetToDateDefaultFormat("YYYY-MM-DD HH24:MI:SS")

    The current setting for the TO_DATE() default format can be displayed with:

    Do CurrentSettings^%apiSQL

    or

    Do $SYSTEM.SQL.CurrentSettings()

  • The following CAST and CONVERT operations are now supported for %FilemanDate and %FilemanTimestamp:

    • CAST (<%FilemanDate value> AS CHAR)

    • CAST (<%FilemanDate value> as DATE)

    • CAST (<%FilemanDate value> as TIMESTAMP)

    • CAST (<%FilemanDate value> as VARCHAR)

    • {fn CONVERT(<%FilemanDate value>, SQL_DATE)}

    • {fn CONVERT(<%FilemanDate value>, SQL_TIMESTAMP)}

    • {fn CONVERT(<%FilemanDate value>, SQL_VARCHAR)}

    • CAST (<%FilemanTimeStamp value> AS CHAR)

    • CAST (<%FilemanTimeStamp value> as DATE)

    • CAST (<%FilemanTimeStamp value> as TIME)

    • CAST (<%FilemanTimeStamp value> as TIMESTAMP)

    • CAST (<%FilemanTimeStamp value> as VARCHAR)

    • {fn CONVERT(<%FilemanTimeStamp value>, SQL_DATE)}

    • {fn CONVERT(<%FilemanTimeStamp value>, SQL_TIME)}

    • {fn CONVERT(<%FilemanTimeStamp value>, SQL_TIMESTAMP)}

    • {fn CONVERT(<%FilemanTimeStamp value>, SQL_VACHAR)}

Connectivity Improvements

New Caché 5.1 connectivity features and enhancements:

  • New ECP Cluster Support

  • New SNMP Support

  • New LDAP Client

  • New Mac OS X server support

New ECP Cluster Support

Enterprise Cache Protocol is now supported in shared disk cluster configurations with OpenVMS and Tru64 UNIX®.

Differences between ECP cluster and failover cluster:

  • Faster failover

  • Active shared disk(s).

  • No network reconfiguration.

  • Roll in and out cluster member for repair, upgrade, maintenance and etc.

  • All cluster members are live.

Features

  • ECP Cluster server will provide higher availability.

  • Locks and transactions are preserved during failover.

  • Only the cluster master serves the ECP clients.

  • The cluster members could be used for other applications.

InterSystems strongly recommends the use of ECP for clustered systems. ECP represents a significant advance over predecessor networking approaches such as DCP. Customers currently using DCP for communications among members of a cluster will see improvements in performance, reliability, availability, and error recovery by converting to ECP.

New SNMP Support

To enable monitoring of Caché by a variety of systems management tools and frameworks, support for the Simple Network Management Protocol (SNMP) has been added. The %SYSTEM.MonitorTools.SNMP class allows for control of SNMP agents and functions. This class contains methods to start and stop the Caché SNMP agent, as well as the CreateMIB() method which generates a custom MIB file based on an application description in the Monitor Framework.

For details, see Using SNMP to Monitor Caché in the Caché Monitoring Guide.

New LDAP Client

Programmatic access to LDAP (Lightweight Directory Access Protocol) servers has been added. See the %Net.LDAP.Client.Session class documentation for details.

New Mac OS X server support

This version of Caché now installs and executes natively on Mac OS X 10.3. The installation kit is a standard ".dmg" distribution produced by PackageMaker.

Support has been added for Mac OS X as a server plus the following client components:

  • ODBC

  • JDBC

  • Objects

  • CSP Gateway for Apache

A native Objective-C binding is also available.

Caché 5.1 Upgrade Checklist

Purpose

The purpose of this chapter is to highlight those features of Caché 5.1 that, because of their difference in this version, affect the administration, operation, or development activities of existing systems.

Background

Caché version 5.1 is a significant improvement in functionality and security over its predecessors. In making this advance, InterSystems goal was to provide a compatible evolutionary path forward whenever possible. However, many of the features, such as the System Management Portal, replace functions in previous releases with new mechanisms. Furthermore, the addition of the new security features required a partial redesign and reorganization of the underlying system. These introduced incompatibilities with previous versions of Caché.

Other Resources

Other InterSystems documents describe the features of Caché 5.1 in more depth and breadth. For example,

Administrators

This section contains information of interest to those who are familiar with administering prior versions of Caché and wish to learn what is new or different in this area for version 5.1. The items listed here are brief descriptions. In most cases, more complete descriptions are available elsewhere in the documentation.

New License Keys Required

Note:

Caché version 5.1 introduces new capabilities and a new key format. The license servers from prior releases and the license server for Caché 5.1 do not recognize each others key formats. Existing users MUST obtain new licenses from InterSystems in order to run Caché version 5.1. Please contact your local sales representative to obtain the new keys corresponding to your existing license or to discuss new licensing options.

If a site wishes to run a 5.1 installation on a system where 5.0.x systems will run concurrently, the systems must obtain the correct license keys from their respective servers. This is done by individually setting the ports to the license servers on each system. Caché 5.0.x systems default the license server port to 4001. The license server port for the version 5.1 system(s) should use a different port number for accessing their license server.

Multiple Caché instances that share a key must all be upgraded to 5.1 together.

Recompilation After Upgrade

As noted in the Release Notes, and elsewhere in this document, the changes made to Caché for this version are extensive and pervasive.

Important:

All user application classes must be recompiled after upgrading to version 5.1. And, all user routines that contain embedded SQL statements must also be recompiled as well.

Caution:

Failure to recompile after upgrade may result in unexplained failures during application execution, and possible data loss.

System Management Portal

Prior to this version, how Caché was administered depended heavily on the platform where Caché ran. With version 5.1, InterSystems introduces a new administrative interface that is common across all platforms. Caché 5.1 now uses a browser-based interface, the System Management Portal, for system management.

An advantage of this approach is that (which few exceptions) it is no longer a requirement that any Caché component be installed on the system you use to manage an installation. Remote management of systems over the web, subject to access control established for the site, is now possible and easy. Cross-release compatibility issues are eliminated because both the data and its formatting information come directly from the system being managed.

This new interface subsumes the functions previously distributed among Explorer, SQL Manager, Configuration Manager, and Control Panel functions of the Windows Caché Cube. Because it combines these functions, operators and some developers will also use the portal to accomplish their tasks as well.

Important:

The version 5.1 management portal cannot be used to manage earlier versions of Caché. The opposite is also true; the management functions of earlier versions cannot be used to manage Caché configurations running version 5.1.

More information on the System Management Portal can be found in the System Administrator documentation.

Portal and Application Name Conflicts

In Caché 5.1, the instance name chosen at installation time, is used to construct the name of the CSP application that runs the System Management Portal. For example, assume a Caché system had a CSP application called, “/appserver”. If the installation name chosen for this system was “APPSERVER”, the upgrade procedures would construct a CSP application to run the System Management Portal called, “/appserver/csp/sys”. After the upgrade this would effectively block access to the previously available CSP application.

Note:

When upgrading from an earlier version, care must be taken to ensure that there is not already a CSP application with the same name as the installation (ignoring differences in case).

Security Advisor

To assist system managers in securing a Caché system, version 5.1 includes a Security Advisor. This utility shows current security-related system configuration information, recommends changes or areas for review, and provides links into other system management Web pages to make recommended changes.

Caché 5.1 contains the initial version of the Security Advisor. Its function and range will expand in future versions. It is accessed through the System Management Portal at Home, Security Management, Security Advisor.

InterSystems strongly recommends a review and resolution of the issues raised by the Security Advisor before allowing a secured system to attain production status.

Defaults for Security Settings at Installation Time

The use of Caché security begins with the installation (or upgrade) of version 5.1. During Caché installation, the person doing the installation is prompted to select one of three initial security settings:

  • Minimal

  • Normal

  • Locked Down

The selection determines the initial configuration settings for Caché services as follows:

Security Setting Minimal Normal Locked Down
User authentication required No Yes Yes
Password pattern default 3.32ANP 3.32ANP 8.32ANP
The _SYSTEM user is enabled Yes Yes No
Roles assigned to UnknownUser %All <None> <None>
Create installation username and prompt for password No Yes Yes

And the following table shows what services are enabled by default:

Service Name Minimal Normal Locked Down
%Service_Bindings Yes Yes No
%Service_CSP Yes Yes Yes
%Service_CacheDirect Yes No No
%Service_CallIn Yes No No
%Service_ComPort No No No
%Service_Console Yes Yes Yes
%Service_DCP No No No
%Service_DDP No No No
%Service_ECP No No No
%Service_LAT No No No
%Service_MSMActivate No No No
%Service_Monitor No No No
%Service_Shadow No No No
%Service_Telnet No No No
%Service_Weblink No No No

Emergency Access

As a contingency, Caché provides a special emergency access mode that can be used under certain dire circumstances, such as severe damage to security configuration information or “unavailability” of any users with the %Admin_Manage:U or %Admin_Security:U privileges. (Although Caché attempts to prevent this situation by ensuring that there is always at least one user with the %All role, that user may not be available or may have forgotten his or her password.)

When Caché is running in emergency access mode, only a single user (the “emergency user”) is permitted. Caché is started in emergency access mode through a command-line switch, which passes a user name and password for the emergency user. This user name does not have to be previously defined within Caché. (In fact, even if the user name is defined in Caché, the emergency user is conceptually a different user.) The emergency user name and password are only valid for a single invocation of emergency mode.

The user starting Caché in emergency access mode must have operating-system level system management privileges. (On Windows systems, the user must be a member of the Administrators group. On UNIX® systems, the user must be root. On OpenVMS systems, the user must have a system UIC.) Caché authenticates this user by checking his or her operating system level characteristics. When Caché is started in emergency access mode:

  • The emergency user is the only permitted user. Any attempt by another user to log in will fail.

  • The emergency user automatically has the %All role.

  • The Console and CSP services are enabled. All other services are disabled. This does not affect the enabled or disabled status of services in the configuration; only the current “in memory” information about services is affected.

  • Caché password authentication is used and unauthenticated access is forbidden for all services.

  • If possible, auditing is enabled for all events. Caché startup proceeds even if this is not possible.

Configuration File Changes

One consequence of the new capabilities added in version 5.1 is that major differences have been made to the form and content of the configuration file that provides many of the initialization values when Caché starts up. This section does not detail every change in the configuration file, only the more apparent ones.

Note:

As of this version, the parameter file MUST be named cache.cpf.

New Parameter Reference

If a site controls operation by editing the .cpf, each of these controls must be examined to make sure they are still applicable. Administrators are strongly urged to review the Caché Parameter File Reference book for the current organization, and the valid parameters and their allowed settings.

Startup Check for Configuration Changes

Caché configuration information is stored outside of Caché and (by design) can be modified when Caché is not running. Therefore, a special protective option have been added to this version. Rather than protecting the contents of the configuration file, Caché controls the ability to start the system or modify the configuration of a running system. This protection is enabled by turning Configuration Security on. (Of course, the configuration file can and should be protected outside of Caché by strictly limiting at the operating system level the ability of users to modify that file.)

During startup, if Caché detects that the configuration (.cpf) file has changed since the last time the Caché instance was started, the user running startup will be asked to enter a username and password. This data will be used to verify that the user is authorized to start Caché with altered configuration parameters. If the user is successfully authenticated, and the user has %Admin_Manage:Use, Caché will be started with the new configuration parameters.

Otherwise, Caché will start with the values of the last known configuration. When this happens, the configuration file supplied will be copied to cache.cpf_rejected (overwriting any file by that name), and the configuration parameters actually used to start Caché will be written to the file specified as the configuration file.

Low-level Management Interfaces

In addition to the new Management Portal, system administrators can exercise low-level control over the security of Caché systems through character-oriented utilities. The available routines are described in The CHUI-Based Management Routines.

Web Server Changes

The evolution of security capability in Caché 5.1 has affected the way hypertext information is served to browsers.

The Caché Private Web Server is Now Apache

Each installation of Caché 5.1 also installs an instance of Apache as its private web server. Sites may change the configuration to use a different one, but Caché will always install its private server regardless.

Default Port Changes

By default, Caché now chooses as the superserver port number the first unused port at or after 1972. Applications which depended on the value, 1972, may fail to contact the superserver. In addition, Caché now uses a separate port number for its private web server. The value chosen is the first unused port at or after 8972. Despite both ending in “72”, the two port number are not correlated; the superserver port might be 1973 and the web server port number could be 8977.

During a custom installation, user may explicitly set both the superserver port and the private WebServer port numbers.

CSPGateway Changes

Caché version 5.1 has changed the CSPGateway implementation on two of the platforms.

  • Support Removed for OpenVMS

    The CSPGateway is no longer supported on OpenVMS. The material referencing it has been removed from the OpenVMS installation script and the code is no longer part of the product for OpenVMS.

    Note:

    Any existing CSP Gateway files will be removed during a upgrade from a previous version.

  • WebServer Parameter for OpenVMS

    The OpenVMS .cpf files now contain the WebServer parameter again in the format

    WebServer=[ON|OFF],<server>:<port>
    

    If no such parameter is specified in the .cpf file, the defaults chosen are “OFF”, “”, and the first available port on or after 8972, respectively.

Default Username and Password

When the CSP gateway connects to Caché the first message it sends over is a login message that can contain a username and hashed password defined in the CSP gateway management pages. If the server $username="" which means that CSP gateway did not connect with Kerberos, it will use the default username and password to attempt to login this service.

If this fails, the CSP gateway halts after recording an entry in the audit log (if auditing is enabled). If it succeeds, then $username will not be null and then the CSP server is allowed to call other functions such as ones to display CSP pages. While $username="" the CSP server will only call the login method.

Caché Permissions on UNIX®/Linux are Those of the Installer

In prior versions of Caché, it was necessary to install Caché under the username, root. This was not a good practice because Caché does not require root privilege for normal operation. In version 5.1, this requirement has been eliminated.

When Caché starts, it now sets its userid to that of the user that installed it and its groupid to “cacheusr”. One consequence of this is that devices which are inaccessible to that installer may also be inaccessible to Caché. If you wish this installation to have access to devices available only to root, Caché must be installed by root.

For example, on many UNIX® systems root owns the Ethernet devices. A version of Caché installed by a non-root user would not (by default) be able to communicate using the Ethernet.

Caution:

The “cacheusr” groupid must have write permission for the files needed to operate Caché, for example, the files in the Journal directory and the directory itself. Failure to meet this requirement will result in erratic operation possibly leading to system failure.

This warning extends to any file and/or directory used by Caché created by an administrator outside of Caché. For example, if the journal files are assigned to a separate disk to improve throughput and system reliability, it is not enough that they be created by a user with, for example, “root” access. They must be writable by the “cacheusr” group.

New Password Hash Function

Caché has always stored its password data for users as the result of applying a hash function to the characters of the password. When you attempt to login, a hash of the characters you enter is calculated and it is compared to the hashed value stored internally. If the two calculated values match, the passwords are assumed to match.

This version of Caché uses a computationally stronger function to compute the password hash; one that produces different hash values than before and is harder to “crack”.

Because different password strings can hash to the same value, there is no way to compute the actual user's password starting from the hash value. This means there is no way to compute the new password hash value starting with the old hash value. Therefore, all userids on the system must be given new password hash values when upgrading to version 5.1.

User data exported by prior versions of Caché (for example those produced by $SYSTEM.SQL.Export(...)) contains the password hash values used for that version. Care must be taken when importing such data into version 5.1. All such users will need new passwords assigned. Users whose ids are imported will have their password hashes reset preventing them from logging in until it is reassigned under 5.1.

An exception to this is users who have null (no assigned) passwords. These users will be processed automatically.

Note:

After an upgrade, Caché 5.1 will assist in the computation of the new password hash values. When a user attempts to login for the first time, the password hash will be calculated using the previous algorithm. This value will be compared against the stored value. If they match, the password hash will be recalculated using the new algorithm and this new value will be stored in the database. Thus the conversion of passwords will be made as existing users login for the first time.

Read-Only Databases

Representation

To improve consistency in handling read-only databases, the way they are identified to Caché has changed in this version. Caché now recognizes read-only databases that are marked as such in properties for that database, or through declared read-only via the ^MOUNT.

Write Daemon Access Determines Database Mode

In Caché version 5.1, when a database is mounted the write daemon checks whether it has sufficient permission to update the database. If it does not, it will force the database to be mounted in read-only mode.

Cluster Changes

Improved Cluster Join Logic

Caché 5.1 has been enhanced so that a cluster member is no longer allowed to fully join a cluster while the trio of switches 10, 13, and 14 (which disables database access) is set on the cluster master. In prior releases, the new system would be allowed to cluster mount and read/write from databases. If the cluster was in the process of performing a backup, this could cause problems.

Now the new cluster member will detect the switches which have been set cluster-wide and set those switches locally while it starts up. This may mean that the Caché startup process on the member attempting to join the cluster will hang if a switch is set which blocks global access. A console log message will be generated if this occurs.

Note:

This version can interoperate with older versions but the new functionality will not be present unless the master and the system joining the cluster have not been upgraded to version 5.1.

Journaling Changes

As a result of experience with prior versions of Caché, journaling in version 5.1 has been significantly improved as one part of the underlying support for highly available systems. The goal of the changes in 5.1 has been to make the behavior of journaling safer and more consistent; for example, journaling is now a property of databases rather than individual globals. The operator interface has been changed to incorporate this new approach, and Caché 5.1 provides for common management and auditing of changes to journal settings via the System Management Portal.

Journaling is Now a Database Attribute

Caché 5.1 sets the specification of the journal state on a databases basis. This greatly improves reliability of the system because it addresses inconsistencies (after crash recovery) that could arise in earlier versions due to change in globals that may or may not be journaled, and that may or may not be involved in transactions explicitly or implicitly via %Save or SQL UPDATE statements. The changes in more detail are:

  • The journaling state is a property of databases, not individual globals. All globals within a database are journaled or not, depending on this setting. There are only two states - YES and No.

  • The default setting of the journal state for new databases is YES. When a database from an earlier version is first mounted, the value is set to YES, regardless of the previous setting for "default for new globals" and regardless of the settings of individual globals within that database.

  • In a transaction, Caché writes changes into the journal, regardless of the settings of the databases in which the affected globals reside. Rollback will work as before.

  • Nothing mapped to CACHETEMP is ever journaled; its journaling behavior is unchanged.

  • Journal restore respects the current settings of the database. Nothing is stored in the journal about the state of the database when the journal is written. The state of the database at time of restore determines what action is taken. This means that changes to databases with JOURNAL=YES will be durable, but changes to other databases may not be. Caché will ensure physical consistency, but not necessarily application consistency if transactions involved databases with JOURNAL=NO.

  • Databases mounted on a cluster have their globals journaled or not, depending on the database setting.

  • The setting for a database can be changed on a running system. If this is done, the administrator will be warned of the potential consequences and the change in state audited.

In recognition of this change, Caché 5.1 also:

  • changed the default purging settings to somewhat mitigate the diskspace consequences of this change;

  • removed the routine, ^%JOURNAL, which in prior releases enabled or disabled journaling on a per-global basis;

  • modified ^%GCREATE and ^%SYS.GCREATE so they no longer ask whether to journal globals.

Caution:

One aspect of the new journal design is that restores are performed only to databases marked to be journaled at the time of a journal restore. The ^JRNRESTO program now checks the database journal state the first time it encounters each database and records the journal state. Journal records for databases not so marked are skipped during restore.

If no databases are marked as being journaled, the ^JRNRESTO program will ask if the operator wishes to terminate the restore. Administrators can change the database status to journaled and restart ^JRNRESTO if desired.

Journaling Z-Globals

In prior releases, the JournalZGlob parameter was used to indicate whether z/Z* globals should be excluded from journaling (even inside transactions). In version 5.1, to make journaling more robust, it has been removed. When upgrading an earlier Caché system with the flag set, the existing individual z/Z* globals in every defined database are given the journal attribute of that database. (For CACHETEMP, the journal attribute defaults to off).

If a site needs to exclude new z/Z* globals from journaling, the administrator will have to map z/Z* globals to a database with the journal attribute turned off.

Note:

Since the globals in a namespace may be mapped into different databases, some may be journaled and some not. It is the journal setting for the database to which the global is mapped that determines how the global will be treated.

To replicate the behavior in prior versions, when the flag to exclude journaling of z/Z* globals is set, the z/Z* globals in every namespace must be mapped to the CACHETEMP database. The difference between CACHETEMP and a database with the journal attribute set to off is that nothing in CACHETEMP, not even transactional updates, gets journaled.

Changes to Journal Purge

Prior to this release, the default behavior was to purge the journal files after 7 days. In Caché version 5.1, the default has been changed.

You may have Caché purge journal files after either X days, or Y successful backups have occurred. Normal recommended settings are:

  • 1 <= X <= 100

  • 1 <= Y <= 10

If both X and Y are > 0, files will be purged after X days or Y successful backups, whichever comes first. If either X or Y is zero, purging is done on the basis of the remaining criteria. Setting X and Y to zero prevents purging entirely.

Journal files are now purged after 2 consecutive successful Caché backups.

Note:

Those customers who do not use Caché backup facilities should consider scheduling the appropriate journal maintenance using, for example, the Caché Task Manager to manage the amount of journal information retained.

Shadowing Changes

This version of Caché has significantly improved facilities for system shadowing. It is better at shadowing to and from clusters. The latency reporting on both the sender and shadow systems have been improved and there is better control over suspending/resuming and starting/stopping shadowing.

Journal Applied Transactions to Shadow Removed

The setting to choose whether or not to journal applied transactions on shadow databases no longer exists. In earlier Caché releases the default behavior was to not journal updates to shadow databases; but you could enable journaling on the shadow by selecting the Journal Applied Transactions check box. This maintained a separate journal file showing the activity on the shadow database.

With the option removed in Caché 5.1, journaling of the shadow databases is determined by the global journal state of the databases themselves. After an upgrade, journaling on the shadow databases is enabled. To mitigate the increased demand on the storage capacity of the shadow, Caché purges the destination shadow copy of a source journal file once it is dejournaled and does not contain any transactions open on the shadow.

InterSystems recommends you journal all databases that are the destination of shadowing. However, if you do decide not to journal the destination shadow databases, you must also disable journaling on the CACHESYS database. Caché stores the journal address and journal file name of the journal record last processed by shadowing in the ^SYS global in the CACHESYS database. This serves as a checkpoint from which shadowing will resume if shadowing fails.

Caution:

On the shadow destination, if you journal the CACHESYS database, but not the destination shadow databases, there is the possibility that if the shadow crashes and restarts, the checkpoint in CACHESYS could be recovered to a point in time which is later in the journal stream than the last record committed to the shadow databases.

Compatible Mode (Record Mode) Shadowing Removed

There is no longer an option to choose the method of journal transmission. All shadowing uses the fast mode, apply changes method.

Prior to Caché version 5.1, there were four methods of journal transmission for shadowing:

  • Fast mode, apply changes

  • Fast mode, don’t apply changes

  • Compatible mode, apply changes

  • Compatible mode, scan changes

Compatible mode (previously called record mode) was most often used for compatibility among heterogeneous platforms, and sometimes to support different Caché releases. Fast mode (previously called block mode) now supports heterogeneous platforms since it automatically performs any necessary byte reordering for different endian systems.

If you wish to support multiple production servers running different Caché releases from a single shadow, then InterSystems recommends that you set up multiple Caché instances on the shadow server, one for each Caché version, and use fast mode rather than compatible mode on older versions. This provides the best performance and reliability.

Important:

A Caché upgrade converts existing compatible mode shadows to fast mode. The converted fast mode shadows may or may not work with the sources, depending on the source configuration. Caché 5.1 automatically performs endian conversion for fast mode shadowing.

Changes in Shadowing Defaults

In Caché 5.1, the following databases are not shadowed by default:

  • CACHEAUDIT

  • CACHELIB

  • DOCBOOK

  • SAMPLES

You can click Add next to the Database mapping for this shadow list on the Home, Configuration, Shadow Server Settings, Edit Shadow Server page of the System Management Portal if you wish to shadow them.

CACHETEMP

Caché 5.1 handles CACHETEMP differently from its predecessors. The changes are a result of security requirements and customer requests.

Expansion and Size Characteristics Preserved

Caché 5.1 preserves the expansion and size settings of CACHETEMP across restarts. After restart, the size reported by Caché will be the minimum of 240MB or the allocated size of the file, whichever is smaller. If the size of the file allocated by the operating system is larger than 240MB, Caché will only initialize the map blocks to describe the first 240MB and will expand the map later as needed. It will not, however, shrink the physical size of the file.

Collation

After restart, the collation of CACHETEMP will be reset to Caché Standard regardless of its prior setting. Those sites that wish a different collation should add code to the “SYSTEM” callback of the ^%ZSTART routine to set the collation desired.

Conditions for CACHETEMP Deletion

Under the following circumstances:

  1. CACHETEMP is a 2KB database

  2. CACHETEMP is mounted when STU (system startup) runs

Caché attempts to delete and recreate CACHETEMP. Condition #2 occurs, for example, if Caché is started in “nostu” mode, and then the operator later runs STU manually.

When CACHETEMP is recreated, the initial size is set to 1MB, the expansion factor to 0 (indicating growth by the larger of 10% or 10MB), and the maximum size to 0 (no limit).

ShutDownTimeout Parameter Now Enforced

Beginning with version 5.1, the ShutDownTimeout parameter will be enforced on all platforms. Shutdown will not spend more than the value of ShutdownTimeout (less about 10 seconds) in user-defined shutdown routines. Once the limit is reached, shutdown will proceed to completion (including final force cleanup) even if user-defined shutdown routines have not completed.

Collation for Locales Now on by Default

When a national collation is available in a locale (for example: Spanish1, Portuguese2, German2), it is now set as the default collation for that locale, instead of "Cache Standard". When a locale has more than one collation (such as German1 and German2), the one with the greatest suffix was selected.

Locales that don't have national collations (English, Hebrew, and so on), continue using "Cache Standard" as their default collation. The changes are summarized in the following tables:

Note:

This affects only the creation of local arrays, because new globals have their collation taken from the database's default (unless explicitly created by %GCREATE).

Locales Whose Default Collation Changed
Locale Collation
chsw Chinese2
csy8 Czech2
csyw Czech2
dan8 Danish1
danw Danish1
deu8 German2
deuw German2
ell8 Greek1
ellw Greek1
esp8 Spanish1
espw Spanish1
fin8 Finnish1
finw Finnish1
plk8 Polish2
plkw Polish2
ptb8 Portuguese2
ptbw Portuguese2
rus8 Cyrillic1
rusw Cyrillic1
ruw8 Cyrillic2
zdsw Japanese1
zduw Japanese1
zip8 Portuguese2
Locales Whose Default Collation Remains Caché Standard
Locale Collation
chtw Traditional Chinese
enu8 English
enuw English
fra8 French
fraw French
heb8 Hebrew
hebw Hebrew
ita8 Italian
itaw Italian
jpnw Japanese
jpuw Japanese (UNIX®)
jpww Japanese UTF-8
korw Korean
nld8 Dutch
nldw Dutch
zdtw Japanese (DTM-J)

Accessing the Online Documentation

On Windows, when trying to access the documentation via the Cube, the userid assigned for the attempt is “UnknownUser”. When installing Caché with a security level of Normal or Locked Down, this username only has %DB_DocBook:R permission.

This is insufficient to read the Caché class reference documentation. Access to the class reference documentation requires that the user attempting to read the class documentation be authenticated.

Running program examples in the online documentation requires %DB_SAMPLES:W. If UnknownUser lacks this permission, then the button labeled Run It will not appear in any of the executable program examples.

Defining one or more roles which have the necessary permissions and assigning it to UnknownUser will establish the prior behavior. Alternatively, you may edit the application definition of “/csp/docbook” to add the role(s) whenever it is run.

Upgrading from a Prior Release

This section covers issues related to upgrading an existing Caché system to version 5.1.

No Upgrade from Field Test Versions

Customers running on any Caché 4.1.x or 5.0.x version may upgrade to Caché 5.1 at installation.

Caution:

InterSystems does not support an upgrade from any of the versions used for field test of Caché 5.1. This includes the version of Caché 5.1 distributed to selected customers at DevCon 2005.

Use of DDP

If you were running DDP on an earlier version of Caché, you must edit your configuration file to allocate the proper number of network slots. They are no longer calculated by default.

  • In the [Net] section of the configuration file, set the value of maxdsmport to the number of ethernet cards used for DDP.

  • In the [config] section of the file, change the fourth parameter of the LegacyNetConn from 0 to 1.

Note:

DDP will not start if these changes are not present.

$SYSTEM.OBJ.UpgradeAll()

The change in the compiler version, the reorganization of globals and routines, and the changes in Caché classes may generate a bewildering swarm of errors if $SYSTEM.OBJ.UpgradeAll() is invoked without prior planning and preparation.

Synthesized Role: %LegacyUnknownUser

In order to mimic prior behavior, during upgrade to version 5.1 a default role is created. This role is named %LegacyUnknownUser. The idea is that after upgrade from 5.0 and earlier versions of Caché where advanced security was not implemented, it will be common for no users to be defined. In this case, all users will be logged in as UnknownUser. If UnknownUser has no access privileges, the customer's operations will not be accessible to existing users until the administrators configure the system.

The %LegacyUnknownUser role is granted Read/Write access to the resource created for each customer-defined database that exists at the time of the upgrade installation and the resources shown

Name:               %LegacyUnknownUser
Description:        Legacy Unidentified Users
Roles granted by this role:
                    <none>
Resources owned by this role:
                    Resource                      Permission
                    --------                      ----------
                    %System_CallOut               U
                    %Service_SQL                  U
                    %Service_Object               U
                    %Service_Console              U
                    %Service_CallIn               U
                    %Service_CacheDirect          U
                    %Development                  U
                    %DB_USER                      RW
                    %DB_SAMPLES                   RW
                    %DB_%DEFAULT                  RW
Users owning this role:
                    <none>

In addition, use access to the following service resources will be granted subject to the indicated conditions:

Service Privilege Condition
%Service_ComPort
Use On Windows, if service is enabled
%Service_Console
Use On Windows, if service is enabled
%Service_LAT
Use On Windows, if service is enabled
%Service_Telnet
Use On Windows, if service is enabled
%Service_Terminal
Use On UNIX® and OpenVMS

After the administrator has configured the system appropriately, the UnknownUser user can either be disabled or the resources assigned to the role%LegacyUnknownUser can be gradually reduced via ^SECURITY or the System Management Portal as additional aspects of the application environment are brought under the control of Caché Advanced Security. This reduction of the privileges of the %LegacyUnknownUser role or its removal is a manual step in the transition. It is not done automatically by Caché.

%LegacyCD and %LegacySQL

These are applied automatically to existing users only during upgrades to ensure that those users continue to have the same level of access in 5.1 that they did previously. New users aren't required to have these roles specifically.

Allow %-Global Access as in Previous Versions

The value of Security.System.PercentGlobalWrite is set true for upgrades. (For new installations it is set false.) This makes access to %-globals consistent with earlier versions. The value can be changed via the ^SECURITY routine.

All Members of a Cluster Must Run the Same Caché Version

All members of an ECP cluster must be running the same version of Caché. If you upgrade one, you must upgrade all the rest.

Removal of CSP Gateway On OpenVMS Upgrade

The CSPGateway is no longer supported on OpenVMS. The material referencing it has been removed from the OpenVMS installation script and the code is no longer part of the product for OpenVMS.

Note:

Any existing CSP Gateway files will be removed during a upgrade from a previous version.

Removal of Global & Package Mappings

During an upgrade from an earlier version, the following mappings to globals will be removed:

  • all globals whose names begin with “^odd”

  • ^rINDEXCLASS, ^rOBJ, ^ROUTINE

  • ^mdd

Packages whose names start with “%Z”, “%z”, “Z” and “z” will have their definitions retained (“^oddDEF”), but will have their compiled class information removed. The “^odd” globals will be recreated when the classes are recompiled via $SYSTEM.OBJ.Upgrade().

In addition, ALL class mappings that were defined in the configuration file (.cpf) will be discarded.

Note:

If access to these globals is required, the administrator must manually construct the required mapping; they cannot be automatically converted. This will be the case if, for example, the system had defined global mappings so that multiple namespaces could share the same class definitions.

Trusted Application Definitions Removed

The Caché 5.1 security model does not support trusted applications. Anytime a user connects (or reconnects), he or she is prompted for a password if password authentication is turned on. If this is not what is desired, the administrator should turn authentication off for the Cache direct service.

Note:

Any 5.0.x trusted application definitions are thrown away during a 5.1 upgrade.

Windows Network Server Username and Password Change

In previous versions, Caché would install and start its services on Windows with a default username of “_SYSTEM”, and a password of “_sys” unless configured otherwise. The values for the username and password were set via the Configuration Manager using the Advanced tab and the “Input/Output” option.

In version 5.1, during upgrade, these values (or the default values if none were set) will be stored in the Windows service definition created for the upgrade.

Administrators may change the values using the Windows Management Interface. From the Windows Start menu, navigate to Programs, then Administrative Tools, and finally Component Services. Select the appropriate Caché service and from the Action menu choose Properties. The username and password are accessed via the Log On tab.

Global Replication Removed

Global replication is no longer supported in Caché 5.1. If found, the upgrade process will remove its use from the system and note this fact in the console log. The capability formerly provided by this can now be achieved by the use of shadowing. Please consult the Shadowing chapter of the Caché Data Integrity Guide for details.

Java and Kerberos

Before using Java on Caché with Kerberos, you must edit certain configuration files, among them krb5.conf. Parameters in this file are set by running

java com.intersys.jgss.Configure

and responding to the prompts. On Windows, Solaris, and Linux, if krb5.conf is not found in the default location, Configure will search for it in the following locations:

  • Windows

    c:\winnt\krb5.ini 
    
  • Solaris

    /etc/krb5/krb5.conf 
    
  • Linux

    /etc/krb5.conf 
    

to obtain any template file information to be used when the file is created in the default location.

Recommendations

InterSystems has several recommendations to administrators setting up a Caché 5.1 system.

Enterprise Cache Protocol (ECP)

InterSystems strongly recommends the use of ECP for distrubuted systems. ECP represents a significant advance over predecessor networking approaches such as DCP. Customers currently using DCP will see improvements in performance, reliability, availability, and error recovery by converting to ECP.

Change Default Password Setting

When installing Caché with a security setting of “Minimal”, the default passwords of all users created are set to “SYS”. InterSystems suggests strongly that the password of these users be set to a different value as soon as possible so that, even though the security level of the system is low, control over access is established from the start of operation.

CACHELIB as Read-Only Database

In Caché version 5.1, for security reasons InterSystems has made CACHELIB a read-only database. This is a change from the previous practice. InterSystems strongly recommends that sites maintain CACHELIB as a read-only database. Those site- and application-defined globals, classes, tables and so on which might previously have been placed in CACHELIB should be moved elsewhere.

Limitations

The following limitations apply to Caché 5.1 or specific operating systems running Caché:

Maintaining Information Coherence Across Systems

On clustered Caché systems, it is highly desirable to maintain the same list of users, roles, applications, and so on across all the systems of the cluster. The initial release of Caché version 5.1 does not provide facilities for propagating changes in one system to others. This must be addressed by assuring that administrators manually make the same changes on each system.

Maintaining Coherence with Kerberos

Sites using Kerberos as the authentication mechanism must manually propagate changes to the list of valid users held by Kerberos to Caché. Future versions of Caché may provide mechanisms for doing this automatically but the initial release does not.

Consolidating Audit Logs

If a site wishes to run audit reports, or other analyses of their devising, on the audit data from several systems (for example, all the systems of a cluster), the individual audit logs must be consolidated manually.

Write Image Journal Files

The format of the WIJ (write-image journal) file has changed for Caché 5.1 to improve recovery in clustered systems. This has two consequences:

  1. If an unresolved failure remains on any system to be upgraded, be sure to restart Caché and do a recovery before you upgrade to the new version.

    If you do not, the journal purge utility will not recognize journal files in the old format and will complain that there are corrupt journal files. To avoid this error, move the old journal files to a backup directory using the appropriate operating system commands before beginning the upgrade.

  2. All members of an ECP configuration must be running the same version of Caché. If you upgrade one, you must upgrade the rest as well.

If you need to restore an older journal file to Caché 5.1, you can use the JConvert and %JRead routines.

Shadowing

A Caché upgrade converts existing compatible mode shadows to fast mode. The converted fast mode shadows may or may not work with the sources, depending on the source configuration. Caché 5.1 automatically performs endian conversion for fast mode shadowing.

Compatible mode (previously called record mode) is not supported.

Important:

Shadowing in Caché 5.1 is not compatible with any prior release of Caché. Both the source server and destination shadow must be running on Caché 5.1.

Clusters

Caché 4.1, Caché 5.0, and Caché 5.1 clusters can coexist on the same hardware, but they cannot cluster together. If these clusters need to communicate with each other they need to use DCP, or preferably, ECP.

Management of Non-% Variables in Embedded SQL

Any non-% variables used by embedded SQL statements within an ObjectScript procedure need to be added to the procedure's public variable list and be Newed within the procedure. While this is still a limitation in Caché, a change has been made to the macro preprocessor to make it easier to manually add these variables to the public and new lists.

When the Home, Configuration, Advanced Settings SQL setting Retain SQL Statement as Comments in .INT Code is “Yes”, along with the SQL statement in the comment, the non-% variables used by the SQL statement are listed in the comment text. This variable listing makes it easier to identify and cut and paste the variable lists into the MAC code public list and a new list.

Unicode in Global Names

Support for Unicode in global names is not yet fully operational and should be avoided.

Caché RPM kits

The Caché RPM kit installs into /usr/cachekit/5.1. Your /usr directory may be mounted read-only or may contain little free space, so you may want to change the location.

Database Interoperability

Databases created on earlier versions of Caché can be mounted on version 5.1 and, once they are upgraded, can be used there. But this process is not reversible. Upgraded databases cannot be moved back to earlier versions.

Note:

InterSystems advises users who wish to move data bi-directionally between systems running version 5.0 and 5.1 to use the %GOF/%GIF routines to move data between the versions.

Upgrade Only Processes Local Databases

In Caché 5.1, $SYSTEM.OBJ.UpgradeAll() only scans local for local databases to upgrade. It ignores remotely mounted databases. These must be upgraded by running UpgradeAll() on the remote systems.

Caché Versions for ECP

Because of the upgrade to the compiler, systems in an ECP configuration must either be:

  • all on version 5.1, or

  • the data server must be on version 5.1 and the application servers can be either on version 5.0 or version 5.1.

Note:

It is possible to run version 5.1 application servers with version 5.0 data servers, but this requires that the routines used by the application servers be mapped to databases local to the application servers. If you believe you have need to do this, please contact the InterSystems Worldwide Response Center (WRC) for assistance.

Moving Applications from 5.1 to Earlier Versions

Porting an application from Caché 5.1 to an earlier release is problematic and depends on what features in this version the application depends on (compiler behavior, new streams implementation, changes in exported XML for applications, new class representations — to name a few). If you believe you have need to do this, please contact the InterSystems Worldwide Response Center (WRC) for assistance.

ODBC and JDBC Compatibility

Due to a change in protocol, the ODBC and JDBC clients supplied with Caché 5.1 are compatible only with Caché servers from version 5.0.13 and later. Attempts to use connections to Caché servers in versions before 5.0.13 will result in errors.

RoseLink

RoseLink currently attempts to access Caché using only the standard SQL username and password. Therefore, it will not be supported on systems whose default installation security level is Normal or Locked Down. This restriction will be lifted in the next maintenance version of Caché 5.1.

In addition, the user must have %Development:Use permission in order to access classes for its use.

Dreamweaver

The connection that Dreamweaver MX uses to access Caché is not available with this version. This restriction will be lifted in a future maintenance release of Caché 5.1.

Perl and Python Language Bindings

The Perl and Python language bindings are supported only on 32-bit versions of Windows.

C++ Language Binding

The C++ language binding is supported only on the Windows platform using Visual Studio 7.1.

Platform-Specific Items

This appendix holds items of interest to users of specific platforms.

Windows
  • Help Format Change

    The usage information for the commands css.exe and ccontrol.exe is now provided in HTML. Executing either command with a first argument of “help” will now invoke the default browser to display the help file.

  • New Japanese Locale

    There is now a new Japanese locale for Windows (jpww). It is like the standard jpnw locale except that the default for Telnet Terminals is UTF8 instead of SJIS. This new locale is now installed by default for new Japanese installs on Windows. Upgrades to Caché maintain the previous locale.

  • Changes to %path% Environment Variable

    To improve system security and the uniformity of locating Caché components, version 5.1 adds the file system directory name

    \Program Files\Common Files\InterSystems\Cache
    

    to the %path% system environment variable on Windows systems. It is added to the HKEY_LOCAL_MACHINE hive so that it applies to all users of this machine.

  • Visual Studio 7.1

    On Microsoft Windows, Caché is now compiled with Visual Studio 7.1. User applications communicating with Caché (for example, those using the CALLIN or CALLOUT interfaces) must be upgraded to this version of Visual Studio.

Windows XP Professional

Mapped drives are not supported on Windows XP Professional — Due to security improvements in Windows XP Professional, Microsoft discourages users from using mapped drives; using them results in different behavior than in the past.

We recommend that XP Professional users follow these procedures to access mapped drives from the GUI tools or from telnet sessions:

  • For remote mapped drives, enter the user name and password in your configuration as before. In addition, edit the ZSTU startup routine and add this line for each drive you have mapped.

    Set x=$zf(-1,"net use z: \\someshare")
    Copy code to clipboard
  • For virtually mapped drives, add this line for each drive mapped with the subst command:

    Set x=$zf(-1,"subst q: c:\somedir\someotherdir")
    Copy code to clipboard

You cannot add more mappings after startup.

Important:

The above procedure is meant for development situations where only one user is expected to log on to Windows, and the user name entered in your configuration is the same user. In any other situation, such as a Terminal Server environment, the results are unpredictable.

The following notice from Microsoft refers to this problem:

[Redirected Drives on Windows XP Professional: On Windows XP Professional, drive letters are not global to the system. Each logon session receives its own set of drive letters A-Z. Thus, redirected drives cannot be shared between processes running under different user accounts. Moreover, a service (or any process running within its own logon session) cannot access the drive letters established within a different logon session.]

Another approach to using the mapped drives is to start Caché like this:

\cachesys\bin\ccontrol start configname
Copy code to clipboard

With this approach you do not have to add anything to the ZSTU routine, and you do not have to enter a user name and password. In addition, drives you map or map with a path using the subst command after startup are available. The limitation of this approach is that Caché only runs as long as the user that starts Caché stays logged on.

Windows Enterprise Server 2003

The version of Internet Explorer distributed with this version of Windows has every security related configuration option disabled. The result is that various pages displayed by the System Management Portal are affected; for example, information generated by scripts will not materialize because the scripts will not be run. The proper behavior can be restored by changing the Internet security level setting from “High” to “Medium”.

The first time a user accesses the System Management Portal on a particular system, Internet Explorer will prompt to ask if this site should be added to the “trusted” list. Answering in the affirmative, will also change the Internet security level for that site to Medium.

Mac

Support for Xalan, an XSLT (Extensible Stylesheet Language Transformation) processor, is only available on OS x 10.4.

OpenVMS
  • ECO Required for Access Using Kerberos on Itanium

    Applications attempting to access OpenVMS servers that use Kerberos authentication must install the patch, HP-I64VMS-TCPIP-V0505-11ECO1-1, available at the ftp://ftp.itrc.hp.com/openvms_patches/layered_products/i64/ ftp site. The ECO is for TCP/IP, not the actual operating system. Without this patch, the server will often transmit erroneous response packets back to clients using the C++ binding, ODBC, JDBC, and Studio.

    Note:

    This ECO applies only to OpenVMS on Itanium hardware. It is not needed for OpenVMS on Alpha.

  • CSP Gateway Removed

    Support for the CSP Gateway on OpenVMS has been removed.

  • Password Masking Limitation in GSS

    When attempting to access Caché via JDBC on systems using Kerberos, if no credentials for the user are found, and the identity of the user is not supplied by the caller, JDBC will ask Kerberos to authenticate the caller. When this happens, due to the characteristics of terminal IO on OpenVMS, echoing of the password will neither be suppressed nor masked.

  • Using the SOAP Client Wizard from Studio

    An attempt to start the SOAP wizard from Studio will fail unless the application, /isc/studio/template, is set up to point to the current web server used for OpenVMS.

  • Caché Processes and /SYSTEM

    All processes that are part of Caché ru with UIC=[1,4]. Therefore, all Caché-related logical devices used by these processes, for example, those mentioned in the .cpf file, must be defined in the system table (defined with /SYSTEM) to avoid access errors.

  • WebServerName and WebServerPort

    In version 5.1, Studio is unable to access the Class Documentation unless both the WebServerName and the WebServerPort are defined. These are found in the Miscellaneous category of the System Management page, Home,Configuration,Advanced Settings.

AIX®
  • IBM Java Runtime and Kerberos

    On systems using the IBM Java runtime environment (AIX® 32–bit, 64–bit and SUSE Linux Enterprise Server), use of kinit is not compatible with Kerberos principal name and password prompting, or the principal name and password API. To use kinit, change the file

    ${java.home}/lib/security/iscLogin.conf
    

    so that the module, com.sun.security.jgss.initiate, has the option

    useDefaultCcache=true
    

    With this runtime, only the Java routine at

    {java.home}/bin/kinit
    

    works and not the native Kerberos routine at

    /usr/krb5/bin/kinit
    
  • NFS-Mounted Filesystems And Exclusivity

    Cache uses O_EXCL (exclusive access) when creating Caché database (.dat and .ext) and lock (.lck) files. However, it is a known limitation that NFS does not guarantee this exclusivity.

Linux

On Netapp NFS-mounted filesystems under Linux, a file created by a suid:sgid executable has different, non-UNIX® standard, owners than on standard filesystems. The sgid bit on the executable fails to take effect, while the suid bit succeeds in setting the owner of the file to the owner of the executable. This behavior has been observed only on Netapp systems.

Red Hat 3.0 / 4.0 And IBM WebSphere MQ

If you plan to use the MQ interface, IBM WebSphere MQ version 6.0 is required when running Caché 5.1 on Red Hat version 3.0 and 4.0.

Linux / AMD64

When upgrading from Caché from a Linux implementation on an Intel processor to Linux on AMD64, a new Caché license is required. As noted on the InterSystems Web site:

“Because of the significant differences between 32-bit and 64-bit CPUs, InterSystems delivers different Caché software for them and, consequently, they are different platforms for licensing purposes. As a result, Platform Specific Caché licenses cannot be transferred from one to the other. (Normal trade-in policies apply.) Platform Independent licenses can, of course, be transferred at no charge.”

SUSE Linux Enterprise Server
  • IBM Java Runtime And Kerberos

    On systems using the IBM Java runtime environment (AIX® 32–bit, 64–bit and SUSE Linux Enterprise Server), a different Kerberos kinit is needed. See the description includes with AIX®.

  • Terminal With Kerberos Authentication

    SUSE Linux Enterprise Server 9 running on AMD64 handles packages slightly differently from other versions of Linux. A result of this is that attempting to use Terminal on this system with Kerberos authentication may encounter errors in the case where the installer has not chosen to install the developer packages. In this instance, the following packages must be installed to ensure proper operation:

    • heimdal-devel

    • heimdal-devel-32bit

    The packages are most easily located by using the search facility to locate all the packages whose name begins with “heimdal”. In most installations (except “full”), the list will show the two packages named above as unselected. Select them and continue with the installation.

UNIX®

Users may install Caché on UNIX® so that cachesys is not the default directory. The directory path is assumed to be in an environment variable, CACHESYS. The ccontrol and csession commands use this environment variable. If it is defined at installation time, Caché is installed in that directory. If it is not defined, Caché is installed in the standard UNIX® location, /usr/local/etc/cachesys.

Both ccontrol and csession expect to find the registry in the same directory where their executable was found. For security reasons, ccontrol verifies that the protection on the registry is root as the owner and writable only by root.

Tru64 UNIX®

For Tru64 systems, unlike other UNIX® file systems, group ownership does not come from the group id of the creating process. Instead, the group ID of the file is set to the group ID of its parent directory.

However, when the vfs subsystem attribute “sys_v_mode” is set to 1, the group ID of the file is set either to the group ID of the process or, if the S_ISGID bit of the parent directory is set, to the group ID of the parent directory If the group ID of the new file does not match the effective group of the process or one of its supplementary group IDs, the S_ISGID bit of the new file is cleared.

In general, this will present no problems since the groupid of all directories created by Caché utilities is properly set to the correct group owner. But there are circumstances which can cause problems. For example, if an administrator uses ^DATABASE to create a database in an nonexistent directory, ^DATABASE will create the directory, but it does not adjust the groupid of the newly-created directory, which is inherited from the parent directory. As a result, the database, with its groupid inherited from the directory, may be inaccessible to cacheusr. Other Cache utilities (e.g., journal and shadow) that create directories have the same problem.

Note:

It is recommended that System Administrators set the sys_v_mode to 1 on all file systems and directories used by Caché to ensure smooth functioning of the system. For further information, please refer to the manpages for the open(2) system call.

HP-UX

The Caché cryptographic random number generator (use, for example, to encrypt and decrypt databases) requires a source of true randomness (entropy) in order to initialize its internal state. All supported UNIX® platforms except HP-UX 11i provide the special device file, /dev/urandom, that provides a source of true entropy based on kernel thread timings. On HP-UX, this functionality is part of the HP-UX Strong Random Number Generator available as a free, optional component supplied and supported by HP.

Caution:

If this component is not installed, Caché uses other sources of entropy available on the system. However, these have not been analyzed for randomness, and therefore the encrypted values generated by Caché are not as strong as they could be otherwise.

Solaris

Applications running on Solaris will fail to obtain an initial set of credentials when using a password. This happens, for example, when trying to access a Caché instance requiring Kerberos authentication via TERMINAL. Sites intending to use Kerberos authentication with Caché will require patches to Solaris, namely,

  • For Solaris 10, 121239–01 and 120469–03 (or greater).

  • For Solaris 9, 112908–22 and 112907–06 (or greater).

Developers

This section contains information of interest to those who have designed, developed and maintained applications running on prior versions of Caché. Although InterSystems placed great importance on upward compatibility in version 5.1, the increased emphasis on security resulted in the redesign and re-implementation of some core parts of Caché. The effects of this necessarily affect existing applications.

The items listed here are brief descriptions. In most cases, more complete descriptions are available elsewhere in the documentation.

System Management Portal

Prior to this version, how Caché was administered depended heavily on the platform where Caché ran. With version 5.1, InterSystems introduces a new administrative interface that is common across all platforms. Caché 5.1 now uses a browser-based interface, the System Management Portal, for system management.

Although mainly for administrators and operators, developers may occasionally need to use some of its functions. A brief summary can be found in the Administrator section of this document and more complete information on the System Management Portal can be found in the System Administration documentation.

Privileged Operation

Caché has always had the concept of “privileged” operations. In Caché 5.1, this concept has been more clearly defined, strengthened and made more granular. Commands, routines, functions, methods and so on that are privileged must meet one of two criteria before Caché will allow them to proceed:

  1. They must be invoked by a unmodified routine that is loaded from the CACHESYS database.

  2. They are invoked by a user who holds a role granting permission to perform the operation. In most cases, privileged operations require %DB_CACHESYS:W, but certain operations may deviate from this.

It either of these conditions is true, then the requested operation will proceed.

Recompile User Applications After Upgrade

User application classes, and routines containing embedded SQL statements must be recompiled after upgrading to this version of Caché as noted in the Administrator section of this document.

CACHESYS and CACHELIB Reorganized

Any robust security implementation shares a number of characteristics with other like systems. For example:

  • The number of functions implementing the security module should be kept as small as necessary.

  • These should be collected together and isolated from other system functions so they can be protected.

  • They must be independently verified for correct operation and benign failure.

As part of the effort to add increase security in Caché, InterSystems has reviewed the low-level routines present in the CACHESYS and CACHELIB databases in light of these requirements. The result is that the contents of these database have been reorganized. CACHESYS (the manager's database) now contains only the low-level routines necessary for system management. Everything else has been moved to CACHELIB.

The following are brief guidelines to the changes.

For System classes:

  • Methods in classes of the %SYS package:

    • These reside in the manager database (CACHESYS) because they invoked protected system routines.

    • Since their names start with “%”, they are mapped into all other namespaces.

  • Method of classes in the %System package:

    • These reside in the CACHELIB database which is mounted read-only by default. They do not invoke protected functionality, but reside there to support legacy applications.

    • Since their names start with “%”, they are mapped into all other namespaces.

  • Methods in the Sys and System packages reside where the %Sys and %System packages reside, respectively. However, because their names do not start with “%” they are visible only within those databases.

For the system functions whose name is of the form, $SYSTEM.<name>:

  • If the name is one associated with a method in an internally known system class, it invokes that method.

  • Otherwise, it attempts to invoke the %System method by that <name>. So $SYSTEM.SomeClass.That Method() is equivalent to ##class(%System.SomeClass).ThatMethod().

And finally, for globals:

  • All globals whose names start with “%q” are mapped to CACHELIB (default read-only).

  • All other globals map to CACHESYS (default read-write).

The mappings can be displayed in more detail using ^%SYS.GXLINFO.

CACHELIB Is Mounted As Read-Only

As part of the reorganization of CACHESYS and CACHELIB, all of the information that is modifiable during normal operation has been collected into CACHESYS. Therefore, CACHELIB is mounted as a read-only database by default.

Access To %-Globals Is More Restrictive

By default, routines do not have write permission on %-globals that reside in other databases. In version 5.1, these rules are now consistently enforced. This can be changed via the System Management Portal at Home,Security Management,System Security Settings by changing the setting for “Enable writing to %-globals” to “Yes”.

Permissions On CNLS

The CNLS application is used to change the locale of a Caché installation. Running it now requires %Admin_Manage:U.

Authenticated Namespace And Routine Override Command Line

In prior versions, when a namespace and routine were supplied as a parameter on a command line, the process created would always use that namespace, and would override any namespace or routine specified by the security mechanisms of that version.

In version 5.1, if Caché is installed with the MINIMAL setting, csession will work as before. If the user needs to be authenticated, the namespace and routine for that user will override any namespace or routine setting supplied on the command line.

Changes To Routines

Routines And Globals Moved

In Caché 5.1, all %-routines and %-globals were reorganized as noted above.

Note:

If there are user- or site-supplied routines whose names begin with “%”, they must obey these same rules. These changes require administrative privilege because, by default, the CACHELIB database is set read-only at installation time and cannot be altered.

Unless routines added at the site need to create globals in CACHELIB during normal operation, InterSystems recommends that, after installing these routines, CACHELIB be made read-only once again.

Routines Renamed By Removing “%”

The review of Caché system functions resulted in a number of routines being designated as system management functions whose use needed to be controlled. Therefore, the following routines have been renamed by removing the “%” from their name, thus placing them within the protection of the manager's database:

  • BUTTONS

  • COLLATE

  • DMREPAIR, DSET

  • LANG*

  • MATH

  • NLS, NLSCOMP, NLSLOAD, NOREP

  • ROLLBACK*

  • SS, SSVNJOB, SSVNLOCK, SSVNROUTINE, ST

  • UPDATECLASS

  • Wcomm, Wpfiles, Wr, Wr1, Wsback, Wsdb, Wsdba, Wsmkey, Wsnls, Wsnls2

Important:

This change means that these routines must be invoked from a non-edited routine in the CACHESYS database and not be part of any indirection; or else be in a process that holds WRITE permission on the CACHESYS database.

Routines Renamed

To further emphasize their relationship to system function, some routines were renamed:

Previous Name
New Name
%DM
%SYS.DATABASE
%FILE
%SYS.FILE
%GCREATE
%SYS.GCREATE
%GD
%SYS.GD
%GIFMSM
%SYS.GIFMSM
%GLO
%SYS.GLO
%GOF1
%SYS.GOF1
%GSETNS
%SYS.GSETNS
%GSET
%SYS.GSET
%GXLINF1
%SYS.GXLINF1
%GXLINFO
%SYS.GXLINFO
%LICENSE
%SYS.LICENSE
%MACHINE
%SYS.MACHINE
%MONLBL
%SYS.MONLBL
%NOJRN
%SYS.NOJRN
%PMODE
%SYS.PMODE
%RI2
%SYS.RI2
%RIMF
%SYS.RIMF
%RI
%SYS.RI
%SYSCONV
%SYS.SYSCONV
%Wcdu
%SYS.Wcdu
%Wgr
%SYS.Wgr
%Wgs
%SYS.Wgs
%Wsys
%SYS.Wsys
Routines Eliminated

During the review, some routines were identified as duplicating functionality provides elsewhere. These were removed:

  • %CLI — The same functionality is available from Caché through $zf(-1). On UNIX®, OpenVMS, and Mac, command line interpretation is done via !<command>. On Windows systems, the DOS command START performs this function.

  • %DKIOERROR — Calls to it should be replaced with $$$ERROR or $SYSTEM.Error usage.

  • %GED — Use %GCHANGE and %Library.Global methods instead.

  • %GDOLD — This routine has been removed.

  • %GROWTH — The functions of this routine have been moved to the SYS.Database class.

  • %GTARGET — This routine has been removed.

  • %LM — The functions of this routine have been included in the SYS.Lock class.

  • %LMFCLI — The functions of this routine have been included in the $SYSTEM.License class.

  • %qserver — The user accessible entrypoints have been moved into $SYSTEM.SQL.

  • %RMAC — This routine has been removed.

  • %START — This routine has been removed.

  • %USER — This routine has been replaced by $USERNAME.

  • %UTIL — This is an internal routine which has been removed. Its message logging function has been converted into a system macro, LOGMSG.

Stub Routines Added

Some frequently-invoked routines were moved to CACHESYS (%DM, %LICENSE, %GD, and %SS) and were renamed. Stub routines that call the new routines were left in their place as a compatibility aid. Applications are encouraged to move to using the new names.

In adding the stub routines, the tag, SYS, has been removed from the %SYS routine.

Routines Deleted

In addition to the changes noted above, internal and obsolete routines were removed from these libraries. If you suspect that this may be affecting your application, please contact the InterSystems Worldwide Response Center (WRC) for assistance.

No Mapping For %LANG Routines

Caché 5.1 ignores any routine mappings for the %LANG* routines that are used to provide language extensions. The routines executed will always be those in he %SYS namespace (CACHELIB).

Class Changes

During the development of version 5.1, a number of changes were made to improve development accuracy and reduce ambiguity when using classes. They are collected in this section

Classes Replaced

The following classes have been removed from the system because they have been replaced with classes providing better functionality. They are listed below:

Class
Replacement
None. This is internal class, used only by other InterSystems-developed classes, has been superceded by the CSP login mechanism.
None. This is an internal class used only by other InterSystems-developed classes.
None. This is an internal class used only by other InterSystems-developed classes.
This class has been replaced by several classes providing specific capabilities for differing circumstances, for example, %Library.BinaryStream, %Library.CacheStream, %Library.CacheStreamLegacy, %Library.CharacterStream, %Library.FileBinaryStream, %Library.FileCharacterStream, %Library.SerialStream, %Library.Stream.
See %Library.TimeStamp for replacement functionality.
The functions of this class have been replaced by those of SYS.Database.
The functions of this class have been replaced by those of Config.ECP and SYS.ECP.
None. This is an internal class used only by other InterSystems-developed classes.
The Monitor facilities were completely redesigned for version 5.2. Applications which formerly used the %SYSTEM.Monitor classes will have to be modified, perhaps extensively. Please refer to the %Monitor and %MonitorTools pacvkages and to the Config.Monitor and Config.MonitorTools classes.
New Classes

The following classes are new in this version of Caché:

Programmers who relied on an unqualified class name resolving to the correct location may discover that the new classes added to %Library now cause naming conflicts if the application defined classes with any of these names.

Name Case Conflict In Class Compilation

In version 5.0, Caché allows a class to define a method which has the same spelling as a method of its superclass, but differs in case. For example, a method called %save(), defined in class that inherits from %Library.Persistent would be considered a different method from the %Save() method of the superclass.

In version 5.1, this situation produces a compilation error. For example, CSP applications that had defined methods of include() or link() will find that these are now in conflict with %CSP.Page.Include() and %CSP.Page.Link() respectively.

Ambiguity Resolution When Simple Class Names Used

If an application makes a reference to a class whose name begins with a percent-sign and which does not specify its package name, the class compiler looks for the class in the %Library package for the class. Thus,

Set BaseDir = ##CLASS(%File).ManagerDirectory()

is interpreted as if it had been written

Set BaseDir = ##CLASS(%Library.File).ManagerDirectory()

Programmers who relied on the unqualified class name resolving to the correct location will discover that the new classes added to %Library may now cause ambiguity in naming if the application defined classes with the same name, for example %Utility.

Class Member Naming Checks Made More Strict

In Caché version 4.1, you were not allowed to define two class members with the same name but different case. In version 5.0, however, a bug was introduced that failed to report these errors.

In Caché version 5.1, this bug is fixed but in order to allow these classes that did previously compile on 5.0 to still compile application developers can disable this check by setting the 'Strict Checking' flag is set to off. This is done by executing the command:

Set ^%qCacheObjectSys("strictchecking") = 0
Note:

In order to set this flag, you will have to change the permission on CACHELIB from Read-Only to Read/Write. This is done using the Management Portal, Home, Configuration, Local Databases.

New Methods Added To %Library.Persistent

Several new methods are implemented in %Persistent and are inherited by all persistent classes. These methods are:

  • %LockId: acquires a lock on an instance of a class

  • %UnlockId: releases a previously acquired instance lock

  • %LockExtent: acquires a lock on every instance in an extent

  • %UnlockExtent: releases a lock ion extent

  • %GetLock: attempts to get a lock on an instance, but will escalate the lock to the extent of which it is a member, if necessary

More detail on these methods can be found in the class documentation for %Library.Persistent.

Note:

Changes to the underlying storage mechanisms that affect persistent objects are detailed here.

Conflicts With User-Written %-Methods

User applications with method names that start with “%” should check to make sure that there are no conflicts with methods supplied by InterSystems in “base” classes such as %LIbrary.Persistent. This version of Caché has significantly increased the number of such names.

IdKeys Now Have <Indexname>Exists and <Indexname>Open Methods

This version of Caché now supplies <indexname>Exists and <indexname>Open methods for IdKeys.

All persistent class have an IdKey. If one is not explicitly defined or inherited from a superclass, then an index named IdKey<n> will be generated where <n> is an integer that is appended to the root of "IdKey", if another index named IdKey already exists. This index is defined as a system generated index.

In prior versions, the index was generated during class compilation. No inheritance resolution was applied to the generated index, and no index methods were generated.

IsValidDT Changes
  • Method No Longer Generated By Default

    In version 5.1, user-written datatype classes that extend the%Library.DataType class no longer have the IsValidDT() method automatically generated. The previous behavior can be restored by executing

    Set ^SYS("ObjectCompiler","GenerateIsValidDT") = 1
    
    Note:

    Each namespace where the older behavior is and then recompile all affected routines.

  • Method Return Type Changed

    In previous versions, the class instance method, %IsValidDT(), returned a value of type %Integer. In Caché 5.1, it more correctly returns a %Boolean.

Methods Supporting SQLCOMPUTECODE

Caché allows classes to define SQL computed properties by declaring them with the attribute, SQLCOMPUTED, and providing SQL code to computed the desired value. The value can be transient, calculated or storable.

For computed properties a <property>Get() method is generated that invokes <property>Compute() as needed. SQLCOMPUTECODE allows for other values to be referenced during computation. These references are to SQL columns (preserved for backward compatibility) and are converted to property names during method generation.

If the SQL column references a column projected from an embedded class, then <property>Compute() will generate an extended reference to the embedded property.

Note:

Using embedded properties in SQLCOMPUTE code breaks encapsulation. One problem with breaking encapsulation is with "triggered" computed fields, that is, when SQLCOMPUTEONCHANGE is declared. Embedded property references are not supported in SQLCOMPUTEONCHANGE.

Changes To Inheritance Processing

In previous versions, the inheritance rules for classes were not always as expected. For example, if a user created a class, User.MyClass, that was a subclass of %Library.Persistent, Caché would automatically inherit the default package name of the superclass, %Library, as a #import into User.MyClass. A consequence of this is that if User.MyClass contained a property declared such as

property A as String;

Caché would try resolve this by looking in both the User and %Library packages. If User had a class, User.String, Caché would report a classname conflict even though the user probably intended to reference User.String. The workaround was to fully qualify the property name as in

property A as User.String;

Caché 5.1 will still inherit any explicit #import settings from the superclasses, but it will not automatically add the superclass package names to the #import. So in the example given the A property would resolve to 'User.String' without any name conflict errors.

Caché still uses the current class packagename in resolving names; User.MyClass will still use 'User' as a #import for its name. But this is no longer true for subclasses.

More explicitly, Caché will always resolve the name in the context where it was first defined and not the current classname. For example, suppose User.MyClass defines a method, X(). If a class MyPackage.MyClass inherts from User.MyClass, when it is compiled Caché will compile the inherited X() method in MyPackage.MyClass but resolve any unqualified classnames used in this method in the context of User.MyClass because this is where X() was defined.

Stream Implementation Has Been Modified

In version 5.1, cloning a class containing a stream member works differently from earlier releases. What happens now is:

  • If the stream member is a serial stream, the oref of the stream is copied to the clone.

  • If the stream is an instance of the “older” stream implementations:

    • %Library.FileBinaryStream

    • %Library.FileCharacterStream

    • %Library.GlobalBinaryStream

    • %Library.GlobalCharacterStream

    the oref of the stream is copied to the clone.

  • In all other cases, Caché will make a copy of the stream contents in a new stream of the same type and place the oref of the new stream into the clone.

If an application wishes to retain the oref of the original stream, it can do so with

Set ..ThatStream = oref.%ConstructClone(0)
Copy code to clipboard
XML Export Replaces CDL

In this version, CDL is no longer available as an export format for classes. Users should export their classes in XML instead. CDL will still be accepted as an import format for this release.

Persistent Superclasses Must Reside Outside of CACHELIB

Subclasses of persistent classes currently store some of their extent information with the extent information of their superclass. Because CACHELIB in Caché version 5.1 is now a read-only database, it is no longer possible to subclass persistent classes residing in CACHELIB by default. Attempting to do so will result in a <PROTECT> error. This is true even if the persistent classes were created locally and stored in CACHELIB.

The only exception to this is classes which are marked as SERIAL. They do not have extent information since their instances are embedded in the class that references them.

TRUNCATE Default Changed For %Library.String

Strings have, among their other parameters, settings for MAXLEN and TRUNCATE. The value of MAXLEN specifies the maximum permissible length of the string. The value of TRUNCATE specifies how to enforce the maximum length limit.

  • If TRUNCATE is set to true, Caché will store only the first MAXLEN characters in a variable declared as type %Library.String ignoring the rest of the string.

  • If TRUNCATE is set to false, an attempt to assign more than MAXLEN characters to the variable will return an error status.

In Caché version 5.1, the default value of TRUNCATE for new instances of %Library.String will be false. In previous versions it had been true. Note that this applies only to new strings created in version 5.1. Older items of type string will still have the defaults from the time they were created.

Support For Legacy %Close() Behavior Dropped

In version 5.0, Caché changed how it handled objects that were closed. The object was destroyed upon %Close if its reference count went to 0. The OREF associated with the object would be removed once it was marked “inactive”; that is, all references to it were gone.

When this behavior was introduced, it was possible to have Caché use “legacy support” for %Close instead — the method used in versions prior to 5.0 — via the call

 Do $ZU(68,56,1)
Copy code to clipboard

In this mode, Caché decrements an object's object-level reference count upon %Close() and removes it from memory when the count reaches 0. No provision was made to prevent re-use of the OREF.

In Cache 5.1, legacy mode has been removed. Calling this function will result in a <FUNCTION> error.

%DeleteExtent() Behavior Improved

In prior versions, the %DeleteExtent() method always returned $$$OK, even if not all instances in the extent were deleted. In version 5.1, its behavior now better matches expectations; it only returns $$$OK if all instances of the extent were successfully deleted.

Method Compilation And Return Values

In previous versions, if a method was declared to return a value, the method compiler would insert a

Quit ""

if the last line of the method did not begin with a Quit command. This approach, however, hid subtle programming bugs because the method the developer wrote did not, in fact, return a value when it was supposed to.

In version 5.1, this is no longer done. The method compiler will only insert a simple

Quit

instead if the last line of the method does not contain one. Thus, invoking a method (function) that does not return a value when it is declared to will result in a <COMMAND> error.

Required Relationship Collections Cannot Be Empty

If an application specifies that a “child” or “many” side of a relationship collection is required, Caché now make sure this contains at least one element. If the relationship is empty at the time the instance is saved, Caché reports an error on %Save, for example:

ERROR #5662: Relationship child/many property 'Sample.Company::Employees
(1@Sample.Company,ID=)' is required so must have at least one member
Cycle Checking For XML Exports

In Caché 5.1 XML-enabled classes check their hierarchy before export to determine if there is a cycle present. This check is on by default, but may be disabled by appending “,nocyclecheck” to the Format property of %XML.Writer.

Note:

If this check is disabled, and a cycle is present, a <FRAMESTACK> error will result.

Task Changes
  • Task Manager Hierarchy Upgraded

    The Task Manager now uses the Caché class hierarchy more completely. All user tasks must now subclass the class, %SYS.Task.Definition.

    Subclasses can thereby introduce additional properties which will be available during the task execution. The user interface will then interrogate the definition to request the property values from the user. For example,

    Class %SYS.Task.IntegrityCheck Extends %SYS.Task.Definition
    {
    Property Directory As %String [ InitialExpression = {$zu(12)} ];
    Property Filename As %String [ InitialExpression = "INTEGRIT.LOG" ];
    
    ClassMethod DirectoryIsValid(Directory As %String) As %Status
    {
      If '##class(%Library.File).DirectoryExists(Directory) 
      {
        Quit $$$ERROR($$$GeneralError,"Directory does not exist")
      }
      Quit $$$OK
    }
    
    /// This method is responsible for executing the task
    /// At the scheduled time, the Task Manager
    /// - creates an instance of this object,
    /// - sets any property values using the stored "settings" for the task,
    /// - and invokes this method to execute the task.
    /// In order to execute a real task, override this method in a subclass.
    Method OnTask() As %Status
    {
      Do Silent^Integrity(..Directory_..Filename)
      Quit $$$OK
    }
    
    }
    
    
  • ContinueAfterError Property Removed

    The ContinueAfterError property in the class %SYS.TaskSuper has been removed because it is too general. Tasks which depended on it must be redesigned to handle their own error conditions.

  • Other Task Improvements

    In addition, the following improvements have been made to task management in Caché 5.1:

    • There is a RunLegacyTask class that provides the ExecuteCode() method for compatibility with earlier versions.

    • If a running task encounters any kind of error, it is suspended.

    • The methods, RunOnce() and RunNow(), will now make a suspended task active.

    • Task startup is prevented if there is no license found for this instance of Caché.

CDL Support Dropped In Following Releases In Favor Of XML

In Caché 5.1, CDL was removed as an option for exporting classes (see XML Export Replaces CDL) in favor of the industry-standard XML. Caché 2008.1 will complete this transition. CDL will no longer be available as a format for import to Caché, either as import or output.

Furthermore, for new platforms added in 2007.1 that are not newer versions of existing platforms, InterSystems may decline to provide support for CDL at all. For the exact details on each Caché platform, please refer to the Supported Platforms documentation.

For those customers that have program archives in CDL format, InterSystems recommends importing them into Caché 2007.1, and exporting them as XML.

SQL Differences

In the transition to version 5.1, the following changes were made in SQL that may affect existing programs.

Caché And SQL Users Unified

In prior versions, the list of valid Caché user names and the list of valid SQL user names were unrelated and were governed by different security mechanisms. In version 5.1, this is no longer true. All SQL user names are Caché user names, and vice versa. What each is permitted to do is determined by the same security mechanism.

Atomic SQL statements

In Version 5.1, the SQL statements DELETE, UPDATE, and INSERT...SELECT have been made atomic. That is, the statement either completes successfully or no rows in the table are modified.

In previous versions, it was the responsibility of the application to detect an incomplete operation and roll back the update (if desired). Now, if any row fails to update, none of the rows in the table will be updated by the statement.

SQL Passwords Are Case-Sensitive

In Version 5.1, for security reasons, SQL uses the same password mechanisms as Caché. One consequence of this is that SQL passwords are now case-sensitive. Previously, they were not.

Table Ownership Interaction With $USERNAME

This means that tables created through the use of DDL will have as owner the value of $USERNAME at the time they were created. When creating a class or table by any other means, the class's OWNER keyword is not defined unless the developer explicitly defines it. When a class is compiled that projects a table and the class's OWNER keyword is NULL, the table's owner is set to _SYSTEM.

This interpretation is the same as in previous versions. What has changed is that there is no default to an OWNER of _SYSTEM when creating tables through DDL in 5.1.

Delimited Identifiers Are The Default

Caché version 5.1 installs with the value for “Support Delimited Identifiers” as true. This means that a double-quoted string (“My String”) is considered a delimited identifier within an SQL statement. Prior versions of Caché had this parameter set to false: a double-quoted string was treated as a string constant or literal string. The value of this parameter can be changed via the System Management Portal at Home,Configuration,Advanced Settings.

%msql Eliminated

This variable was used in ObjectScript to specify a valid user name for SQL access from embedded SQL. A valid user name was one that was registered in the User Table.

In Caché 5.1, the SQL username is now extracted from the $USERNAME special variable which is set when the user is authenticated.

Cached Query Changes
  • Cached Query Interaction With Read-Only Databases

    In Caché 5.1, SQL queries that require Cached Queries will not work against read-only databases unless the ^mcq global is mapped to a database mounted as read-write. An example of this interaction is attempting to create a table in a read-only database using SQL DDL.

  • Cached Query Changes Are Not Journaled

    In version 5.1, when cached queries are modified, the changes are no longer journaled. This prevents changes to cached queries inside a transaction from being written to the journal. Thus, shadowing will not apply cached query changes across systems.

  • Purging Cached Queries Is Immediate

    Caché 5.1 no longer supports the concept of purging cached queries after N days, where N is a number of days defined in the configuration setting. When an application calls Purge(), it will purge all cached queries.

  • Cached Queries On Read-Only Databases

    This version of Caché permits applications to Prepare and Execute Dynamic SQL cached queries that do SELECTs against the table of the database.

    Note:

    Any attempt by the application to purge such queries will result in a <PROTECT> error. Purging cached queries requires write access to the database they are associated with.

  • Cached Query Purge Does Not Propagate To Other Systems

    If a package is mapped to multiple namespaces within a single system, a compile/delete of a class in that package that has cached queries created by it will purge all the cached queries that use that class in each of the namespaces However, if the package mappings are to different machines via ECP and the class is recompiled/deleted, the purge of the cached queries will only occur on the system where the class is compiled. Cached queries form this class on other networked systems must be manually purged.

  • ^mcq(“init code”) Execution Order Changed

    The global node, ^mcq(“init code”), can be set to contain Caché commands to be executed when a connection is made via JDBC or ODBC. In Caché 5.0.12, the order of events during the connection was:

    1. Read the connection info from the client

    2. Run the code in ^mcq(“init code”)

    3. Run the login code

    In Caché 5.0.13, the sequence was changed to:

    1. Run the code in ^mcq(“init code”)

    2. Run the login code, including reading the connection info from the client

    In Caché 5.1, this sequence is now

    1. Run the login code, including reading the connection info from the client

    2. Run the code in ^mcq(“init code”)

Ambiguous Names In SQL Queries
  • In FROM Clause

    In previous versions of Caché, ambiguous field names in SQL queries were assumed to be associated with the earliest table mentioned in the FROM clause. This same situation in version 5.1 reports an error. The ambiguous names must be qualified to show their origin.

  • In ORDER BY Clause

    Caché 5.1 reports an error if the field names in an ORDER BY clause are ambiguous, for example:

    SELECT TOP 20
           %ID as ID,
           ((ID * 2) # 20) as ID
    FROM Sample.Company
    ORDER BY ID 
    
    

    This corrects a previously undetected bug in the query processor. Previous versions of Caché would associated the ambiguous name (ID in this case) with the last occurrence of that column name.

Privileges Required To Set Certain Options

You must have %Admin_Security:Use permission to execute the following SQL SET OPTION statements:

SET OPTION SUPPORT_DELIMITED_IDENTIFIERS = {TRUE | FALSE}

SET OPTION PKEY_IS_IDKEY = {TRUE | FALSE}

If you do not, the attempt to execute the statement will return an SQLCODE value of —99; this is a Privilege Violation error value. The reason is that these modify Caché configuration settings and you must be privileged to change them.

Changes To SQL GRANT And REVOKE

The SQL GRANT and REVOKE commands no longer support the following general administrative privileges:

  • %GRANT_ANY_PRIVILEGE

  • %CREATE_USER

  • %ALTER_USER

  • %DROP_USER

  • %CREATE_ROLE

  • %GRANT_ANY_ROLE

  • %DROP_ANY_ROLE

In previous versions, SQL permissions were separately maintained. In version 5.1, these privileges are managed by Caché. SQL code which attempts one of these operations will be interpreted as granting a role having this name.

  • CREATE ROLE Details

    In order to create a role definition through the SQL CREATE ROLE statement, the user must hold the %Admin_Security:Use privilege. If this privilege is not held by the user, an error -99 will be returned with an appropriate message.

  • DROP ROLE Details

    In order to drop a role definition through DROP ROLE, at least one of the following must be true:

    • The user has the %Admin_Security:Use privilege.

    • The user is the owner of the role.

    • The user was granted the role with the admin option.

SQL %THRESHOLD Removed

The SQL %THRESHOLD feature is no longer used in this version of Caché. Code that attempts to grant a threshold, for example,

GRANT %THRESHOLD ### TO SomeUser
Copy code to clipboard

will now receive an error at compile time. And code such as

REVOKE %THRESHOLD FROM SomeUser
Copy code to clipboard

will no longer revoke the threshold. The interpretation has changed; Caché 5.1 will attempt to revoke a role called %THRESHOLD from the user.

SQL Privileges On SAMPLES Granted To User _PUBLIC

At installation time, all SQL privileges for all tables, views, and procedures in the SAMPLES namespace are granted to the user named, _PUBLIC.

SQL Catalog Info For System Tables

The SQLTables() query of the %Library.SQLCatalog class returns a list of tables and views defined in the current namespace. In earlier versions, this list included System tables. In version 5.1, the System table information will only be included if the query is executed while in the %SYS namespace.

Collated Fields May Return Results In Different Order

Due to optimizations made in Caché SQL for 5.1, the results returned by queries on collated fields may be different. For example, consider

SELECT Dept, AVG(Salary)
FROM Personnel
GROUP BY Dept

where Dept is collated according to %SQLUPPER where the values entered in various rows are indiscriminate about case — some are uppercase, some lowercase, some with capital letters beginning each word, and so on. However, because of the GROUP BY clause, all departments are to be collected according to their value when converted to uppercase.

In prior versions of Caché, when this query's results were returned, the value of Dept returned was the actual value of one of the selected rows. In Cache 5.1, the value returned for Dept will always be represented in its collated form, in this case, %SQLUPPER.

This means that two queries such as

SELECT IdNum, Dept
FROM Personnel

and

SELECT Dept, COUNT(IdNum)
FROM Personnel
GROUP BY Dept

may not return the expected results. The first will return the actual values stored in the Dept column and the second will return those values converted to uppercase. This may not be what is desired by the application.

The prior behavior can be restored via the %exact qualification for Dept as in

SELECT %exact Dept, COUNT(IdNum)
FROM Personnel
GROUP BY Dept
Control Of Time Precision

There is a new SQL configuration setting which allows the specification of the precision of the time value returned by the GETDATE(), CURRENT_TIME(), and CURRENT_TIMESTAMP() SQL scalar functions. The default time precision can be set using the new API call:

PreviousValue = $SYSTEM.SQL.SetDefaultTimePrecision(value)

where value is the precision (the number of decimal places for the millisecond portion of the time value).

The default is 0; milliseconds are not returned in the values returned by these functions. The function returns the previous (or default) time precision setting. For example: After executing

Do $SYSTEM.SQL.SetDefaultTimePrecision(3)

GETDATE() will return a value in the format: 'YYYY-MM-DD HH:MM:SS.sss'. An application can still override this default by passing a specific time precision value to GETDATE(). For example: GETDATE(5) returns: 'YYYY-MM-DD HH:MM:SS.sssss'.

Note:

This setting is used during the code-generation phase of the SQL engine. If you change the default time precision setting, you must purge any cached queries and recompile any class queries, embedded SQL routines, etc. for the new setting to take affect for that SQL statement.

Caution:

While CURRENT_TIME() will return the time with a precision as specified in the default time precision setting, the LogicalToOdbc conversion of this time value does not support milliseconds. So if you have a default precision defined and CURRENT_TIME() is returned in a query through ODBC or JDBC, the milliseconds will be dropped from the value.

Owner Checked On DDL Create And Drop

When a users executes DDL to create or drop a procedure, query, or method in an existing class, Caché will not allow the action if the class has an OWNER defined, and the user is not the OWNER of the class.

ODBC & JDBC Permission Checking

Caché now checks the EXECUTE privilege for Stored Procedures invoked through ODBC and JDBC. A user may not call the procedure through ODBC or JDBC if the user has not been granted EXECUTE privilege on the procedure.

When looking at the list of procedures in the System Management Portal or from an ODBC or JDBC catalog query, the user will only see procedures that the user has privilege to call.

When creating a procedure through DDL, the creator user name is set as the default owner of the procedure. (This may be changed later by editing the class definition.) When the procedure is compiled, the owner of the procedure is granted EXECUTE privilege WITH GRANT OPTION if the owner does not have the %All role. If there is no owner specified in the class definition that projects the procedure, the owner is considered to be the user compiling the class.

When a procedure is dropped, or the class that contains the procedure definition is deleted, any execute privileges that had been granted on the procedure are dropped.

^mdd Information Moved to ^oddEXTR

The ^mdd global has been removed. In prior versions, this held SQL–related information. The information has been incorporated into the ^oddEXTR structures.

You can mount an earlier database in Caché 5.1, but to use it you must upgrade it with the commands

Do $SYSTEM.OBJ.UpgradeAll()
Do $SYSTEM.OBJ.CompileAll()

After you run UpgradeAll and CompileAll, you cannot use the database in anything earlier than Cache 5.1.

Comparisons Involving NULL

This release of Caché corrects previous improper behavior in some SQL predicates involving constants and host variables whose values were NULL.

Application relying on the previous incorrect behavior of NULL testing for constants and host variables might have to be modified. This affects predicates of the form

field <> parameter
field > parameter
field >= parameter

where the value of “parameter” may be set to the NULL value. (Predicates involving the comparison operators “<”, “<=”, and “=” behaved correctly.) This means that existing predicates such as

field <> :hostVar

will have different behavior if :hostVar is bound to "" in COS. According to SQL three-state logic, this predicate should evaluate to NULL and fail rather than treating NULL as a value and succeeding for every field value other than NULL.

The previous behavior of a specific query could be restored, if necessary, by adding specific tests for NULL. AN existing query such as:

 field<>:hostVar

needs to be rewritten as

(field <> :hostVar OR (:hostVar IS NULL AND field IS NOT NULL))

to produce the same results as before.

System Error Code Changes

New Error Codes

This version of Caché adds news system error codes:

New System Error Messages
Error Code Description
<ALARM> An internal timer for user events has expired.
<COLLATECHANGE> There was an attempt to change the collation algorithm while subscripted local variables are defined.
<DDP JOB OVERFLOW> A Cache job's internal job number is greater than 1544 and it is attempting to access a DSM database using DDP. The job number is too large for DDP to handle.
<DSKFUL> An attempt to write data to a disk file failed because the file reached its maximum size; some of the data was written but not all.
<EXTERNAL INTERRUPT> Another process has attempted to interrupt this process.
<LICENSE ALLOCATION EXCEEDED> This configuration has exceeded the number of license units it has been allocated from the pool of total units available.
<NETWORK UNLICENSED> The application has attempted to access a remote directory, but there is no license for Caché networking.
<RESJOB> An attempt was made to terminate a reserved job.
<TRANSACTION LEVEL> The application has too many nested transactions pending.
Error Codes Removed

This release of Caché no longer supports the system error, <DISCONNECT>.

Globals Reorganized

Caché version 5.1 has reordered the subscripts in the globals that store user and system messages: ^CacheMsg and ^%qCacheMsg. The new order is domain, language, and id which allows subscript mapping of the message globals by domain.

The class dictionary version number is upgraded to 20 which will result in the user being asked to run $SYSTEM.OBJ.Upgrade() which will reorder the subscripts of existing ^CacheMsg globals. All error macros, routines and methods will keep the original arguments in the same order. therefore, no change in user code will be needed unless an application directly addresses the message global.

ObjectScript Changes

New System Variables
  • $USERNAME

    In version 5.1, $USERNAME contains the name by which a user is known “inside” Caché for security purposes.

    For example, suppose a user can successfully login to a Windows XP system with the username, “Smith”. If that user then attempts to access the Caché online documentation via the Cube, he or she is assigned the name, “UnknownUser”, for security purposes. If UnknownUser has no access to the online documentation, the user may be asked (depending on how Caché security is configured) to authenticate himself by supplying a userid and password known to Caché.

    Caché 5.1 also retains the routine, ^%USER that prints the name of the user running the current process as it is known to the operating system.

    Note:

    ^%USER and $USERNAME are not required to be identical. The former results from the operating system login. The latter is based on a user successfully authentication to Caché security.

    For example, suppose a user logs on Windows XP as user "Smith". However, if that user selects Documentation from the Caché Cube, the DocBook CSP application starts with a $USERNAME of "UnknownUser".

  • $ROLES

    This variable contains a comma-separated list of all the roles held by the current user at any point during execution.

ObjectScript Compiler Upgrades

The ObjectScript compiler has been improved in version 5.1. As a result, it now generates code that cannot be run on previous releases. An attempt to do so will result in a <RECOMPILE> error.

The converse is not true. Compiled code from Cache 5.0 systems will run unchanged on version 5.1.

The following table gives the relationship between a version of Caché and a version of the ObjectScript compiler. The version number is made up of a “major” number and a “minor” number separated by a decimal point. The major and minor version of the ObjectScript compiler are returned by the ObjectScript functions $ZUTIL(40,0,68) and $ZUTIL(40,0,69), respectively.

Caché Version
Compiler Version
3.2
9.6
4.0
9.7
4.1
9.7
5.0
10.0
5.1
10.1

A routine compiled on a version of Caché can be run on another version of Caché without re-compilation if

  1. the major version of the compiler for each Caché release is the same, and

  2. the compiler version of the system on which the routine will be run is greater than or equal to the compiler version of the system where the routine was compiled.

Note:

The Caché Basic compiler uses the same version number as the ObjectScript compiler and is subject to the same compatibility rules.

Caution:

This change means that ECP configurations are limited to having their servers on version 5.1 with clients on either version 5.0 or 5.1. ECP servers running Caché 5.0 cannot serve code compiled under version 5.1.

Permission Requirements For Some ObjectScript Commands

Because of their effect, some commands under certain circumstances now require the user to have specific permissions for them to succeed.

Command
Permission
Set
When applied to the special variable $ROLES, this is a privileged operation. Unless the application has the privilege to change $ROLES, the value will not be altered.
View
%DB_<XXX>:R is required to read blocks from the database; %DB_<XXX>:W is required to modify the database.
$ZTRAP Change

The reference material for $ZTRAP states that when the name of the error trap starts with an asterisk (*), it indicates that Caché should invoke the error handler at the context level where the error occurred. However, if the error trap is within the context of a procedure, then Caché cannot simultaneously establish the error context and the proper context for the local variables of the procedure. In Caché 5.1, the compiler has been changed to detect this usage and report it as an error at compile time.

Applications that wish to use this feature must ensure that either

  • the subroutine containing the error recovery code is not a procedure, or

  • the error handling logic is altered so it does not need to be run in the context of the error.

$ZERROR Contains Additional Information For Some Errors

In the event an error occurs, information about it is stored in the system variable, $ZERROR. In Caché 5.1, the string stored in $ZERROR includes more information than in previous versions.

For example, when a routine attempts to use a variable that has not been defined, $ZERROR now includes the name of the undefined variable. Whereas in previous versions of Caché the value of $ZERROR might look like this:

<UNDEFINED>zMethodName^Pkg.Class.1

in version 5.1, it looks generically like this:

<ERRCODE>Tag^Routine+line *someinfo

A consequence of this change is that error handling routines that made assumptions about the format of the string in $ZERROR may now require redesign to work as before. For example, the following will no longer work in version 5.1:

Write "Error line: ", $PIECE($ZERROR, ">", 2)

and should be changed to be something like

Write "Error line: ", $PIECE($PIECE($ZERROR, ">", 2), " ", 1)

The following table gives a list of errors that include additional info and the format of that information. The new info is separated from the previous text by a space.

Error Code Description
<UNDEFINED>
the name of the variable (including any subscripts used)
<SUBSCRIPT>
the subscript reference in error
<CLASS DOES NOT EXIST>
the referenced class name
<PROPERTY DOES NOT EXIST>
the name of the referenced property and the class name it is supposed to be in, separated by a comma
<METHOD DOES NOT EXIST>
the name of the method invoked and the class name assumed to contain it, separated by a comma
<PROTECT>
the name of the global referenced and the name of the directory containing it, separated by a comma
<NOROUTINE>
the name of the routine being invoked

The names of variables local to routines (or methods) as well as the names of class properties and methods are indicated with an asterisk preceding the name. Global variable names are prefixed with a caret as expected.

Examples:

<UNDEFINED> *x
<UNDEFINED> *abc(2)
<UNDEFINED> ^xyz(2,"abc")
<PROPERTY DOES NOT EXIST> *SomeProp,Package.Classname
<METHOD DOES NOT EXIST> *AMethod,SamePackage.DifferentClass
<PROTECT> ^%GlobalVar,c:\cx\mgr\notyours\

$ZUTIL
  • Permission Changes

    The authority needed to execute certain $ZUTIL functions is more specific in version 5.1 than prior releases. Unless otherwise noted in the following table, published $ZUTIL options require no special permission.

    Function Number
    Requirement
    Explanation
    5 %DB_<XXX>:R
    Subject: change to another namespace
    In version 5.1, changing to a different namespace requires that the user have %DB_XXX:Read where XXX is the name of the database that contains the globals for the namespace.
    58 N/A
    Subject: set the privilege level required to use the XECUTE command.
    This function has been removed in Caché 5.1.
    69 %Manager
    Subject: set system-wide defaults
    This function must be invoked from a non-edited routine in the CACHESYS database and not part of any indirection; or else be in a job that holds the WRITE permission on the CACHESYS database.
    78 %Manager
    Subject: search journal file for open transactions.
    This function must be invoked from a non-edited routine in the CACHESYS database and not part of any indirection; or else be in a job that holds the WRITE permission on the CACHESYS database.
    130 %Manager
    Subject: set or return the domain ID or index
    This function must be invoked from a non-edited routine in the CACHESYS database and not part of any indirection; or else be in a job that holds the WRITE permission on the CACHESYS database.
    131 N/A
    In previous releases, the subfunction, 1, returned the system identifier string consisting of the current system name followed by a colon (:), the IP address (Windows) or MAC address (UNIX®) followed by a comma (,), and the pathname of the mgr directory.
    In version 5.1, it returns the name of the system, followed by a colon (:), and the name of the Caché instance that is running.
  • $ZUTIL(4) Change

    This function is used to stop processes in Caché. In version 5.1, it has been refined to protect system jobs from interference by unprivileged applications. For example, $ZUTIL(4, <pid>) will no longer terminate a daemon. Instead, it will return an error status of 0.

    Moreover, a process which is exiting and running %HALT, or any of its subroutines such as %ROLLBACK, will not respond to this function. The process issuing the RESJOB will now receive an error status of -4 meaning that the target ignored it.

    Shutting down the system with “ccontrol stop” will terminate these processes as it has in the past. This can also be done in version 5.1 with the function, $ZUTIL(4, <pid>, -65).

    Caution:

    When , $ZUTIL(4, <pid>, -65) is used for this purpose, any open transactions will not be rolled back even though the locks which protected it will be released.

  • $ZUTIL(49) Change

    The information it returns has been extended to better describe the database:

    • $ZU(49, <sfn>, 3) — Version 5.1 adds several fields to the end of the previously returned information:

      • <SysNumber>: the remote system number, or zero if the database is local

      • <DirPath>: the database path on the system

      • <ResourceName>: the resource associated with the database

      • <BlockSize>: the database block size in KB

      • <Collation>: the database collation

      • <DirectoryBlock>: the DB directory block number. For ECP clients this is the local cache directory block number of the database in CacheTemp. The ECP server does not send the database directory block number to the clients.

      All the values returned are separated from each other by “^”.

$ZF

Permission checking is also applied to some operations of $ZF; the following table lists the permission needed for those whose execution is restricted.

Function Number
Permission
Explanation
—1 %System_Callout:U
Executes a program or command as a spawned child process and waits for the child process to return.
—2 %System_Callout:U
Executes a program or command as a spawned child process and returns immediately.

The other $ZF functions remain unprivileged operations as in previous versions of Caché.

Storage Changes

%CacheStorage Changes
  • New Property Method: GetStored()

    A new property method is now implemented for all storable properties of persistent classes that are using default storage (%CacheStorage). It is <propertyname>GetStored(). This method accepts an object ID value (not an OID) and returns the logical value of the property as stored on disk. If <property>StorageToLogical() is used then it is applied to convert the stored value to a logical value.

    This method is not valid for collections stored in a subnode structure. If an object identified by the supplied ID does not exist, an error will be reported. This method is not implemented for properties that are not stored: transient, multidimensional, or calculated properties. In these cases, Caché will report a <METHOD DOES NOT EXIST> error.

  • ID Counter Check Available With Default Storage

    A new API function to check some system assigned object ID counters is now available in this version. An id check expression is generated by the compiler for each class using default storage with system assigned id values. (Child classes using default storage and system assigned IDs do not have this expression.)

    The function, $$CheckIDCounters^%apiOBJ(.errorlog), will examine all extents in the current namespace. If an idcheckexpression is found, it will be invoked. The id check expression will fail if the last id in the extent has a value higher than the id counter location. In this case, an entry is placed in the errorlog array, subscripted by extent name. The id check expression is also included in errorlog so that the user can repair the problem.

    An application should invoke this utility with:

    Set sc = $$CheckIDCounters^%apiOBJ(.errarray) 
    

    After the utility returns, sc will be set to a standard status message. If errors were found, they will be stored in the multidimensional array, errarray.

  • %ExistsId() Is Now Generated For Default Storage

    In Caché 5.1, this generated method will now validate the id value passed. If any components of the id are null, %ExistsId() will return zero (the object does not exist). If all components are not null, then the object reference will be checked for existence.

    Note:

    This method is meant to be called in the class that is an extension of %Library.Persistent. Passing in an ID value that is not constructed by the class of which it is an instance not recommended as it breaks encapsulation.

%CacheSQLStorage Changes
  • Valid Row References Required For $PIECE Access Types

    Applications using %CacheSQLStorage that employ two or more subscript levels of Access Type, $Piece, and have specified Data Access expressions for those subscript levels, must supply a valid Row Reference in the map definition. This version of Caché will no longer automatically generate one under these circumstances.

  • New Dynamic Value Substitution

    Caché now supports the use of

    • {%%CLASSNAME}: expands to the name of the class without quotes, for example,

      Do ##class({%%CLASSNAME}).MyMethod() 
      Copy code to clipboard
    • {%%CLASSNAMEQ}: expands to the name of the class within quotes,

      Set ThisClass = {%%CLASSNAMEQ}
      Copy code to clipboard
    • {%%TABLENAME}: expands to the name of the table within quotes,

      Set MyTable = {%%TABLENAME} 
      Copy code to clipboard

    in the following locations within a %CacheSQLStorage map definition:

    • Map Subscript

      • Data Access expression

      • Invalid Conditions

      • Next Code

      • Access Variable Expressions

      Map Data

      • Retrieval Code

Java

Package Names May Not Be SQL Reserved Words

If a Caché class is to be projected to Java, and any component of the package part of the projected class name matches an SQL reserved word (ignoring case), the attempt to project the class will report an error that the metadata for the Java class is missing its column names. This error can be avoided by using package names that are not the same as any SQL reserved word.

Terminal

Terminal Is Always Unicode Now

There is now only one version of TERMINAL which runs internally using Unicode characters. By default, it starts with the ISO network encodings "Local Encoding 2" and "Network Encoding 2". In order to display characters > 255 you must change the encoding to UTF8. As a result of this enhancement, the “Pass 8–bit Characters” setting has been removed.

When there are multiple instances of Caché, some Unicode and some 8 bit, it is good practice to set the encoding explicitly for each TERMINAL instance. Then the defaults no longer apply.

Argument Changes

Terminal no longer supports command arguments /size, /pos, and /ppos. It has been enhanced to handle characters internally in Unicode and provide for the proper translations to and from servers in different locales.

Password Echo

In previous releases, when the TERMINAL prompted for a password, it did not echo any characters to the output device. As of version 5.1, when Caché password login is enabled, each character of the password will be echoed as an asterisk (*). Any application that performs a login supplying a userid and password at the TERMINAL prompt must be made aware of the echoing behavior if it does pattern matching on the characters TERMINAL transmits.

Launching From The Windows Cube

In this version of Caché, the way the Cube determines whether to use Telnet or TRM for a remote TERMINAL session has changed. Servers are placed in the list displayed under the "Remote System Access" menu according to these rules:

  1. Remote servers are always shown as enabled.

  2. Local servers where the IP address of the server is not that of the local host, and the server name is not a local instance name will be treated like remote servers because the server is not associated with a local instance.

  3. Local servers (where the IP address is the local host address, and the server name is the same as a local instance name) will be grayed if the configuration is down or telnet is disabled for that instance. Otherwise the server name will be enabled.

  4. A telnet connection will always be available when using the Remote System Access menu to launch a terminal.

When the terminal is launched from the main cube menu:

  • If the active preferred server is associated with that instance, a private terminal connection will be made. The ttle bar for the TERMINAL windows will contain “TRM” followed by the process id and the name of the Caché instance. If an instance is not running, the instance will be started before launching the terminal.

  • Otherwise, a telnet connection will be made. The terminal's title bar will contain the hostname followed by “NTI - Cache Telnet". The cube never starts an instance of Caché different from the one it was installed with.

Local And Network Encodings Are Now Distinct

The local and network translation settings for Terminal are now stored separately for 8-bit and Unicode installations to permit the user to choose different behavior for Unicode and 8-bit installations which may exist on the same host. In prior versions, they had been the same.

SOAP Parameter Location Changes

This version changes the location of the parameters that control SOAP logging behavior. In previous versions these were in ^%SYS. In 5.1, they reside in the namespace from which the SOAP request is made. The parameters at issue are:

  • ^ISCSOAP(“Log”) — set to “1” when web services requests and client responses should be logged

  • ^ISCSOAP(“LogFile”) — the full pathname of the file where the logged information is placed

Callin And Callout

On The Windows Platform

On Microsoft Windows, Caché is now compiled with Visual Studio 7.1. User applications communicating with Caché using the CALLIN or CALLOUT interfaces must be upgraded to this version of Visual Studio.

CSP Changes

CSP Grace Period Changed

As part of the licensing changes introduced with Caché version 5.1, how CSP treats sessions has changed.

If a CSP session visits more than one page and the session is ended either from a session timeout or from the application setting %session.EndSession=1, CSP will release the license immediately rather than adding on an extra grace period.

If the session is just active for a single page, CSP will hold the session open for a five–minute grace period when the session is ended.

CSP Page Timing Statistics Default To Off

In Caché 5.1, the class parameter, PAGETIMING, has been changed to have a default value of zero. In earlier versions, its default value was 1. The zero value turns off the collection of page timing statistics for all classes that inherit from it. If an application relies on the page timing statistics being collected for CSP pages, then it will need to be modified to inherit from a superclass that has PAGETIMING set to 1.

Collation For Locales Now On By Default

Please refer to the discussion in the Administrator section.

Caché Dreamweaver Extension Revised

The Dreamweaver extension has been extensively revised to improve its security in version 5.1. It now uses the C++ binding exclusively. Users who wish to use this extension must have the %Development privilege. The extension will continue to work for those users without this privilege but no data from Caché will be visible in accordance with our security rules.

As a result of this change, the following must be true:

  • The Dreamweaver extension now requires Cache 5.1 on the server for operation.

  • The following directory must be present in the %path% environment variable,

    \Program Files\Common Files\Intersystems
    

Operators

System Management Portal

Prior to this version, how Caché was administered depended heavily on the platform where Caché ran. With version 5.1, InterSystems introduces a new administrative interface that is common across all platforms. Caché 5.1 now uses a browser-based interface, the System Management Portal, for system management.

A brief summary can be found in the Administrator section of this document and more complete information on the System Management Portal can be found in the System Administration documentation.

PERFMON And %SYS.MONLBL Coordination

These utilities each use some of the same Caché data structures for gathering data. So they should not execute at the same time; otherwise there is a risk that they may compromise each other's data. In version 5.1, program checks have been added to prevent their simultaneous execution.

Backup Information Changes

Beginning with version 5.1, the location where the backup database list is maintained has been changed. ^SYS("BACKUPCHUI") is no longer used. The list is maintained by the methods, Backup.General.AddDatabaseToList() and Backup.General.RemoveDatabaseFromList(). Moreover, InterSystems strongly recommends against setting it manually since this works at cross-purposes with the methods. Use the System Management Portal or the ^BACKUP utility instead.

New Question In Journal Restore

In prior versions, the journal restore routine, ^JRNRESTO, did not properly handle the restoration of journal files written on different operating systems. The error occurred in the handling of directory names specified by those systems.

Caché version 5.1 now accounts for this by asking whether the journal was produced on a different kind of operating system. However, if a site is using a script to drive the journal restore, the script will have to be modified to provide an answer to the new question.

Cluster Member Startup Improved

The logic for a Caché instance to join a member of a cluster has been improved to avoid confusion between systems making up the cluster. For details, please see the Administrator portion of this book.