Skip to main content

This is documentation for Caché and Ensemble. See the InterSystems IRIS version of this content.Opens in a new tab

Please read this important announcement about the future of Caché and Ensemble maintenanceOpens in a new tab. Contact your account team to migrate to InterSystems IRIS.

Configuring Distributed Systems

An ECP application consists of one or more ECP data server systems — data providers — distributing to one or more ECP application server systems — data consumers. The primary means of configuring an ECP application is using the ECP Settings page of the Management Portal (System Administration > Configuration > Connectivity > ECP Settings).

Once you have decided how to distribute your data, configuring an ECP application is very straightforward:

  1. Enable each system that provides data as an ECP data server. See the Configuring an ECP Data Server section for instructions.

  2. If you are using Security, see the Managing ECP Privileges section for a discussion on how resources, roles, and privileges are managed in an ECP configuration.

  3. Specify each system that requests data as an ECP application server for each data server with which it wishes to communicate. See the Configuring an ECP Application Server section for instructions.

  4. In addition, configure each ECP application server system so that it can see remote data in the defined ECP data servers. See the Configuring ECP Remote Data Access section for instructions.

  5. ECP shares the buffer pool with the local instance of Caché; therefore, InterSystems recommends allocating additional buffers to accommodate ECP. See the Memory Use on Large ECP Systems section of the “Developing Distributed Applications” chapter of this guide for details.

A system operating as an ECP data server can simultaneously act as an ECP application server, and vice versa. You may configure your ECP application and data servers in any order; you do not need to enable an ECP data server before defining an application server.

Configuring an ECP Data Server

To configure a system as an ECP data server, you must first enable the ECP service from the Services page of the Management Portal (System Administration > Security > Services). Click %Service_ECP, select the Service enabled check box, and click Save. This is the only configuration setting required to use this system as an ECP data server.

Alternatively, from the ECP Settings page, click Edit next to The ECP service is Disabled to navigate to the same Edit Service page. When you click Save, you return to the ECP Settings page.

To see a list of ECP application servers that have been configured to connect to this data server, click the Application Servers button on the ECP Settings page.

Note:

For a detailed explanation of Caché services, see the “Services” chapter of the Caché Security Administration Guide.

Update the Maximum number of application servers setting to specify the maximum number of application servers that can possibly access this data server simultaneously. Caché allocates a limited number of application server nodes. Increase the default value of 1 up to a maximum of 254 to avoid a system restart, which is required when the number of connections becomes greater than the number of allocated nodes.

Note:

If you increase the maximum number of application server, you must restart Caché.

The ECP data server is now ready to accept connections from valid ECP application servers.

You may wish to restrict access to the data server. See the following sections for ways to do this:

Restricting ECP Application Server Access

You can restrict which systems can act as ECP application servers for an ECP data server system by performing the following steps:

  1. From the Services page, click %Service_ECP.

  2. In the Allowed Incoming Connections box, click Add and enter a single address (for example, 192.9.202.55 or mycomputer.myorg.com) or a range of addresses (for example, 18.61.202–210.* or 18.68.*.*).

If you enter IP addresses in the Allowed Incoming Connections list, the ECP data server only accepts incoming ECP connections from application servers whose IP is in the list. If the list is empty, any application server can connect to this system if the ECP service is enabled.

After you add an IP address, it appears in the list of Allowed Incoming Connections with options to Delete the address from the list and Edit the Roles of the connection.

This process of managing roles on ECP data and application servers is part of Caché security. For details on how Caché roles and privileges work in general see the “Roles” chapter of the Caché Security Administration Guide. The following section details how these features work with ECP.

Specifying ECP Privileges and Roles

For each specified IP address or range of addresses, click Edit to display the Select Roles area that allows you to specify the roles associated with the connection from the IP address. By default, the connection holds the %All role. If you specify one or more other roles, these roles are the only roles that the connection holds. Hence, a connection from an IP address with the %Operator role has only the privileges associated with that role, while a connection from a different IP address with no associated roles (and therefore %All) has all privileges.

To specify the roles associated with an IP address:

  1. Select roles from those listed under Available and click the right arrow to add them to the Selected list.

  2. To remove roles from the Selected list, click them and then click the left arrow.

  3. To add all roles to the Selected list, click the double right arrow; to remove all roles from the Selected list, click the double left arrow.

  4. Click Save to associate the roles with the IP address.

The Managing ECP Privileges section discusses how Caché manages privileges within an ECP configuration.

Managing ECP Privileges

The following discussion assumes that resources and roles refer to the same assets on each machine. To be granted access to a resource on the ECP data server, the role held by the process on the application server and the role set for the ECP connection on the data server must both include permissions for the same resource.

By default, Caché grants the ECP data server the %All privilege when the data server runs on behalf of an ECP application server. This allows it to return any data in any database that the application server requests. Caché restricts access to this data on the application server based on the privileges of the user requesting the data on the application server.

For example, for a user on the application server who only has privileges for the %DB_USER resource, data in the USER database on the data server is available (which by default is assigned the %DB_USER resource), but attempting to access the SAMPLES database on the data server results in a <PROTECT> error. If a different user on the application server has privileges for the %DB_SAMPLES resource, then the SAMPLES database on the data server is available.

You can also restrict the set of roles on the data server based on the IP Address of the application server. For example, on the data server you can specify that when interacting with an application server named NODE_A the only available role is %DB_USER. In this case, users on the application server granted the %DB_USER role can access the USER database on the data server. However, users on the application server with %DB_SAMPLES access receive a <PROTECT> error if they try to access the SAMPLES database on the data server (since the data server is only set up with %DB_USER access).

The following are exceptions to this behavior:

  • Caché always grants the ECP data server the %DB_CACHESYS role since it requires Read access to the CACHESYS database to run. This means that a user on an ECP application server with %DB_CACHESYS can access the CACHESYS database on the ECP data server.

    To prevent a user on the application server from having access to the CACHESYS database on the data server, there are two options:

    • Do not grant the user privileges for the %DB_CACHESYS resource.

    • On the data server, change the name of the resource for the CACHESYS database to something other than %DB_CACHESYS, making sure that the user on the application server has no privileges for that resource.

  • If the ECP data server has any public resources, they are available to any user on the ECP application server, regardless of either the roles held on the application server or the roles configured for the ECP connection.

Changes both to the configuration of the ECP connection and to the public permissions on resources require a restart of Caché before taking effect.

Security-related ECP Error Reporting

The behavior of security-related error reporting with ECP varies depending on whether the check fails on the application server or the data server and the type of operation:

  • If the check fails on the application server, there is an immediate <PROTECT> error.

  • For synchronous operations on the data server, there is an immediate <PROTECT> error.

  • For asynchronous operations on the data server, there is a possibly delayed <NETWORK DATA UPDATE FAILED> error. This includes Set operations.

Configuring an ECP Application Server

To configure a system as an ECP application server, you define an ECP data server from which to retrieve data. Add this remote ECP data server by performing the following steps:

  1. From the ECP Settings page, click Data Servers to display a list of currently configured ECP data servers.

  2. Click Add Server to add a data server.

  3. Enter the following information for the data server:

    • Server Name — Enter a logical name for the convenience of the application system administrator.

    • Host DNS Name or IP Address — Specify the host name either as a raw IP address (in dotted-decimal format or, if IPv6 is enabled, in colon-separated format) or as the Domain Name System (DNS) name of the remote host. If you use the DNS name, it resolves to an actual IP address each time the application server initiates a connection to that ECP data server host. For more information, see the IPv6 Support section in the “Configuring Caché” chapter of the Caché System Administration Guide.

      Important:

      When adding a mirror as an ECP data server, do not enter the virtual IP address (VIP) of the mirror, but rather the DNS name or IP address of the current primary failover member. Because the application server regularly collects updated information about the mirror from the specified host, it automatically detects a failover and switches to the new primary failover member. See the “Mirroring” chapter of the Caché High Availability Guide for information about mirror failover and VIPs.

    • IP Port — The port number defaults to 1972; change it as necessary to the superserver port of the Caché instance on the data server.

    • Select the Mirror Connection check box if this data server is the primary failover member of a mirror.

  4. Click Save.

Once you add a remote ECP data server, it appears in the list of defined data servers this application server can connect to at the bottom of this same portal page. Add additional ECP data servers to the list using the Add Remote Data Server link. Remove or edit server definitions using the Delete and Edit links, respectively. You may also click Change Status of the connection. See the “Monitoring Distributed Applications” chapter for details.

You may add as many data servers as allowed by the Maximum number of data servers setting. Update this value to specify the maximum number of server connections the application server may need later so that Caché reserves enough system resources so as not to require a restart each time you add a data server. Increase the default value of 2 up to a maximum of 254.

Note:

If you increase the maximum number of data servers, you must restart your Caché.

Your system is ready to act as an ECP application server. No further user intervention is required; when the ECP application server needs access to the ECP data server, it automatically establishes a connection to the server.

Configuring ECP Remote Data Access

After defining a list of one or more ECP data servers for an ECP application server, configure the ECP application server system so that it has access to data stored in the ECP data server system. Do this by defining a remote database on the ECP application server system.

A remote database is a database that is physically located on an ECP data server system, as opposed to a local database which is physically located on the local application server system.

To define a remote database on the ECP application server, perform the following steps:

  1. Navigate to the Remote Databases page of the Management Portal (System Administration > Configuration > System Configuration > Remote Databases).

  2. Click Create New Remote Database to invoke the Database Wizard, which displays a list of the logical names (the name you used when you added it to the list of ECP data servers) of the remote data servers on the application server.

  3. Click the name of the appropriate ECP data server and click Next.

  4. The portal displays a list of database directories on the remote ECP data server. Select one of these to serve as the remote database.

  5. Enter a database name (its name on the ECP application server; it does not need to match its name on the ECP data server) and click Finish. You have defined a remote database.

Next, define a new namespace (or modify an existing namespace) to view the data in the remote database as you would in a local database.

Note:

By using the Namespace Wizard in the Management Portal, you can define a namespace and a remote database at the same time, thereby combining these two procedures for adding a remote database.

To define a new namespace that views the data in a remote database perform the following steps:

  1. Navigate to the Namespaces page of the Management Portal (System Administration > Configuration > System Configuration > Namespaces).

  2. Click Create New Namespace.

  3. Fill in the form with the following fields:

    • Enter a name for the new namespace.

    • Click Remote Database.

    • If you created a remote database as described previously, select it; otherwise click Create New Database and follow the previous Database Wizard instructions.

    • If you use CSP, select Create a default CSP application for this namespace.

  4. Choose a database for the new namespace. Select the remote database from the list (remote and local databases are listed together) and click Next.

  5. Click Save. You have a new namespace that is mapped to a remote database.

Any data retrieved or stored in this namespace is loaded from and stored in the physical database on the ECP data server and updated in the local application server system cache if it is already cached.

ECP Security Notes

First, all the instances in an ECP configuration need to be within the secured Caché perimeter (that is, within an externally secured environment). This is because:

  • ECP is a basic security service (not a resource-based service), so there is no way to regulate which users have access to it. For more information on basic and resource-based services, see the “Available Services” section of the “Services” chapter of the Caché Security Administration Guide.

  • Caché does not support SSL/TLS to secure ECP connections. For more information on the use of SSL/TLS, see the “Using SSL/TLS with Caché” chapter of the Caché Security Administration Guide.

Also, when using encrypted databases on ECP data servers, it is recommended to encrypt the CACHETEMP database on all connected application servers. The same or different keys can be used. For more information on database encryption, see the “Managed Key Encryption” chapter of the Caché Security Administration Guide.

Next sectionMonitoring Distributed Applications
Previous sectionECP
FeedbackOpens in a new tab