Introduction to the CSP Gateway

Gateway components and physical installation paths

Later sections in this guide describe how the Gateway components should be configured with all supported web servers. The installation paths for components should be regarded as examples rather than taken literally. Also, the InterSystems installers create and maintain separate Gateway installations for the internal (private) web server and any third-party web server that might be present on the same host. In this context ‘third-party web server’ refers to a web server that is not part of the software installed by InterSystems.

The precise installation location of Gateway components is not particularly critical provided:

• The physical installation paths match those given in the hosting web server configuration where appropriate.

• The security settings, in relation to required access for individual components, are adjusted appropriately. This is particularly important for Gateway components that are accessed directly by the web server since web servers are usually locked down to the extent that the files they are able to access (and executables that can be run) are carefully controlled. It should be borne in mind that security considerations will also be important for any Gateway configuration (and log) files that are accessed by Gateway binaries that are themselves bound to the web server core executable.

• The security policy of the hosting web server is respected. Some web servers – notably that shipped with Secure Linux (SELinux) – are configured such that it is not possible for them to access files that lie outside their own file system. This restriction will clearly have an impact on where certain web-server-facing Gateway components can be installed.

There are four types of Gateway component to consider.

1. Binaries to be loaded by the web server (API based extensions).

   This includes Windows DLLs and UNIX Shared Objects:

   CSPms[Sys].dll
   CSPn3[Sys].(dll|so|exe)
   CSPa*[Sys].(dll|so)
   mod_csp*.(so|exe)
   

   

   The physical location where these are installed should match the corresponding configuration directives in the hosting web server configuration. This includes directives indicating which third-party modules should be loaded. The web server requires permission to read and load these modules. Modules named CSP* require permission to read and write to the Gateway configuration and log files (CSP.ini, CSP.log). These are usually created in the same location as the Gateway binaries.

   When considering access control for these modules, bear in mind that it is the web server worker processes that need to be able to access the modules together with any dependent configuration and log files. For example, in the case of Apache, the server is usually started with superuser permissions but the worker processes that actually serve web requests run with a much lower level of authority (as indicated by the User and Group directives in the Apache configuration file). It is the User and Group specified for the worker processes that should be granted permission to load the Gateway modules and (where appropriate) the ability to read and write to the configuration and log files (CSP.ini and CSP.log).

2. Executables to be called by the web server (CGI modules). (Not all configurations require these executables.)

   [nph-]CSPcgi[Sys][.exe]
   

   

   The physical location where these are installed should match the corresponding configuration directives in the hosting web server configuration. This will include directives indicating which web requests should be processed by these CGI modules.

   The worker processes of the hosting web server require execute permission for these modules. There are no further dependencies.

3. Static files to be returned by the web server.

   Note:

   With current CSP Gateway configurations, CSP is often configured to serve static files directly from Cache as opposed to having the web server return them. This section does not apply to such configurations.

   JavaScript modules (such as CSPBroker.js, CSPxmlhttp.js, and so on)

   Java applets (such as CSPBroker.class, CSPBroker.jar, and so on)

   Images (such as created-with-csp.gif, and so on)

   The worker processes of the hosting web server require Read permissions for these files.

4. The CSP Network Service Daemon (NSD)

   Note:

   Not all configurations require this facility.

   CSPnsd[Sv][.exe]
   

   

   This can be installed anywhere and the web server does not need to be aware of its physical location since communication between these two points is over TCP (usually port 7038).

   The NSD requires permission to read and write to the Gateway configuration and log files (CSP.ini and CSP.log), which are usually created in the same location.

   Note:

   For security reasons, do not install this module in a location that is accessible by the web server. This module should not share a location with the modules listed in steps 1, 2 or 3. Many web server configurations described in this document explicitly exclude this module from the list of accessible files that can be accessed by the web server. However, it is much safer to physically install the NSD elsewhere in the file system.

The Gateway, where possible, locates the content of large files (such as large JavaScript files in Zen applications) together with the accompanying HTTP response headers in permanent storage. The essential control information (expiry time and so on) and the index of all cached files is in the shared memory sector. With this architecture, each cache entry consumes a small amount of cache space (in terms of memory usage): quite possibly no more than one cache block per entry.

Cached content is stored in files of type .dat in the Gateway's temp directory, placed by the install script directly beneath the Gateway's installation directory. For example, in a typical IIS installation this is in: C:\inetpub\CSPGateway\temp. The location needs full read/write/delete permissions for the hosting web server worker processes.

Configuring the CSP Gateway for Ensemble

The CSP Gateway provides web connectivity for both the Caché and Ensemble product lines. The Gateway components are the same in both cases. A Gateway installation provided with a Caché installation can provide web connectivity to an Ensemble installation and vice versa.

For simplicity, this document is written for use with Caché installations. However, with the exception of default path names, the instructions contained in this document apply equally to Ensemble. The default Ensemble names replace Cache with Ensemble.

Gateway Management Module configuration

Gateway architectures that work directly to a hosting web server’s API typically consist of two modules: A Management Module (for example, CSPmsSys.dll) and a runtime module (for example, CSPms.dll). The runtime Module is responsible for processing requests for CSP files and the Management Module provides the Gateway’s Management interface. In the CSP Gateway, the runtime Module assumes responsibility for loading and routing management requests to the management module. All requests for the CSP Gateway (CSP and management) are processed by the runtime Module. The Management Module must be installed in the same location as the runtime Module

File Types Served by CSP

Files of type .csp, .cls and .zen are processed in Caché by CSP. All other files (static files) can be served by the web server or CSP. CSP can serve any type of file that is placed in the CSP applications path (including static files). Setting up CSP to serve static files simplifies the web server configuration for CSP applications because you, thus, do not need to create aliases in the web server configuration to represent the locations where an application’s static files are held. Setting up CSP to serve static files resolves issues of contention when a single (that is, common) web server serves two different versions of Caché, each requiring different versions of certain static files (for example, hyperevent broker components).

To have CSP serve static files for a particular CSP application, place the static files in the CSP application’s file system in the correct location relative to the CSP files that make up the application (not in the web server’s own documents file system).

Consult the sections in this book for your platform.

SET ^%SYS("CSP", "MimeFileClassify", "JS") = $listbuild("text/javascript", 0 ,"ISO-8859-1") 

Serving Static Files from Caché

You can configure web servers and Gateway installations so that Caché assumes responsibility for serving static files. The Management Portal and the Zen samples are supplied configured for Caché to serve all components in the application. However, it is still possible to configure the web server so that it retains responsibility for serving statics.

Hybrid Multi-Process/Multi-Threaded Web Server Architecture

The Gateway contains enhanced support for the hybrid multi-process/multi-threaded web server architecture. Apache version 2.x.x under UNIX is an example of a web server implemented according to this architecture.

With earlier builds of the Gateway, the stand-alone modules (CSPa2*[Sys].so) were supported but each and every hosting web server worker process would bind to and create an independent instance of the Gateway. In other words, each worker process would load its own instance of the Gateway which would, in turn, load its own copy of the running configuration, manage its own connection table and form cache. The main observable symptom of this behavior being the variability in the Gateway System Status form each time Refresh was selected. Each refresh of the Status form would result in the status for just the worker process and Gateway instance that happened to be processing the request for the status form. Also, state-aware mode (preserve mode 1) could not be supported in the multi-process architecture since this mode of operation depends on being able to route all requests for a session to the same Caché process which, in turn, means that all connections to Caché must be held in a single multithreaded process.

An architecture in which all Gateway resources are duplicated across all web server processes is unnecessary and wasteful.

The core Gateway resources are now held in the shared memory sector. All web server worker processes hare a common running configuration, connection table and form cache. The Gateway System Status form shows the status for the whole web server installation instead of just that of a single worker process. The status form’s connection table includes an extra column with the web server process ID with respect to each connection to Caché.

Finally, state-aware sessions are supported in the multi-process architecture. Although the connection pool (to Caché) is distributed amongst several web server processes, the Gateway uses an InterProcess Communications (IPC) protocol to route requests for state-aware sessions to the correct hosting process in the web server environment.

Gateway Registry

The Gateway is supplied with the new Caché based Gateway Registry. All web server and Gateway installations are registered with Caché as they connect. The registry contains the infrastructure to allow Caché code to interact with connected Gateway installations for the purpose of reading and writing the configuration and monitoring the system status and Event Log.

Enable Sticky Sessions on Hardware Load Balancer on High Availability Solutions

For High Availability solutions running over CSP, InterSystems recommends that you use a hardware load balancer for load balancing and failover. InterSystems requires that you enable sticky session support in the load balancer; this guarantees that -- once a session has been established between a given instance of the gateway and a given application server -- all subsequent requests from that user run on the same pair. This configuration assures that the session ID and server-side session context are always in sync; otherwise, it is possible that a session is created on one server but the next request from that user runs on a different system where the session is not present, which results in runtime errors (especially with hyperevents, which require the session key to decrypt the request). See your load balancer documentation for directions on how to enable sticky session support.

Enable Script to Reactivate Gateway Configuration

You can enable an external (non-Cache based) script to reactivate the Gateway’s configuration.

Scripts should add the following line (case-sensitive) to the SYSTEM section of the Gateway configuration file:

   [SYSTEM]
   RELOAD=1

The Gateway caretaker daemon checks the RELOAD flag approximately every minute and, if correctly set, reloads and reactivates its configuration and removes the flag from the file. The following message is written to the Event Log after a successful reload operation:

   Gateway Management
   Gateway Configuration Reloaded and Reactivated

Building the Private Web Server

The (default) full Apache server is usually created with the following sequence of commands:

./configure --prefix=<install-dir>
make
make install

The minimal Apache build is typically created as follows:

./configure --prefix=/usr/cachesys/httpd --with-port=57772 \
            --with-pcre=$srcdir/pcre \
            --enable-mods-static="log_config mime alias unixd authz_core" \
            --disable-ssl \
            --enable-so --without-gdbm --without-ndbm \
            --without-berkeley-db --with-included-apr --with-expat=builtin \
            --with-mpm=prefork --disable-shared
make
make install

Notice that many of the services that are normally required for a production grade installation are excluded.

While this server can be used to host other CSP applications it is strongly recommended that a full, independent web server installation is used for this purpose. It should be remembered that any changes made to the configuration of the Management Portal Apache installation are overwritten when the hosting Caché installation is upgraded.

The Management Portal Apache installation uses the following CSP Gateway modules for communicating with Caché:

• Windows: CSPa2.dll and CSPa2Sys.dll

• UNIX®: CSPa2.so and CSPa2Sys.so

Managing the Private Web Server

Under normal operational conditions, the Management Portal Web Server for a particular instance of Caché is started when Caché is started and closed down when Caché is closed down. Occasionally it may be necessary to restart the Management Portal Web Server without disrupting the corresponding Caché server. For example, a web server restart is necessary if a configuration change is made to the web server (httpd.conf).

Use the following commands to start and stop the Management Portal Web Server.

Start the Management Portal Web Server:

<cache-install-dir>\httpd\bin\httpd -k start -n <instname>httpd -c "Listen <port>"

Stop the Management Portal Web Server:

<cache-install-dir>\httpd\bin\httpd -k stop -n <instname>httpd

Caché installed in: C:\cachesys

Caché instance name: CACHE

TCP port for Apache: 57772

Start:

C:\cachesys\httpd\bin\httpd -k start -n CACHEhttpd -c "Listen 57772"

Stop:

C:\cachesys\httpd\bin\httpd -k stop -n CACHEhttpd

Start the Management Portal Web Server:

<cache-install-dir>/httpd/bin/httpd -d <cache-install-dir>/httpd -c "Listen <port>"

Stop the Management Portal Web Server:

kill `cat <cache-install-dir>/httpd/logs/httpd.pid`

Caché installed in: /usr/cachesys

TCP port for Apache: 8972

Start:

/usr/cachesys/httpd/bin/httpd -d /usr/cachesys/httpd -c "Listen 8972"

Stop:

kill `cat /usr/cachesys/httpd/logs/httpd.pid`

You can also use the following methods of the class Config.StartupOpens in a new tab:

• RestartWebServerOpens in a new tab – Restarts the private web server.

• StopWebServerOpens in a new tab – Stops the private web server.

Limitations of the Private Web Server

This section discusses the differences between the configuration of the PWS and that of a typical production grade Apache installation.

Windows-based Apache installations use a special multithreaded form of the Apache Multi-Processing Module (MPM) which is better suited to the way the operating system is optimized. Therefore, the behavior of the PWS under Windows is similar to that of a production grade Apache build as far as the ability to handle concurrent load is concerned.

If high availability and production-grade security is a requirement, or there is a need to integrate with other sources of web information, or a need for a high degree of control over the web server, a separate production-grade build of Apache is recommended - ideally operating on its own server. If, on the other hand, low volumes of HTTP traffic are expected, and there are limited demands for high availability and security, then the PWS may be suitable for deployment under these circumstances.

The PWS defaults to using the Apache Group’s prefork Multi-Processing Module (MPM). This is a non-threaded server model: the number of requests that can be concurrently served is directly related to the number of Apache worker processes in the pool.

The PWS is configured to occupy the smallest possible footprint by allowing a maximum of two worker processes to be created for the pool. The following settings will be found in the Apache configuration (httpd.conf) for the PWS:

MinSpareServers 1
MaxSpareServers 2

By contrast, the default Apache configuration for a production grade build is usually as follows:

StartServers       5
MinSpareServers    2
MaxSpareServers   20
ServerLimit      256
MaxClients       256

This configuration allows Apache to create 5 worker processes at startup, increasing to a maximum of 256 as the concurrent load increases. Because of these differences in configuration, the performance of the PWS is noticeably inferior to that of a production grade Apache build. This performance deficit becomes more noticeable as the concurrent load increases. However, it is possible to change the configuration of the PWS to match that of a full Apache installation (shown above). Apache must be completely restarted after changing these parameters.