%Net.HttpRequest provides an interface to issue HTTP requests to a web server and read the response.
This allows you to interact with other web sites, for example you could issue a request to get
a stock quote from another site, then parse the information returned to store the stock value
in the local database. This implementation is designed to the HTTP/1.1 specification
In normal use you create a single %Net.HttpRequest object and use it to issue as many
requests as you need to the web server, each %Net.HttpRequest object can be thought of as
a single instance of an internet browser.
On completion of each request it returns a
Set httprequest=##class(%Net.HttpRequest).%New() Set httprequest.Server="www.intersystems.com" Do httprequest.Get("/") Do httprequest.HttpResponse.OutputToDevice() Do httprequest.Get("/cache/") Do httprequest.HttpResponse.OutputToDevice()
http://www.demo.com/page.html?PARAM1=TEST&PARAM2=HELLOYou use the
Set httprequest=##class(%Net.HttpRequest).%New() Set httprequest.Port=1972 Do httprequest.SetParam("PAGE","INDEX") Do httprequest.Get("/csp/docbook/%CSP.Documatic.cls") Do httprequest.HttpResponse.OutputToDevice()
Form data can also be sent with this class. This is done by first inserting the form name/value pairs using the
Do httprequest.InsertFormData("element","value") Do httprequest.Post("/cgi-bin/script.CGI")
If true then we report we can accept gzip compressed data to the web server (the default), if false then do not send the header saying we accept gzip data. property AuthenticationErrors as
If an attempt to get an authentication handle or token for a scheme fails, then the underlying error is saved to the AuthenticationErrors property. property AuthenticationSchemes as
The value of AuthenticationErrors is a $list which contains the errors. Each entry is of the form:
%Net.HttpRequest will attempt to always respond with an Authorization header with a supported authentication scheme to a 401 status code. The supported authentication schemes are specified by the AuthenticationSchemes property as a comma separated list of case-sensitive authentication schemes. The possible authentication schemes are Negotiate, NTLM and Basic. property Authorization as
The caller needs to supply Username and Password properties. If the Username property is not specified on Windows and a 401 status code associated with WWW-Authenticate header for SPNEGO or NTLM is received, then the current login context will be used.
When responding to a 401 status code, the authentication scheme that is tried is picked from the intersection of those specified by WWW-Authenticate headers in the response and rhe list in the AuthenticationSchemes property. The schemes are checked in the order of the AuthenticationSchemes list. The first scheme which is supported and configured for IRIS will be used to return an Authorization header with that scheme. In order to encourage the user to try more secure schemes first, the default value for Authentication schemes is "Negotiate,NTLM,Basic".
Sets/get the 'Authorization:' header field in the Http request. property ContentCharset as
A user agent that wishes to authenticate itself with a server-- usually, but not necessarily, after receiving a 401 response--may do so by including an Authorization header field with the request. The Authorization field value consists of credentials containing the authentication information of the user agent for the realm of the resource being requested.
This class understands the Basic authentication mechanism and if you set the
Usernameand Passwordproperties then it will send this information suitably encoded to the server with each request. Note that Basic authentication is not secure and sends to username and password over the network in just base64 encoded form.
This is the charset to encode the contents with. This is actually specified in the HTTP Content-Type header with something like: property ContentEncoding as
Content-Type: text/html; charset=UTF-8
You must set this property after you set the
ContentTypeor it will overwrite this value.
Sets/gets the 'Content-Encoding:' entity header field in the HTTP request. This acutally referes to the attribute on the property ContentLength as
Read only property that returns the length of the property ContentType as
EntityBody. This can also be obtained by looking at 'http.EntityBody.Size' and is only included for consistancy as this is the value written out for the 'Content-Length:' HTTP request entity header.
Sets/gets the 'Content-Type:' entity header field in the HTTP request. If it is not specified and there is an property CurrentAuthenticationScheme as
EntityBodythen it default to 'text/html'.
A Content-Type specifies the media type of the
EntityBodydata. A Content-Encodingmay be used to indicate any additional content coding applied to the type, usually for the purpose of data compression, that is a property of the resource requested. The default for the content encoding is none.
The authentication scheme that was used to do the most recent authentication. property Date as
Sets/get the 'Date:' header field in the HTTP request. property EntityBody as
The Date general-header field represents the date and time at which the message was originated, having the same semantics as orig-date in RFC 822. The field value is an HTTP-date. A date should only be included if the message contains an entity body and even then it is optional. An example is
Date: Tue, 15 Nov 1994 08:12:31 GMT
When an Entity-Body is included with a message, the data type of that body is determined via the header fields Content-Type and Content- Encoding. These define a two-layer, ordered encoding model. property FollowRedirect as
This is a stream so to insert into this stream use:SeeDo oref.EntityBody.Write("Data into stream")
%AbstractStreamfor more information about streams.
If true then automatically follow redirection requests from the web server. These are signaled by the HTTP status codes of the form 3xx. The default is true for GET and HEAD methods. Otherwise the default is false. property ForceReuseDevice as
If set to true then this will force the %Net.HttpRequest class to reuse the existing connection to the web server if the socket is already open. If there is any error it will be reported to the caller. This has been introduced to support reliable SOAP messaging, so should not normally be set by other code. property From as
The From request-header field, if given, should contain an Internet e-mail address for the human user who controls the requesting user agent. The address should be machine-usable, as defined by mailbox in RFC 822  (as updated by RFC 1123 ): property HTTPVersion as
From = "From" ":" mailbox
An example is:
This header field may be used for logging purposes and as a means for identifying the source of invalid or unwanted requests. It should not be used as an insecure form of access protection. The interpretation of this field is that the request is being performed on behalf of the person given, who accepts responsibility for the method performed. In particular, robot agents should include this header so that the person responsible for running the robot can be contacted if problems occur on the receiving end.
The Internet e-mail address in this field may be separate from the Internet host which issued the request. For example, when a request is passed through a proxy, the original issuer's address should be used.
The HTTP version we should report to the server when making the request. Defaults to '1.1'. property HttpResponse as
This holds the property Https as
%Net.HttpResponseobject which contains all the data that the web server returned from this http request. If you wish to hold onto this after you make another http request or you close the %Net.HttpRequest object you should do:Set response=httprequest.HttpResponse
If not using a proxy server and this is true then it issues a request for an https page rather than the normal http page. This also changes the default property IfModifiedSince as
Porton the target system to 443 the https port.
If using a proxy server use of https is determined by
ProxyHTTPSand Httpscontrols use of a secure SSL connection to the proxy server. In this case the default ProxyPortbecomes, 443 the https port.
The If-Modified-Since request-header field is used with the GET method to make it conditional: if the requested resource has not been modified since the time specified in this field, a copy of the resource will not be returned from the server; instead, a 304 (not modified) response will be returned without any Entity-Body. property InitiateAuthentication as
If-Modified-Since = "If-Modified-Since" ":" HTTP-date
An example of the field is:
If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT
The purpose of this feature is to allow efficient updates of cached information with a minimum amount of transaction overhead.
A client may initiate a connection to the server with an "Authorization" header containing the initial token for the server for a chosen authentication scheme. %Net.HttpRequest is asked to initiate the connection by setting the InitiateAuthentication property to a string which is a single scheme name. This form will bypass the initial 401 error from the server when the client knows that the server will accept the specified scheme. Supported schemes are Negotiate, NTLM and Basic. The default for InitiateAuthentication is Basic. However, if InitiateAuthentication=Basic and Username="", then InitiateAuthentication will be ignored. property LocalInterface as
If specified use this local network interface when opening the TCP/IP socket. This becomes the /BINDTO open parameter. property Location as
Name of item to retrieve from the web server. This is always an absolute path, so if you wish to retrieve the document that is refered to from the URI 'http://machine/cache/index.html' the Location will be property MaxAuthenticationDepth as
All the request methods such as
Gettake the location as the first parameter to make calling the method easier. Note the location does not contain a leading '/' character as this is implicit.
The maximum of authentication round-trips. property NoDefaultContentCharset as
If the property OpenTimeout as
ContentTypeis 'text/' type so we are sending text then if NoDefaultContentCharset= 1 (true), then if there is no ContentCharsetentity body charset we will use iso-8859-1 as the charset to send the entity body in. If the property is 0 (the default) then if there is no entity body charset we will use utf-8 on a unicode Cache, and the system default locale on an 8bit Cache.
If specified is the amount of time we will wait for the TCP/IP connection to open. If not specified then it will default to property Params as
An array of strings containing any query name/value pairs associated with this request. These are added to the request URL when the request is issued. property Password as
The property Port as
Passwordis used to construct the response for SPNEGO or NTLM. If not specfied on Windows, then the current login context will be used for SPNEGO and NTLM.
If you manually set the
Authorizationheader this property will be ignored.
The TCP/IP port number to connect to. This is normally port 80 (the default) unless the web server has been configured to run on an different port. property PostGzip as
If set to true then any data posted to the HTTP server will be gzipped before being sent. You must be sure the remote server can accept gzip data before setting this property. property Pragma as
The Pragma general-header field is used to include implementation- specific directives that may apply to any recipient along the request/response chain. All pragma directives specify optional behavior from the viewpoint of the protocol; however, some systems may require that behavior be consistent with the directives. property ProxyAuthorization as
Pragma = "Pragma" ":" 1#pragma-directive
pragma-directive = "no-cache" | extension-pragma
extension-pragma = token [ "=" word ]
Sets/get the 'Proxy-Authorization:' header field in the Http request. property ProxyHTTPS as
A user agent that wishes to authenticate itself with a proxy-- usually, but not necessarily, after receiving a 407 response--may do so by including an Proxy-Authorization header field with the request. The Proxy-Authorization field value consists of credentials containing the authentication information of the user agent for the realm of the resource being requested.
If using a proxy server and this is true then it issues a request for an https page rather than the normal http page. This allows a proxy server that supports https to support a secure connection from this %Net.HttpRequest class. This also changes the default property ProxyPort as
Porton the target system to 443 the https port. If using a proxy server, use of https is determined by ProxyHTTPS.
The port to connect to the proxy web server on. This is often port 8080 (the default). property ProxyServer as
If you need to request a web page through a proxy server you specify the proxy server host name in this property. If this property is defined then the http request will be directed at this machine and it will forward the request to the machine at property ProxyTunnel as
Locationand return the response.
If true then use the HTTP CONNECT command to establish a tunnel through the proxy to the target HTTP server. The address of the proxy server is taken from property ReadRawMode as
ProxyServerand ProxyPort. If ProxyHttpsis true then once the tunnel is established we will negotiate the SSL connection. The Httpsproperty is ignored as the tunnel establishes a direct connection with the target system.
If true then the body of the response will be read in using RAW mode, ie. with no characterset translate. If false (the default) then it will use the charset specified in the response headers. property RecChars as
Number of bytes received (not characters) property Referer as
The Referer request-header field allows the client to specify, for the server's benefit, the address (URI) of the resource from which the Request-URI was obtained. This allows a server to generate lists of back-links to resources for interest, logging, optimized caching, etc. It also allows obsolete or mistyped links to be traced for maintenance. The Referer field must not be sent if the Request-URI was obtained from a source that does not have its own URI, such as input from the user keyboard. property RequestHeaderCharset as
Referer = "Referer" ":" ( absoluteURI | relativeURI )
If a partial URI is given, it should be interpreted relative to the Request-URI. The URI must not include a fragment.
The character set to send the HTTP request header in. According to the RFC the HTTP header should only contain ASCII characters as the behaviour with characters outside this range is unspecified. This class defaults to using UTF-8 as this leaves all the ASCII characters unchanged. You should never need to change this parameter. property ResponseStream as
Optional property, if set to a stream then this is the stream that will be used to write the data from the web server to rather than constructing a stream of its own. This allows you to specify which type of stream you wish to read the data into, for example a file stream rather than a global stream which is the default. property ReturnGzipResponse as
If we read a gzip response from the web server and this property is true then the stream returned in the property SSLCheckServerIdentity as
HttpResponsewill be a %Stream.FileBinary/CharacterGzip pointing at the gzip file we read. If this property is false we will uncompress the file stream into a %Stream.GlobalBinary/Character stream which involves an extra copy operation. The defaults to faster operation, however you can reset it to 0 for backward compatibility.
When making an SSL connection check the server identity in the server certificate matches the name of the system we are connecting to. This defaults to being on and matches based on the rules layed out in section 3.1 of RFC 2818. property SSLConfiguration as
The name of the activated TLS/SSL configuration to use for https requests. property SSLError as
If request uses an SSL connection and a SSL handshake error has occurred, then SSLError contains text describing the SSL error. property SentChars as
Number of characters sent property Server as
The IP address or machine name of the web server that you wish to connect to. This defaults to 'localhost', i.e. your current machine if not specified. This also sets the 'Host' http header field. property SocketTimeout as
When using a 'keep-alive' HTTP connection this is the maximum time we will keep the socket open for in seconds. If the socket is open for longer than this then it may well have been closed by a firewall etc. so we will start a new connection automatically. Set this property to 0 to disable keep-alive persistent HTTP connections. property TTFC as
Time to first character property TTLC as
Time to last character property Timeout as
The amount of time to wait for a response from the web server before assuming that the web server is not responding. This defaults to 30 seconds. property UserAgent as
The User-Agent request-header field contains information about the user agent originating the request. This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations. Although it is not required, user agents should include this field with requests. The field can contain multiple product tokens and comments identifying the agent and any subproducts which form a significant part of the user agent. By convention, the product tokens are listed in order of their significance for identifying the application. property Username as
User-Agent = "User-Agent" ":" 1*( product | comment )
The default value of this field is:
User-Agent: Mozilla/4.0 (compatible; Cache;)
The property WriteRawMode as
Usernameis used to construct the response for SPNEGO or NTLM. If not specfied on Windows, then the current login context will be used for SPNEGO and NTLM.
Previously setting the Username property, as a side-effect, sent a Basic Authorization header to the server. This behavior will only continue for HTTP 1.0 requests. For HTTP 1.1 requests an unsolicited Authorization header will never be sent unless the InitiateAuthentication property is set. If you manually set the
Authorizationheader this property will be ignored.
If Username'="" and you do not wish %Net.HttpRequest to initiate authentication as Basic, then InitiateAuthentication must be set to "" (for not initiate) or to the initiate scheme.
If true then the body (EntityBody) of the request will be written using RAW mode, ie. with no characterset translate. If false (the default) then it will use the charset specified in the request headers. property WriteTimeout as
Set this to the timeout to use when writing to the remote HTTP server. The default is -1 means it will wait indefinitely for the remove server to accept the written data, change it to another value to specify the timeout in seconds. The minimum value accepted is 2 seconds.
This is a Set accessor method for the classmethod CheckSSLCN(server As
Verify the SSL server we are connected to has the correct certificate. In order to call this method the SSL/TLS connection must be active and must be the current device, also at least one message must have been sent down the socket in order for the SSL/TLS handshake to have taken place so we have the certificate. If Cache is acting as a client and makes a request to a server by IP address, the server's certificate need only contain the matching IP address in the Subject DN for us to verify the server's identity; the subjectAltName extension does not have to be present. This is a slight difference to the wording of RFC 2818. method CountFormData(name As
Returns the number of values associated with the name-value pair name. method CountParam(name As
Returns the number of values associated with the name-value pair name. method DeleteCookie(name As
Remove a cookie from this %Net.HttpRequest object. The name, path, and domain must match the ones used to insert the cookie in the first place. method DeleteFormData(name As
Removes this key from the collection method DeleteParam(name As
Removes this key from the collection method Get(location As
Issue the Http 'get' request, this will cause the web server to return the page requested. If this completes correctly the response to this request will be in the method GetFullCookieList(ByRef cookies) as
HttpResponse. The location is the url to request, e.g. '/test.html'. This can contain parameters which are assumed to be already URL escaped, e.g. '/test.html?PARAM=%25VALUE' sets PARAM to %VALUE. It can also be a full url with the server and optional port number e.g. 'http://server.com/test.html'. If test is 1 then instead of connecting to a remote machine it will just output what it would have send to the web server to the current device, if test is 2 then it will output the response to the current device after the Get. This can be used to check that it will send what you are expecting. This calls Resetautomatically after reading the response, except in test=1 mode or if reset=0.
Passed cookies by reference and return an array of cookies so you can see which cookies have been set, or persist them into a database if wished. The return from this method is the number of cookies that are in the array. The format of the array is: method GetHeader(name As
Allows you to obtain the current value for any of the Http headers that have been set. method GetParam(name As
Returns the value of name-value pair name. method Head(location As
If more than one value is associated with name, then the values are subscripted using index starting with 1. index
If the name-value pair is not defined, returns the value of default.
Issue the Http 'head' request, this will cause the web server to just return the header of the response and none of the body. If this completes correctly the response to this request will be in the classmethod HorologToRFCDateTime(horolog As
HttpResponse. The location is the url to request, e.g. '/test.html'. This can contain parameters which are assumed to be already URL escaped, e.g. '/test.html?PARAM=%25VALUE' sets PARAM to %VALUE. It can also be a full url with the server and optional port number e.g. 'http://server.com/test.html'. If test is 1 then instead of connecting to a remote machine it will just output what it would have send to the web server to the current device, if test is 2 then it will output the response to the current device after the Head. This can be used to check that it will send what you are expecting. This calls Resetautomatically after reading the response, except in test mode or if reset=0.
Helper function to convert from a $H value into the date/time format suitable for use in an HTTP request. It converts the timezone from the current $H value to the GMT timezone as well as outputting the correct format. method InsertCookie(name As
Add a cookie to this %Net.HttpRequest object. The name is the name of the cookie, the value is the value this cookie is set to. The path is the path to store the cookie under, it will only be output for this path and any subpaths on the server. The domain is the name of the machine the cookie is downloaded from. Then expires is the date/time when this cookie will expire. The secure parameter is if the cookie should be send over secure channels only, i.e. https. method InsertFormData(name As
Set the value of name-value pair name to value. method InsertParam(name As
If more than one value is associated with name, then the values are subscripted using index starting with 1. The value inserted can be either a %String or a stream.
Set the value of name-value pair name to value. method IsFormDataDefined(name As
If more than one value is associated with name, then the values are subscripted using index starting with 1.
Tests if a name-value pair name is defined. method IsParamDefined(name As
Tests if a name-value pair name is defined. classmethod LoadPublicSuffix(filename As
Load in the filename which is the downloaded public suffix list from publicsuffix.org classmethod MatchSuffix(domain As
Determine if this domain is a public suffix (publicsuffix.org) or not, return true if it is and false if it is not. method NextFormData(name As
Returns the next key in sequence method NextParam(name As
Returns the next key in sequence method OutputFormData(stream As
method OutputHeaders() as
Output the full list of Http headers that will be sent to the machine method OutputParams(params As
Serverfor the item Locationto the current device. This does not include any entity headers associated with the EntityBody.
Output all the parameters to the current device. The params is any parameters that you wish to be included in the parameters output, these are assumed to already be URL escaped. classmethod ParseContent(in As
Parse a name=value,name2="newvalue" type list method Post(location As
Issue the Http 'post' request, this is used to send data to the web server such as the results of a form, or upload a file. If this completes correctly the response to this request will be in the method Put(location As
HttpResponse. The location is the url to request, e.g. '/test.html'. This can contain parameters which are assumed to be already URL escaped, e.g. '/test.html?PARAM=%25VALUE' sets PARAM to %VALUE. It can also be a full url with the server and optional port number e.g. 'http://server.com/test.html'. If test is 1 then instead of connecting to a remote machine it will just output what it would have send to the web server to the current device, if test is 2 then it will output the response to the current device after the Post. This can be used to check that it will send what you are expecting. This calls Resetautomatically after reading the response, except in test=1 mode or if reset=0.
Issue the Http 'put' request, this is used to upload data to the web server, it is not used that often. If this completes correctly the response to this request will be in the method RemoveHeader(name As
HttpResponse. The location is the url to request, e.g. '/test.html'. This can contain parameters which are assumed to be already URL escaped, e.g. '/test.html?PARAM=%25VALUE' sets PARAM to %VALUE. It can also be a full url with the server and optional port number e.g. 'http://server.com/test.html'. If test is 1 then instead of connecting to a remote machine it will just output what it would have send to the web server to the current device, if test is 2 then it will output the response to the current device after the Put. This can be used to check that it will send what you are expecting. This calls Resetautomatically after reading the response, except in test=1 mode or if reset=0.
Remove a Http header value method Reset() as
Reset the %Net.HttpRequest class so that it can issue another request. This is much faster than closing this object and creating a new %Net.HttpRequest object. This also moves the value of method ReturnHeaders() as
Locationto Referer. It is called by the Sendautomatically after issuing a request.
Return the full list of Http headers that will be sent to the machine method ReturnParams() as
Serverfor the item Location. This does not include any entity headers associated with the EntityBody.
Return the list of parameters that this request will output. method Send(type As
Method that actually sends HTTP request to the server. This is normally called from method SetHeader(name As
Get, Post, Put, Headbut if you wish to use a different HTTP verb you can call this directly passing the type as the verb you require.
Add a header to the http request. Most of the normal headers that you may want are covered by properties of this class such as method SetParam(name As
Datehowever if you wish to add a non standard header then call this. For example:Note that headers such as Content-Type, Content-Encoding, and Content-Length are part of the entity body rather than the http main headers and as such as forwarded to theDo httprequest.SetHeader("MyHeader","Data to display") ContentType, ContentEncodingand trying to set the Content-Length is just ignored as this is a read only property. Also any attempt to set the 'Connection' header is ignored at this request class does not support persistent connections.
Set the value of the name-value pair name. The optional index is used to associate multiple value with a single name.