Securing Web services

For Your Eyes Only

Web services are present everywhere nowadays. Functionality and data is being exposed more rapidly than ever, by more and more platforms, applications … One of the more challenging aspects of the ever increasing service-orientation is dealing with security properly: the access to all this data and functionality should be strictly governed, for external services (services exposed outside the organization) but also for intra-organizational services.

In this BLOG-post, I will describe how I securely exposed an unsecured service, using Open Source tools.

Contents

1 Securing Web services
1.1 Introduction
1.2 Background
1.3 Environment
1.4 Cryptographic Intermezzo
1.5 Providing the Service
1.6 Service Mediation
2 Keys
2.1 Your Own Personal CA
2.2 Keystores
2.2.1 Client key store
2.2.2 Service key store
2.2.3 Generating Certification Requests
2.3 Self-certifying your keys
2.3.1 Converting keys
2.4 Updating the key stores
2.4.1 Importing the CA certificate
2.4.2 Importing Certificate Request Replies
2.4.3 Importing the public keys
3 Securing the service
3.1 Security Revisited
3.2 Providing a Secured Service
3.2.1 Groundwork – Providing Access to the key store
3.2.2 Securing the service (finally)
3.3 Consuming a Secured Service
3.3.1 SOAPUI
3.3.2 Security – outbound first
3.3.3 Security – inbound
3.3.4 Applying Security Configurations
3.3.5 Invoking the Secured Service
3.3.6 Almost there
3.4 Inspecting the Encrypted Messages
3.4.1 Change the endpoint
3.4.2 Message inspection
4 Concluding Remarks
5 References

1 Securing Web services

1.1 Introduction

In this BLOG-post, I will describe how I securely exposed an unsecured service, using Open Source tools.

1.2 Background

For a new project, we have a legacy application from which we want to expose several functionalities as web services. Apart from exposing these legacy functionalities as web services, which might be a tour-de-force in itself depending on the application and the integration platform, we also want to control access to exposed services to a select group of service consumers (under our control).

General security aspects we may want to consider in this scenario, like:

For our typical scenario, we aim to address the identity and authorization aspects mainly within the legacy application itself, leaving the authentication, integrity and confidentiality to be addressed by the middleware.

1.3 Environment

To reduce the financial implications of this solution to customers, I decided to restrict myself to exclusively use Open Source tools or tools otherwise available without licensing fee. Furthermore, since I do not like cluttering my laptop with all kinds of installation for development and/or evaluation, I decided to setup the environment inside a virtual machine.

In order to emulate a “real” development experience, I like to run the development/testing tools from outside the VM itself, requiring some additional network setup in the form of port forwarding.

Ingredients:

All software has been installed inside the guest (VM image), with the exception of the virtualization software itself and SOAPUI: these have been installed on the host computer (i.e. my laptop).

VM Configuration

Installing the Virtual Machine running Oracle Enterprise Linux is quite easy and well-documented on the web. My Virtual Machine is configured to use 4 GB, a dynamically allocated disk of 20GB, NAT with port forwarding and shared folders to allow easy file exchange between my host machine (laptop) en the guest OS (Virtual Machine).  I have setup several forwarding rules for my VM in order to access components within my VM from my host machine directly. As you can see, I have forwarded most ports on my host system to the identical port number on the VM (except the SSH port, since port numbers < 1024 are considered “privileged” ports on *NIX systems). You can configure the port forwarding on the networking tab of Virtual Box, in the “Advanced” section:

Port forwarding

Since I will name my guest “oel6”, I also created an alias “oel6” for my host machine for 127.0.0.1; on windows this is done in the file C:\Windows\System32\drivers\etc\host, assuming you have your Windows OS installed in C:\Windows. Note that is also on 64-bits Windows versions it is still the System32 directory!

Another caveat is that you must install the Virtual Box Guest Additions in your Linux VM to be able to access the shared folder functionality.

Installation

For the installations, I have created an unprivileged user (milco) in my Linux system and provided this user access to the /opt file system. From this account, the installations are usually as straightforward as extracting an archive file into the desired directory. However, some installations require some additional setup, usually in the form of environment variables to be set.

Being an old-fashioned ‘90s UNIX hacker, I have only just recently discovered that on Linux platforms site specific customizations can be provided as scripts inside the /etc/profile.d/ directory, since all files are being sourced by the system-wide /etc/profile script, so I have used this method to define the JAVA_HOME and the ANT_HOME environment-variables amongst other things:

Disabling the Firewall

By default the Linux kernel firewall is switched on after installing Oracle Enterprise Linux: this module will filter the incoming IP-traffic and allow or disallow based upon the rules defined in the tables. To make sure that traffic from outside the VM can reach the middleware components, we take the easy approach and completely disable the firewall. Of course, this should never be done in a production situation!

Run the follow command from a privileged account to switch the ipfilters component off for all run-levels:

chkconfig –level 012345 iptables off

Integration Scenario

The next diagram depicts our simple integration scenario: we will expose a secured service from the service bus, routing the message payload to an unsecured service implementation.

SOAPUI will have the role of the service consumer, consuming the secured service. As the service bus implementation, we have selected the WSO2 Enterprise Service Bus, the high performance open source Service Bus provided by WSO2 (running more than 109 transactions per day for eBay). For the service implementation we have chosen on of the sample Axis2 services provided as part of the WSO2 ESB install, the Axis2 SimpleStockQuoteService.

For demonstration purposes, we will restrict ourselves to a single operation, getSimpleQuote. This operation will return some (randomly generated) stock information, given a stock ticker symbol. The screenshot below contains an example invocation of the unsecured Axis2 service implementation:

So, we want to establish secure communication between the service consumer (some client) and the service provider, which we assume to be a legacy application that cannot be easily changed. Therefore, we will expose a secured service endpoint from the Service Bus to the outside world, and we will require that the service only be invoked in a secure manner: data must be both encrypted (to ensure that the message cannot be seen by unauthorized intermediaries or eavesdroppers) and digitally signed (confirming the initial sender and making sure that the message has not been altered in any way).

Note that the secure communication channel (encryption and digital signing) pertains to the request as well as the response message!

1.4 Cryptographic Intermezzo

Cryptography basics

In the next steps, we will use public key (asymmetric) encryption [PKE]. This mechanism relies on two different keys, a public and a private key. These two keys are mathematically related. One key will encrypt the message from plain text to encrypted text (cipher text); the other key will decrypt the encrypted text.

Furthermore, we will require a proof of integrity of the message. Usually, this is implemented as a message digest that is digitally signed and attached to the message. A message digest is the calculation of the property of the message; where the function should produce large changes in output for small variations of input and it should be infeasible to invert the digest process (e.g. generating the message from the digest should not be possible).

The “digest” will be signed by the private key of the sender, to vouch for its correctness.

1.5 Providing the Service

We’ll start off by exposing the Web Service using the Service Bus; the Service Bus is a compound design pattern for SOA, which essentially provides a unified point for service access and security, as well as transformation capabilities (service style – SOAP vs. REST, data structures, transports etc.) [ERL-SOADP].

We start off by making sure that the endpoint exposed by the actual service implementation in Axis2 will not refer to the IP address, but to the hostname instead. Therefore, we should set the hostname parameter in the axis2.xml configuration file (found at $ESB_HOME/samples/axis2Server/repository/conf/axis2.xml, where $ESB_HOME refers to the installation directory for the WSO2 ESB):

Building and deploying the service to the Axis2-server is as easy as just running ant in the correct directory (assuming you’ve correctly set up the $ANT_HOME en $JAVA_HOME environment variables):

Next, we need to start the Axis2 server ($ESB_HOME/samples/axis2Server/axis2server.sh) and the WSO2 ESB ($ESB_HOME/bin/wso2server.sh) and configure the service mediation using the service bus.

1.6 Service Mediation

In this simple scenario, there is no real service mediation. We will just expose the service on a different address and (later on) restrict access to secure access only. For this purpose, we will simply create a Pass-Through Proxy, exposing a WSDL that is already provided in the samples repository:

Fire up your browser and point it to the ESB Admin Console URL (https://localhost:9443/carbon), log in using the default credentials (admin/admin). You may ignore any security exceptions regarding the SSL certificate.

In the Main Section, choose “Add Proxy Service”:

From the proxy service templates offered, choose the simple “Pass Through Proxy” Template; at this stage, we have no need to add any other logic.

Next, we need to specify the proxy name, the actual endpoint of the implementation (i.e. the Axis2 Service) and the WSDL we want to expose (“publish”) to the outside world.

Select http://localhost:9000/services/SimpleStockQuoteService?wsdl as the target endpoint, the endpoint we are exposing is defined in the sample_proxy_1.wsdl file in the sample repository. After confirming by pressing the “Create” button, the proxy service will be deployed. You can test the deployed proxy service if you like to, either by obtaining its (exposed) WSDL or using the “TryIt” option in the WSO2 ESB Console itself.

2 Keys

For security, we are going to generate two private-public key pairs: one for the ESB’s service and one pair for the client, so they can share their public keys. However, these keys must be certified in order to be usable. You can think of certification as binding the key to the claimed identity. Normally, this would involve generating certification requests from your keys and having your public keys certified by a CA (Certification Authority, [CA]). For our test purposes, it is sufficient to certify the keys ourselves to save time and cost.

2.1 Your Own Personal CA

The OpenSSL [OPENSSL] toolkit allows you to generate your own key to certify keys. For OpenSSL to work, it needs access to a configuration file (openssl.cnf) en you need to create some empty files for housekeeping purposes (serial and index.txt), all in a directory of your choice.

For the configuration file, I have used a file I obtained from the Internet, with some slight modifications.

OK, now we are all set to generate our own Certification Authority keyset: it’s going to be a military grade X.509 key, 2048 bits RSA. Accept the defaults for the questions asked, or supply your own data.

A word of caution regarding the passphrases and keys: do NOT forget the values you entered, otherwise you are lost without your keys and need to obtain new ones!

2.2 Keystores

Next up are the keys for the service and the client themselves; these keys are going to be stored in a Java Keystore file, and we will be using the JDK-supplied keytool to generate these keys.

2.2.1 Client key store

We’ll generate a 1024-bit RSA-key, with the alias “serviceclient” and protect both the key store and the key itself with different passwords (I used “serviceclient” and “serviceclientkeypassword” respectively):

You can view the contents of the key store by using the command keytool –list –keystore serviceclient.jks, optionally switching on verbose output by adding the –v switch:

2.2.2 Service key store

For the service key store, we’ll use the same key parameters, alias is now “serviceprovider” and also protect both the key store and the key itself (serviceprovider and serviceproviderkeypassword):

2.2.3 Generating Certification Requests

After generating both the service and the client’s keys, it is time to generate the requests to certify the keys; normally, these would be certified by a regular CA, but for this example we are going to do these ourselves by using the CA keys we generated earlier.

Generate the certificate requests, serviceclient.cert.req (for the service client) and serviceprovider.cert.req (for the service provider); we’ll supply the key store password on the command line and will be prompted to enter the key password:

2.3 Self-certifying your keys

After generating the certificate requests, you can sign your keys using OpenSSL. After this process, you’ve got two keys that are certified by the CA (you) as belonging to the claimed organizations (service client and service provider).

2.3.1 Converting keys

We have obtained our certified keys. However, their format is plaintext and we need a binary format to import them into our Java key stores. Again, OpenSSL comes to the rescue by offering a key conversion tool to convert our keys (*.pem) into a binary (DER) format:

2.4 Updating the key stores

After obtaining the key certificates in the correct format, it is time to import them into the key stores.

2.4.1 Importing the CA certificate

Now this part is a little tricky: we first need to make certain that we trust the CA before importing any of the keys certified by the CA. We can do this by importing the CA’s certificate before any of the other certificates into both the service client’s and the service provider’s key stores; in both cases you should obviously answer ‘y’ to trust the certificate you are importing:

2.4.2 Importing Certificate Request Replies

Continuing, we need to import the certificate requests’ replies into the key stores. We import the reply for the service client into the service client’s key store and the reply for the service provider into the service provider’s key store … it’s that simple! Again, you will be prompted for the key passwords, since we have only supplied the key store passwords on the command line:

2.4.3 Importing the public keys

Finally we have arrived at last step of the key generation process: importing the public keys into the key stores, crosswise. This will enable the service provider to use the service client’s public key and make sure the service provider’s public key is available to the service client, where all keys are CA certified (note that since these are the public keys, you will only need to supply the key store password !).

Both key stores now contain three entries: their own private key, the CA’s key and the partner’s public key:

3 Securing the service

OK, the keys are certified and ready to be used now expose the critical SimpleStockQuoteService in a secured way. As you may recall, we want to both encrypt and sign the messages flowing between the service provider (for the outside world, this is the ESB) and the service consumer.

3.1 Security Revisited

As the service consumer, we need to securely invoke the service. In our scenario, we have to sign and encrypt (parts of) the message. By default, the SOAP message body is affected, unless we specify specific elements to be encrypted/signed.

The service consumer will need to encrypt the message using the service provider’s public key, so that the message can only be decrypted by the service provider (using its private key). Furthermore, to sign the message the service consumer needs to calculate the message’s hash (digest) and encrypt the resulting value using the consumer’s private key. This will enable the partner (service provider) to establish without doubt that the message has been sent by the service consumer and has not been altered in transit. To prevent “Replay attacks”, one or more timestamps are included in the message (and the digest). For any replies sent from the service provider to service consumer the same process is used, this time the message is encrypted with the service consumer´s private key, and signed using the service provider’s private key.

3.2 Providing a Secured Service

3.2.1 Groundwork – Providing Access to the key store

First, we need to make the service provider´s key store accessible to the ESB. To do this, we can place the serviceprovider.jks store in a secured directory (to prevent unauthorized access) and configure it from the ESB Console’s Configure > Key Stores menu option, followed by Add New Key Store.

Provide the path to the key store, key store password, alias for the private key and select “JKS” as key store type; in the second step you need to provide the private key’s password.

You should see a message confirming that the key store was successfully imported.

3.2.2 Securing the service (finally)

From the Services > List on the Main tab, select the “SimpleStockQuoteServiceProxy” service:

You can see that this service is (still) insecure from the Open Lock icon in the Quality of Service Configuration section. Click on the “Security” link to enable security for the service.

We will use a predefined security policy that requires signed and encrypted message exchanges; if your required security policy is not provided out of the box, you can create your own:

Click “Next” configure the security details: both our private key and our trusted keys reside in the same key store:

“Finish” to secure the service: a confirmation will be shown, and the security icon will change to a closed lock (QoS-section):

3.3 Consuming a Secured Service

The last part in this scenario involves invoking a secured service: we will use SOAPUI, a Services testing tool available freely from SmartBear [SOAPUI].

3.3.1 SOAPUI

Create a new SOAPUI project and provide the secured service’s WSDL (you can use the WSDL 1.1 link from the ESB Service List; WSDL 2.0 is not yet supported in SOAPUI). Uncheck the “Create Requests” checkbox; we will be invoking a single operation only:

Create a new request for the GetSimpleQuote-operation (I am using the SOAP 1.1 binding):

If you enter a stock symbol and just send the unsecure message, the request will eventually time out since the Service Bus will not accept this message.

3.3.2 Security – outbound first

Open the project view to configure the security.

Navigate to WS-Security Configurations > Keystores and add a key store using the plus-icon. Provide the path to the serviceclient.jks key store and provide the key store’s password (service client); this is where the shared folder functionality between VM and host comes in handy:

Now we need to define what steps are to be executed in the message exchange for the request message. We will create a new Outgoing WS-Security Configuration:

For this configuration, we will add the following elements:

- A timestamp, with TTL=300000 (request remains valid for 5 min = 300000 ms; you will see in the output that I am wrong on this assumption … the TTL is specified in seconds and the checkbox does not relate to the TTL).

- Signature using the service client’s private key (so you must supply the private key password):

- And finally: Encryption, using the service providers PUBLIC key (so no password is needed for the key):

3.3.3 Security – inbound

Also for the responses (inbound messages) we need to define the security options, but this time this is quite straight-forward since we only need to specify the key store that we want to use.

Create a new Incoming WS-Security Configuration and configure the key store to be used. The password you need to supply is the password for the private key entry (used for decryption – you specified the client’s key store password in the Key stores configuration already!), so this is “serviceclientkeypassword”:

3.3.4 Applying Security Configurations

The final step in the preparation is to apply the security configurations to the request; open the request for the getSimpleQuote operation and click on the “Aut” link in the left bottom of the Request tab: here you specify which configuration will be applied when invoking your web service operation, for request and response messages.

3.3.5 Invoking the Secured Service

The secured service can now be invoked using the triangular (Submit) icon on the request. If you have a standard JDK/JRE installation, the request will fail due to export restrictions placed on encryption technology. By default, the maximum key length for almost all encryption algorithms is restricted. The exception you will most likely run into is a “java.security.InvalidKeyException”, that will tell you “Illegal key size or default parameters”.

In order to resolve this problem, you have to download the Java Unlimited Strength Jurisdiction Policy Files from Oracle, for your JDE/JRE version (if this is allowed in your country). Download the files, extract the JAR-files and replace the original JAR-file in $JAVA_HOME/jre/lib/security/. Note that you probably need to do this for both the server (VM) and the host that runs SOAPUI. (You should probably identify which Java install is used for SOAPUI, using Help > System Properties; most likely it is the embedded JRE; this will also identify the java runtime environment version you are using):

3.3.6 Almost there

Almost, first restart both the ESB-server and the SOAPUI-client, to make sure they pick up the modified policy files. Now, send the request again and verify that there is a plain text response: you’ve done it!

You can verify that you are receiving a plain text SOAP Body element, together with a lot of SOAP Header elements that are used by the service to provide signature and encryption information.

On the response tab, you can inspect the WSS processing:

3.4 Inspecting the Encrypted Messages

By using a TCP-tunnel, you can inspect the messages being exchanged. One particularly nice tool I recently discover is the Membrane SOAP/HTTP Monitor [MEMB]. Simply run the membrane-monitor executable from the unpacked distribution and your almost ready to sniff the messages … the only thing that needs to be configured in the Membrane Monitor is the tunnel itself, i.e. what port to listen on for messages and what port to send the messages to.

The destination port is easy; this should be the port number that the ESB is listening on (8280), since that is the final destination for your message. The port Membrane Monitor itself runs on can be any unprivileged port, but to establish a suggested relation I like to offset this by a couple of hundreds and have chosen 8480:

Specify to listen on port 8480 for incoming traffic:

And redirect the traffic to port 8280 (on the localhost):

3.4.1 Change the endpoint

To make sure that the request message is intercepted by the Membrane Monitor, we need to change the endpoint in SOAPUI to send the message to the correct port number. The current endpoint sends the message to the port number that the ESB is listening on (8280), but now we want to inspect the message and therefore we need to pass it from SOAPUI to the Membrane Monitor’s port (8480): change the port number in the current endpoint to 8480.

3.4.2 Message inspection

Invoke the service: the request message is first delivered to the Membrane Monitor (listening on localhost:8480), from there it is forwarded to the ESB (localhost:8280). Since this is a synchronous web service call, the response will travel the opposite direction as the connection(s) stay open. You should verify that you’ve received a response message in SOAPUI that can be properly processed (i.e. having a valid signature and can be decrypted).

Now, if you go back to your VM and inspect the messages intercepted by the “message sniffer”. As you can see, the request message has an encrypted SOAP-body, which cannot be read – apart from the encryption declarations:

If you closely inspect the message, you will notice that there is a “Created” and an “Expires” timestamp. Apparently, the TTL should be specified as seconds (not milliseconds), as the difference between the two is approximately 3.5 days ( 300,000 seconds ? 3.47 days) – I stand corrected!

Also the response message as intercepted is gibberish cipher text:

4 Concluding Remarks

Using Open Source and freely available software, it is easy to expose secured endpoints for unsecure web services or endpoints exposed by legacy applications. One reason for using a service bus for this scenario is that you may not be able to enable security for the unsecure web service itself, e.g. when it is a third-party web service that you do not have the source code for. Or you may now want to expose this web service outside of its domain (or externally) and there are other (internal) consumers that are allowed to invoke this service unsecured. A third reason may be that you want to virtualize your services, or provide a centralized and secure access.

One point to note is that we do still expose the unsecure endpoint, if you still forward port number 9000 in your advanced NAT configuration. This can be easily resolved by removing this, making it not accessible in this VM scenario. In a real-life scenario, a firewall should do the trick to allow only traffic to selected ports (one of the short-cuts I have allowed myself is disabling the firewall on the guest OS).

Adding additional service consumers should be easy now; each service consumer will need its own key store, with its private key and the public key of the service provider, whereas the service provider’s key store should be augmented with the service client’s public key. It goes without saying that the keys should be certified …

5 References

Citation Reference
[ANT] http://ant.apache.org/
[CA] http://en.wikipedia.org/wiki/Certificate_authority
[ERL-SOADP] SOA Design Patterns – Thomas Erl et al., Pearson Education, Boston (2009)
[JDK] http://www.oracle.com/technetwork/java/javase/downloads/index.html
[MEMB] http://www.membrane-soa.org/soap-monitor/
[OEL] http://www.oracle.com/technetwork/server-storage/linux/downloads/index.html
[OPENSSL] http://www.openssl.org/
[OVB] https://www.virtualbox.org/wiki/Downloads
[PKE] http://en.wikipedia.org/wiki/Public-key_encryption
[SOAPUI] http://www.soapui.org/
[WSO2ESB] http://wso2.com/products/enterprise-service-bus/