HTTPS Client using X.509 Certificate

Hypertext Transfer Protocol Secure (HTTPS) is a communication protocol for secure communication. HTTPS is widely used on the internet to secure the communication on public websites. Technically it is not a protocol itself; rather, it is the result of layering the HTTP over SSL/TSL protocol to add the security capabilities of SSL/TSL to the standard HTTP protocol. More formally it is HTTP Over TLS according to RFC 2818 on IETF.

All the communication between the client and the server is encrypted using public key certificates. On connection request from the client, HTTPS server presents a certificate to the browser with its public key and the subsequent communication between the server and the client is encrypted using the public key in the certificate.

Public-key infrastructure (PKI) is especially suitable for web because it still provides some protection even if only one side of the communication link is authenticated. As long as the server is authenticated, any client can download the public key certificate from the server and use the public key to encrypt the data before sending it over to the server. That however means it is the responsibility of the client to examine the public key certificate and make sure that the certificate is valid for that server, i.e., the server is actually who it says he is.

Normally every certificate is signed by some authority to mark that the certificate is trusted and can be used for secure communication. Most websites use well know certificate authorities (e.g. VeriSign/Microsoft/etc.) to sign the certificate for them. Most of the browsers have a list of well know certificate authorities. When a server presents a certificate signed by one of the certificate authorities in the browser’s list, the browser use them to establish a secure connection with the server. However, when the signatory is not in the list of browser then they present a message to the user that the certificate is not signed by a trusted authority and let the users decide whether they want to trust the server or not.

Most of the browsers come equipped with this functionality. When you connect to a website using HTTPS URLs the browsers take care of the details for the users. If the certificate presented by the server is signed by one of the well-known Certificate Authority then the browsers simply use that certificate and in case of a self-signed certificate the browsers ask users to validate and accept the certificate before establishing the data communication.

The bottom line is that the certificate must be marked as trusted before it can be used to establish a connection with the server.

• Using Certificate to open HTTPS Connection

Java uses the keystore to achieve this functionality. Any certificate that exists in the keystore is considered trusted and can be used to establish a connection with the server. To create a HTTPS connection programmatically, we first need to add the certificate to the keystore. A very simple way to get the certificate is to point your browser to the https:// url and ask to view the certificate and then download it.

I used FireFox to get the certificate.

• Go to the https url, because it is self-signed, FireFox will display the warning that the connection is untrusted.
• Go to I Understand the Risk and click Add Exception.
• That will open the Add Exception dialog, click on the View button to see the certificate details.
• This will open the certificate details for you. Click on the Details Tab to see the details of the certificate.
• On this tab you have an option to Export the certificate.

When you click Export, it will open the save file dialogue. Save the certificate to an appropriate location on your disk. The certificate will be saved with either .crt or .perm extension.

Once you have the certificate then the next step will be to add that in your keystore. You can do it by using the keytool command in java as follows.

keytool -import -keystore yourKyeStore.jks -file YourCert.crt

This command will create a keystore for you and import the certificate to your keystore. If the keystore does not exist then it will create a keystore for you. For an existing keystore, it will ask for the password of your keystore, and for a new keystore it will ask for a password for the keystore and then ask again to confirm the password and on confirmation create a new keystore with the provided password.

At this point it will show you the detials of the certificate and ask you if you trust the certificate.

C:\temp\certs>keytool -import -keystore store.jsk -file cert.crt
Enter keystore password:
Re-enter new password:

Owner: CN=www.certificateserverurl.com, OU=Sun GlassFish Enterprise Server, 
 O=Sun Microsystems, L=Santa Clara, ST=California, C=US
Issuer: CN=www.certificateserverurl.com, OU=Sun GlassFish Enterprise Server, 
 O=Sun Microsystems, L=Santa Clara, ST=California, C=US
Serial number: 5013bd9b
Valid from: Sat Jul 28 11:23:23 BST 2012 until: Tue Jul 26 11:23:23 BST 2022
Certificate fingerprints:
         MD5:  90:03:8C:BA:32:1F:AD:96:40:CE:49:1D:A3:A3:F6:72
         SHA1: 8D:AE:25:8F:9C:3A:70:81:55:03:5E:B7:92:D2:0A:E6:CB:99:A3:59
         Signature algorithm name: SHA1withRSA
         Version: 3

Extensions:

#1: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: F9 A7 36 D2 9B 4D 10 68   0F 22 2F 31 16 39 59 1A  ..6..M.h."/1.9Y.
0010: 65 70 F6 56                                        ep.V
]
]

Trust this certificate? [no]:  y
Certificate was added to keystore

C:\temp\certs>

Enter "y" on this prompt to add the certificate to your keystore.

Once you have the certificate added to the JVM Truststore, all you need to do is to tell your Java Program to use the truststore before opening a HTTPS connection. This is how you do it:

 System.setProperty("javax.net.ssl.trustStore", keystore);
 System.setProperty("javax.net.ssl.keyStorePassword", "changeit");
 
 //con = openHttpsConnection(queryString);
 URL url = new URL(queryString);
 con = (HttpsURLConnection)url.openConnection(); 
 

Java adds all the trusted certificates in a keystore. Here you are specifying the keystore and the password for the keystore to enable java to search for an appropriate certificate from this store. When a connection request is made the server will present the certificate to the java client. The java client will look into this keystore to see if we already have this certificate added as a trusted certificate in the keystore. If the certificate is found then a HTTPS connection can be established.

Supposing you have a user search service running on a HTTPS server. Below is the code you will use to call that secure service to search for registered users in your server.

package user;

import java.net.URL;

import javax.net.ssl.HttpsURLConnection;

import user.xml.UserXMLHandler;
import user.xml.generated.Users;

public class TestUserServlet {
 
 private static final String strUrl = "https://www.yourserver.co.uk/users/";
 private static final String keystore = "C:\\temp\\certs\\store.jks";
 
 private static final String find = "profile?";
 
 private static final String FNAM = "firstName";
 private static final String LNAM = "lastName";
 
 public static void main(String s[]) throws Exception{
  
  Users users = null;
  
  System.out.println(strUrl);
  
  TestUserServlet test = new TestUserServlet();
  
  users = test.doFindUser("raza", "abidi");
  System.out.println(users.toString());
  
 }
 
 private Users doFindUser(String firstName, String lastName) throws Exception{
  
  String queryString = strUrl + find + FNAM + "=" + firstName + "&" + LNAM + "=" + lastName;
  
  HttpsURLConnection con = null;
  
  try{
   
   // Set the keystore to use
   System.setProperty("javax.net.ssl.trustStore", keystore);
   System.setProperty("javax.net.ssl.keyStorePassword", "changeit");
   
   URL url = new URL(queryString);
   con = (HttpsURLConnection)url.openConnection();
  }catch(Exception e){
   e.printStackTrace();
   if(con!=null)
    con.disconnect();
   
   throw e;
  }
  
  if(con==null){
   throw new Exception ("Failed to open HTTPS Conenction");
  }
  
  Users users = null;
  try{
   users = new UserXMLHandler().getUsersXML(con.getInputStream());
  }finally{
   con.disconnect();
  }
  return users;
 }
}

All we are doing here is telling java which keystore to use to find a certificate before trying to open a connection to the secure server. Java will look into the keystore and will use the most appropriate certificate from the list of certificates in the keystore. The certificate already has information about the issuing server and java will use that information to find out which certificate to use to establish a secure connection with the server.

• HTTPS Connection without Certificate

I would strongly advise against using this approach when connecting to a secure server. Getting a certificate from the secure server and installing it in the JVM keystore is a pretty trivial task and there is no need to bypass that process otherwise what is the point of implementing a secure server connection. However, sometimes, most probably in your test environments, you may need to connect to the HTTPS server without using the certificate. On these occasions the workaround is to provide your own implementation of the TrustManager and override the security methods to trust any given certificate.

Even in this scenario, all the communication between server and client will still be encrypted, the only problem is that you blindly trust every certificate as it is and that opens the system vulnerable to security breaches. That is the reason why you should avoid bypassing the keystore approach. However, for odd occasions when it is needed, here is how you do it.

package user;

import java.net.URL;
import java.security.cert.CertificateException;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

import user.xml.UserXMLHandler;
import user.xml.generated.Users;

public class TestUserServlet {

 private static final String strUrl = "https://www.yourserver.co.uk/users/";

 private static final String find = "profile?";

 private static final String FNAM = "firstName";
 private static final String LNAM = "lastName";


 public static void main(String s[]) throws Exception{

  Users users = null;

  System.out.println(strUrl);

  TestUserServlet test = new TestUserServlet();

  users = test.doFindUser("raza", "abidi");
  System.out.println(users.toString());

 }

 private Users doFindUser(String firstName, String lastName) throws Exception{

  String queryString = strUrl + find + FNAM + "=" + firstName + "&" + LNAM + "=" + lastName;

  HttpsURLConnection con = null;

  try{
   con = openHttpsConnection(queryString);
  }catch(Exception e){
   e.printStackTrace();
   if(con!=null)
    con.disconnect();

   throw e;
  }

  if(con==null){
   throw new Exception ("Failed to open HTTPS Conenction");
  }

  Users users = null;
  try{
   users = new UserXMLHandler().getUsersXML(con.getInputStream());
  }finally{
   con.disconnect();
  }
  return users;
 }

 // Creating our own implementation of an all trusting trust manager
 private HttpsURLConnection openHttpsConnection(String queryString) throws Exception{

  // Create a trust manager that does not validate certificate chains
  TrustManager[] trustAllCerts = new TrustManager[] { 
    new X509TrustManager() {

     @Override
     public void checkClientTrusted(java.security.cert.X509Certificate[] arg0,
       String arg1) throws CertificateException {
      // TODO Auto-generated method stub

     }

     @Override
     public void checkServerTrusted(java.security.cert.X509Certificate[] arg0,
       String arg1) throws CertificateException {
      // TODO Auto-generated method stub

     }

     @Override
     public java.security.cert.X509Certificate[] getAcceptedIssuers() {
      // TODO Auto-generated method stub
      return null;
     }

    } 
  };

  // Install the all-trusting trust manager
  final SSLContext sc = SSLContext.getInstance("SSL");
  sc.init(null, trustAllCerts, new java.security.SecureRandom());
  HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

  // Create all-trusting host name verifier
  HostnameVerifier allHostsValid = new HostnameVerifier() {
   public boolean verify(String hostname, SSLSession session) {
    return true;
   }
  };

  // Install the all-trusting host verifier
  HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);

  URL url = new URL(queryString);
  HttpsURLConnection con = (HttpsURLConnection)url.openConnection();

  return con;
 }
}

Here we are providing our own implementation of the TrustManager instead of using the trusted certificates from our keystore. All the magic is happening in the openHttpsConnection() method where we are passing the URL to connect to and the method is returning a HTTPS connection.

Notice the lines where we are calling the static methods setDefaultSSLSocketFactory and setDefaultHostnameVerifier of HttpsURLConnection class. Here we are setting our own HostnameVerifier and SSLSocketFactory implementation.

Notice the verify() method of the HostnameVerifier class. This method is simply returning true for every host and thus any host becomes valid for the HTTPS connection.

Also notice how the SSLContext is initialised. Here we are passing the customer implementation of X509TrustManager interface where we are simply overriding all the security methods with empty implementations; essentially doing nothing to prevent a security breach.

That means that any certificate from the HTTPS server is accepted as trusted and used for encrypting communication between the client and the server. This allows you to connect to a HTTPS server without installing the certificate in the JVM before making any connections.