Host name verification in HTTPS

As I mentioned in my blog on Missing images in CFDocument, in case you are using HTTPS, you must ensure that the certicate host name matches the host name in the URL. Lets say your certificate is issued to "www.mysite.com" then the request URL must have the host as "www.mysite.com". It can not be accessed using 'localhost', '127.0.0.1', that machine's IP address or machine's name.
Till JDK1.3, Sun's SSL implementation never used to verify the host name of the certificate. Since JDK1.4, it now verifies the hostname to prevent URL spoofing (When I request for some URL, some other guy in between intercepts and sends his own certificate and I will remain under the impression that I was getting the certificate of the requested server and hence a threat).

In case you want to access the URL using localhost or IP address or using machine's name, there is a workaround possible but that would invlove wrting some java code.
HttpsURLConnection that is used to make the connection, uses an interface HostnameVerifier to verify the host name of the certificate. A default implementation is used by default. You can provide your own implementation of this interface and set it on HttpsURLConnection. That will give the control of host name verification in your hand and you can verifiy it the way you want.

Unfortunately, this interface and HttpsURLConnection are present in both javax.net.ssl and com.sun.net.ssl package. So depending on which SSL packages are being used, you will have to implement appropriate interface and you will have to set this on appropriate HttpsURLConnection. To be sure, let your implementation class implement both the interface and set it on both the HttpsURLConnection by calling the static method

setDefaultHostnameVerifier(HostnameVerifier)

A simplistic implementation which disables any host name verification could be like

    class MyHostnameVerifier implements com.sun.net.ssl.HostnameVerifier, javax.net.ssl.HostnameVerifier{
        public boolean verify(String urlHostName, String certHostName){
            return true;
        }

        public boolean verify(String urlHost, SSLSession sslSession){
            return true;
        }
    }

set this verifier to both HttpsURLConnection at appropriate place.

    MyHostnameVerifier verifier = new MyHostnameVerifier();
    javax.net.ssl.HttpsURLConnection.setDefaultHostnameVerifier(verifier);
    com.sun.net.ssl.HttpsURLConnection.setDefaultHostnameVerifier(verifier);

Thread dumps...

Many a times our customers complain of Coldfusion Server(CF) hanging, becoming non-responsive, taking 100% cpu or some request taking too much time. In order to understand the reasons why this happens, only way is to analyze the "Thread dumps". Not many of our CF users are java geeks and therefore when we ask them to send a thread dump, they have a very little clue about it. I hope they wouldn't be clueless anymore ;)

What exactly is "Thread dump"
"Thread dump" basically gives you information on what each of the thread in the VM is doing at any given point of time. This makes "Thread dump" an excellent debugging tool. It can tell you the states of each of the thread in the VM, where exactly each thread is in the execution path at that point, which thread is waiting and where is it waiting and lot more. It also shows you the stack of each thread and can help you track the execution path.

How do I take a Thread dump?
For any Java VM, if you are running it from a command prompt, you can get the thread dump by pressing Ctrl+Break (for windows) or Ctrl+\ (for unix machines) on the terminal running the server.
In case you are running the VM in background or as services, you can send a "SIGQUIT" signal to the process to get the thread dump. To send this signal to VM on unix machine, you can use "kill -SIGQUIT [pid]" where 'pid' is the process id of the server process. This will send the signal to the VM to dump the thread stack on the error stream (On Sun JVM) or to a new file (IBM's JVM).

You can also use the Stacktrace tool to get the thread dump. It is a nice java webstart application that you can run on the machine running the server. All you need is to specify the "process id" and it will nicely show the threaddump in its own window.

To be more specific for CF, if the standalone CF server or CF on J2EE server has a terminal, use Ctrl+Break or Ctrl+\ depending on the platform. If its running as background process or as services, use Stacktrace tool I mentioned earlier.

Here is how the thread dump would look.

Full thread dump Java HotSpot(TM) Server VM (1.5.0_04-b05 mixed mode):

"web-4" prio=5 tid=0x28999418 nid=0x1460 runnable [0x2fbaf000..0x2fbafd18]
       at java.io.WinNTFileSystem.canonicalize0(Native Method)
       at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:374)
       at java.io.File.getCanonicalPath(File.java:531)
       at java.io.FilePermission$1.run(FilePermission.java:218)
       at java.security.AccessController.doPrivileged(Native Method)
       at java.io.FilePermission.init(FilePermission.java:212)
       at java.io.FilePermission.(FilePermission.java:264)
       at java.lang.SecurityManager.checkRead(SecurityManager.java:871)
       at java.io.File.exists(File.java:700)
       at jrunx.resource.FileResource.exists(FileResource.java:98)
       at jrunx.resource.FileResource.getURL(FileResource.java:140)
       at jrun.servlet.JRunServletContext.getResource(JRunServletContext.java:192)
       at jrun.servlet.file.FileServlet.service(FileServlet.java:148)
       at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:91)
       at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
       at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:257)
       at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:541)
       at jrun.servlet.http.WebService.invokeRunnable(WebService.java:172)
       at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:318)
       at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:426)
       at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:264)
       at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

"web-3" prio=5 tid=0x2814e470 nid=0xd70 in Object.wait() [0x2faaf000..0x2faafd98]
       at java.lang.Object.wait(Native Method)
       - waiting on <0x064b5a90> (a jrunx.scheduler.JSemaphore)
       at jrunx.scheduler.JSemaphore.acquire(JSemaphore.java:74)
       - locked <0x064b5a90> (a jrunx.scheduler.JSemaphore)
       at jrun.servlet.network.NetworkService.accept(NetworkService.java:348)
       at jrun.servlet.http.WebService.createRunnable(WebService.java:104)
       at jrunx.scheduler.ThreadPool$DownstreamMetrics.createRunnable(ThreadPool.java:285)
       at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:347)
       at jrunx.scheduler.ThreadPool$UpstreamMetrics.createRunnable(ThreadPool.java:239)
       at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"web-2" prio=5 tid=0x271954c0 nid=0x170c in Object.wait() [0x2ef6e000..0x2ef6fa18]
       at java.lang.Object.wait(Native Method)
       - waiting on <0x02af8eb8> (a java.awt.image.PixelGrabber)
       at java.awt.image.PixelGrabber.grabPixels(PixelGrabber.java:254)
       - locked <0x02af8eb8> (a java.awt.image.PixelGrabber)
       at java.awt.image.PixelGrabber.grabPixels(PixelGrabber.java:209)
       at com.lowagie.text.Image.getInstance(Unknown Source)
       at com.lowagie.text.Image.getInstance(Unknown Source)
       at com.lowagie.text.pdf.PdfGraphics2D.addAltText(Unknown Source)
       at ice.pilots.html4.MacromediaCSSExtension.addonImageProcessing(MacromediaCSSExtension.java:186)
       at ice.pilots.html4.ObjectBox.paintChunk(OEAB)
       at ice.pilots.html4.InlineBox.paintChildren(OEAB)
       at ice.pilots.html4.InlineBox.paintChunk(OEAB)
       at ice.pilots.html4.InlineBox.paintChildren(OEAB)
       at ice.pilots.html4.BlockBox.paint(OEAB)
       at ice.pilots.html4.OutlinePainter.drawBox(OEAB)
       at ice.pilots.html4.BlockBox.paint(OEAB)
       at ice.pilots.html4.OutlinePainter.drawBox(OEAB)
       at ice.pilots.html4.BlockBox.paint(OEAB)
       at ice.pilots.html4.CSSLayout.paint(OEAB)
       - locked <0x02cebb10> (a java.lang.Object)
       at ice.pilots.html4.ThePrinter.printPage(Unknown Source)
       at coldfusion.document.DocumentSection.process(DocumentSection.java:230)
       at coldfusion.document.DocumentSection.print(DocumentSection.java:108)
       at coldfusion.document.DocumentExporter.export(DocumentExporter.java:237)
       at coldfusion.document.DocumentFrame.exportContent(DocumentFrame.java:118)
       at coldfusion.document.DocumentProcessor.processContent(DocumentProcessor.java:130)
       at coldfusion.document.DocumentProcessor.ProcessContent(DocumentProcessor.java:59)
       at coldfusion.tagext.lang.DocumentTag.processContent(DocumentTag.java:1218)
       at coldfusion.tagext.lang.DocumentTag.access$100(DocumentTag.java:84)
       at coldfusion.tagext.lang.DocumentTag$3.run(DocumentTag.java:1179)
       at java.security.AccessController.doPrivileged(Native Method)
       at coldfusion.tagext.lang.DocumentTag.doAfterBody(DocumentTag.java:1174)
       at cfcmyk2ecfm937582361.runPage(E:\CFusionMX7\wwwroot\testfolder\doctest\cmyk.cfm:3)
       at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:152)
       at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:349)
       at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
       at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:209)
       at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:51)
       at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
       at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:69)
       at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
       at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
       at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
       at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
       at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:115)
       at coldfusion.CfmServlet.service(CfmServlet.java:107)
       at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:78)
       at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:91)
       at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
       at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:257)
       at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:541)
       at jrun.servlet.http.WebService.invokeRunnable(WebService.java:172)
       at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:318)
       at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:426)
       at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:264)
       at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
.
.
.

If you anayze this thread dump, you can see that "web-2" thread is handling request for "cmyk.cfm" and is waiting for the images to be loaded i.e (actually at Image.getInstance()). At this point, cfdocument has sent another http request for the image file that it has to render and that request is handled by another thread "web-4" which is running and is currently checking if the image file requested exists on the file system. Isn't it very powerful ?

You can checkout a very nice article Thread dump and stack traces written by Rajiv (my ex-colleague and mentor in Pramati) which I feel is a must read for Java developers.

Missing images in CFDocument

I have seen huge no of postings on CF forum where CFDocument users have complained about seeing red-cross (or red-x ) for images . In this post, I would like to list the reasons why they happen and how they could be resolved .

Before we proceed with different cases of red-x, lets see in brief, how images are rendered in CFDocument.

During the processing of cfdocument tag, CF engine interprets/executes the content inside the cfdocument tag, creates html content out of it and renders it in the memory. While rendering this html content, if any image tag is found, a separate HTTP request is made to retrieve this image content. A separate HTTP request is necessary because image in the generated html can be local as well as remote. For Java geeks out there, we use URLConnection.getContent() to retrieve the image data. A red-x means that CFDocument was not able to retrieve the image.

Now lets see different scenarios one by one

1. Image name has space in it. In CFMX 7.0, you can get red-x for images if the image file name has any space in it. For example if the image file name is "my picture.jpg", only a red-x will appear in the pdf/flashpaper. It happens because the url created for the image is not encoded. A workaround for this is to either use encoded url for the images i.e replace 'space' in the name with '%20' OR dont have spaces in the file name at all :). This bug has been fixed in Merrimack (Coldfusion 7.0.1). So if you are still on 7.0, upgrade :)
   
   
2. If your server is behind firewall. As we mentioned earlier, CF server needs to send an HTTP request for the images. If the firewall prevents any outgoing connection from the server, CF will not be able to retrieve them and will show a red-x in place of them. You will need to setup your firewall in such a way that server can send an HTTP request to itself.
   
   
3. If your server is behind a proxy. If Coldfusion server is connected to the external world using a proxy, then also CFDocument will not be able to load the images. This is because currently there is no way you can specify proxy configuration for CFDocument tag.
   Current solution to solve this is to define the following system properties for the JVM. You can specify these in "runtime/bin/jvm.config" if you are using standlone or on JRun server.
   
   -DproxySet=true -DproxyHost=[hostname] -DproxyPort=[port] -Dhttp.proxyHost=[hostname] -Dhttp.proxyPort=[port]
   
   
4. If you are using HTTPS and your images do not appear in the pdf/flashpaper, you must ensure the following
• CF server's certificate is trusted. In other words, certificate of the CA who issued the certificate for you, must be present in the trusted certificate store (runtime/lib/trustStore). You can use keytool to list/view/import/.. certificate in the certificate store.
  If CF is using a self signed certificate, CF's certificate must be present in the trustStore.
• The certificate is valid and has not expired.
• Host name of the server must match the host name to which the certificate was issued.
  
  
5. If the resources on your webserver are protected using some kind of authentication like basic authentication or digest authentication, cfdocument can not retrieve those resources. That is because you can not provide any authentication information to cfdocument tag currently. This means that cfdocument can not retrieve images if it is protected using authentication and you will see a red-x. One solution for this is to replace all image urls with "file" urls.
   See this entry for more details on this workaround. The other solution is to write little java code to set a 'java.net.Authenticator'. I will post a separate entry for this.


6. You get red-x for images and you have verified that its none of the above mentioned cases. Time to check the web server now. We have seen some cases where the web server is configured to allow requests only from a certain set of browsers (User agents to be precise) perhaps to prevent spiders and bots from overloading the server. When CFDocument creates a URLConnection for the image, it sends a "User-Agent" header, that looks like "User-Agent:Java/1.4.2_07", in the HTTP request. If the web server does not recognize "Java" user-agent, it returns a status code of 404 (resource not found) and hence the images can not be displayed. Solution for this case is to either change the configration for the web server or set your own user agent using the following system property on the JVM.
   

       -Dhttp.useragent="ColdFusion"
   

   You can give any name here as userAgent in place of "ColdFusion".
   

Hope this helps people in resolving isues related to missing images in CFDocument. In case you are getting a red-x even after verifying all the cases mentioned above, please let me know.

Related Entry :

• Workaround for cfdocument missing images
• Accessing password protected URL
• New CFDocument Hotfix released
  
• Enhancement to CFDocument in CF8