I am glad to see the ColdFusion community so excited with CF 8 performance. We got very encouraging response regarding the performance in our pre-release forums and we are thankful to everyone who gave such invaluable feedbacks. Though people started seeing a very good performance gain in their application on ColdFusion 8, we never talked about the performance benchmark numbers until CFUnited 2007, where Ben showed the performance numbers for ColdFusion 8. No wonder this is the #1reason in Top 8 reasons why you want ColdFusion 8 :-).
Extracting performance up to this extent was not an easy task. We analyzed nearly 2.4 million lines of real life CF application code, to zero-in on the most commonly used tags and functions. Main challenge was after that - analyze the generated java code for all the tags, change the compiler to generate more optimized code, run it through profiler and optimize CF engine for each of the tags and functions and their various combinations. And this went on and on in many many iterations. Overall it was real fun :-)
I still remember the most exciting moment when I ran the load test for CFC, after I had made some code changes, and the result was freaking unbelievable. It looked too good to be true and I was literally running around with the code changes to run it on other machines and verify it . That one small change gave nearly 6x gain :-). It is not that the CF7 code was inefficient or poorly written. It was only matter of extracting juices as much as possible and putting some smart intelligence.
Check out the ColdFusion 8 performance whitepaper which talks in much detail about the performance numbers for different areas, the methodology used for benchmarking etc. Manju Kiran, who was my QA-buddy for most of the features I worked on, did a tremendous job in setting up and running the benchmark and creating the meat of this wonderful document (and of course keeping me on toes).