- If you know that the URL is protected : This is the simple one. Since you know the URL is protected, add the required authorization header in the request and you are done. What if you are not making the socket connection but instead using java.net.URL or java.net.URLConnection to retrieve the content of url? You can use setRequestProperty() method of URLConnection to add any request header you want. So the code would look like
URL url = new URL(someurlstring)
URLConnection conn = url.openConnection();
String encoding = new sun.misc.BASE64Encoder().encode("username:password".getBytes());
conn.setRequestProperty ("Authorization", "Basic " + encoding);
InputStream in = conn.getInputStream()
... - What if you dont know in advance whether the URL is protected or not? You can not arbitrarily add the authorization heder for all the URLs. The answer to this is "java.net.Authenticator". You need to install an instance of java.net.Authenticator using setDefault() method of Authenticator. Whenever URLConnection sees that the url is protected, it will use this installed authenticator to get the username and password and set the required authorization header automatically.
Here is how the code snippet for doing this
Authenticator.setDefault(new MyAuthenticator());
....
public class MyAuthenticator extends Authenticator
{
protected PasswordAuthentication getPasswordAuthentication()
{
return new PasswordAuthentication ("username", "password");
}
}
This particular technique is particularly useful for people who need a workaround for retrieving password protected images in cfdocument. Here is what you need to do.- Use the source below to create Authenicator class
import java.net.*;
public class MyAuthenticator extends Authenticator
{
private String user;
private String passwd;
public MyAuthenticator(String user, String passwd)
{
this.user = user;
this.passwd = passwd;
}
protected PasswordAuthentication getPasswordAuthentication()
{
return new PasswordAuthentication(user, passwd.toCharArray());
}
}
Compile this and put MyAuthenticator.class in wwwroot/WEB-INF/classes. - Put the following code in your application.cfm or application.cfc
<cfif not IsDefined("Application.authenticator")>
<!--- Do initializations --->
<cfset authenticator= createObject("java", "java.net.Authenticator")>
<cfset myauthenticator = createObject("java", "MyAuthenticator").init("username", "password")>
<cfset authenticator.setDefault(myauthenticator)>
<cfset Application.authenticator=myauthenticator>
</cfif>
replace "username" and "password" with actual username and password. - give the request for the cfdocument page for which you were getting the red-x for image. The images should appear this time.
- Use the source below to create Authenicator class
Sunday, February 25, 2007
Accessing password protected URL
How do you access a password protected URL (requiring basic authentication) programmatically? well, there are two ways
Thursday, February 15, 2007
A page in CFSwitch-CFCase' diary
Undoubtedly I am one of the most powerful programming construct of ColdFusion language. And definitely more powerful than all my cousins in other programming languages. All my cousins and even my brother (read java switch-case) work only on integers but I work on almost every kind of objects be it string or any numeric data or even date. Since I can work on almost all datatypes, I make cf developers life so much simpler. I was created this way because the world I was going to be in was UnTyped - where every one was equal and where there was no discrimination between datatypes and I really thank God (ColdFusion Architects) for creating such a wonderful world.
Recently I heard someone talking about me that I am not as fast as my cousins. Some one even talked about ignoring me and taking help of kiddo cfif-cfelse. I dont want to say anything against anyone coz I know "with great power comes great responsibility err..cost".
Its sheer hardwork that makes me so much more powerful than all my cousins. This is what I do when any object comes to me. First I try to see if it is numeric. I do that because that is what all my cousins are used to and I have to remain as fast as them in that case. If it is not numeric, then I check if it can be date. If it is not even date, I try converting it in String. Once I arrive at the data, I use my own data structure to match it with appropriate CASE. So when the data is string, i will take some more time as compared to what i will take when data is numeric. Is that so bad? My cousins dont even do that!
Sometimes I compare myself to a busy lawyer who likes working on many CASEs. I dont like to work just for 2-3 cases. I prefer my grandson cfif-cfelse take care of those small no of cases.
I hope someday people will read these pages and if even after reading this they think that I am too slow and I should be ignored I only want to say "God, forgive them, for they dont know what they are doing!".
Recently I heard someone talking about me that I am not as fast as my cousins. Some one even talked about ignoring me and taking help of kiddo cfif-cfelse. I dont want to say anything against anyone coz I know "with great power comes great responsibility err..cost".
Its sheer hardwork that makes me so much more powerful than all my cousins. This is what I do when any object comes to me. First I try to see if it is numeric. I do that because that is what all my cousins are used to and I have to remain as fast as them in that case. If it is not numeric, then I check if it can be date. If it is not even date, I try converting it in String. Once I arrive at the data, I use my own data structure to match it with appropriate CASE. So when the data is string, i will take some more time as compared to what i will take when data is numeric. Is that so bad? My cousins dont even do that!
Sometimes I compare myself to a busy lawyer who likes working on many CASEs. I dont like to work just for 2-3 cases. I prefer my grandson cfif-cfelse take care of those small no of cases.
I hope someday people will read these pages and if even after reading this they think that I am too slow and I should be ignored I only want to say "God, forgive them, for they dont know what they are doing!".
Performance Tips : ColdFusion List
how many of you have written/seen code like this?
While there is nothing wrong with it syntactically or functionally, performance wise it is very poor. Why? ColdFusion list is nothing but String (delimited by delimiter). ColdFusion does not have any way to build any intelligence to keep it in any other datastructure because you can use it like a normal string also. So what happens when you call any List function on this string? We parse the string using the delimiter and get the delimited tokens and process that.
Now lets take ListGetAt(list, index) function. It will keep parsing and getting the token unless it reaches the required index. Imagine doing in a loop. We will be parsing the same string again and again and traversing from the beginning everytime till we reach the next loop index. So, in the Nth iteration, it will start from beginning and tokenize N times. Thus by the time you have completed the loop, you have parsed/tokenized the string N*(N+1)/2 times. Isn't that too costly? Lesson - Never ever use ListGetAt() in a loop. Either iterate using OR convert the list into array using ListToArray() and iterate over it. Using cfloop is the most optimized way to do this.
Even if you are not iterating over list but you need to call ListGetAt() many times, it is better to convert it to array and then search the index in that.
Same thing applies to search functions like ListFind, ListContains etc. If you need to call these multiple times on the same list, you will be better off converting the list to array and searching in that.
If you need to append many items to the list, then also you will get a better performance by converting the list to array and doing all appends on that.
This does not mean you should not use list at all or you should always convert the list to array and work on that. If the number of operations that you are doing on the list is less, you should stick to list because converting the list to array is also costly. If you are inserting an element in the middle of list, list will be better than array in most cases.
<cfset mylist="jan,feb,mar,apr,may,jun,jul,sep,oct,nov,dec">
<cfloop from="1" to=#ListLen(mylist)# index="i">
<cfset month = ListGetAt(mylist, i)>
<!--- do something with this month --->
<cfoutput>#month#</cfoutput>
</cfloop>
While there is nothing wrong with it syntactically or functionally, performance wise it is very poor. Why? ColdFusion list is nothing but String (delimited by delimiter). ColdFusion does not have any way to build any intelligence to keep it in any other datastructure because you can use it like a normal string also. So what happens when you call any List function on this string? We parse the string using the delimiter and get the delimited tokens and process that.
Now lets take ListGetAt(list, index) function. It will keep parsing and getting the token unless it reaches the required index. Imagine doing in a loop. We will be parsing the same string again and again and traversing from the beginning everytime till we reach the next loop index. So, in the Nth iteration, it will start from beginning and tokenize N times. Thus by the time you have completed the loop, you have parsed/tokenized the string N*(N+1)/2 times. Isn't that too costly? Lesson - Never ever use ListGetAt() in a loop. Either iterate using
Even if you are not iterating over list but you need to call ListGetAt() many times, it is better to convert it to array and then search the index in that.
Same thing applies to search functions like ListFind, ListContains etc. If you need to call these multiple times on the same list, you will be better off converting the list to array and searching in that.
If you need to append many items to the list, then also you will get a better performance by converting the list to array and doing all appends on that.
This does not mean you should not use list at all or you should always convert the list to array and work on that. If the number of operations that you are doing on the list is less, you should stick to list because converting the list to array is also costly. If you are inserting an element in the middle of list, list will be better than array in most cases.
Tuesday, February 06, 2007
ColdFusion Array : pass by reference or Value.
Ben rightly pointed out in my last post that since ColdFusion array are always passed by value, the second technique can not be used if you want to build the array over multiple method calls. In each of the function call, ColdFusion will create a copy of the array passed and that cost (cost of creating a new instance and copying old array to new one) might even exceed the cost of appending string.
ColdFusion array is one unique data structure in ColdFusion. Unique in the sense that this is the only data structure that is passed by value and not by reference. I do feel it is a limitation sometimes but thats legacy and can not be changed. (You would not want us to break your code. Would you? ;) )
There does exist one hack if you absolutely need to pass the array by reference. Here is a code snippet that uses pass by reference.
So what did we do here? We created an ArrayList from the ColdFusion array. Since ColdFusion Array is an implementation of java.util.List, almost all Array functions work on all implementations of java.util.List. And this list implementation will not be passed by value but will be passed by reference. Thats the power of using java in ColdFusion !
ColdFusion array is one unique data structure in ColdFusion. Unique in the sense that this is the only data structure that is passed by value and not by reference. I do feel it is a limitation sometimes but thats legacy and can not be changed. (You would not want us to break your code. Would you? ;) )
There does exist one hack if you absolutely need to pass the array by reference. Here is a code snippet that uses pass by reference.
<cfset x = ArrayNew(1)>
<cfloop from=1 to=5 index=i>
<cfset Arrayappend(x,"something")>
</cfloop>
<cfset x = CreateObject("java", "java.util.ArrayList").init(x)>
<cfset foo(x)>
<cfset foo(x)>
<cfset x[8] = "after the method call">
<cfset x[9] = "end of method call">
<cfset foo(x)>
<cfdump var="#x#">
<cffunction name="foo">
<cfargument name="arr">
<cfset ArrayAppend(arr, "from function")>
</cffunction>
So what did we do here? We created an ArrayList from the ColdFusion array. Since ColdFusion Array is an implementation of java.util.List, almost all Array functions work on all implementations of java.util.List. And this list implementation will not be passed by value but will be passed by reference. Thats the power of using java in ColdFusion !
Monday, February 05, 2007
The Mystery of "Too many open files"
Last week we saw an interesting problem while running the regression tests on Linux. Immediately after the tests were started, the VM started throwing error "FileNotFoundException : Too many open files". Ofcourse the files were there but the VM was trying to say that there are too many file descriptors open which were not closed. It was hitting the open file limit set by the OS. This was kind of absurd because we always close all the files that we open. And moreover we had never seen this problem before. So we started suspecting 1.6 VM on which we were running it.
Immediately after this, it got worse. The exception changed into "SocketException: Too many open files" and all the socket connection started getting rejected. So merely after serving 100 requests, the server was down to its knees.
A quick google search suggested to increase the open file descriptor limit on the machine.
"ulimit -aH" that gives the max limit for number of open files returned 1024.
I added every possible way to increase it. Here are few
1. In /etc/security/limits.conf
* soft nofile 1024
* hard nofile 65535
2. Increase ulimit by "ulimit -n 65535"
3. echo 65535 > /proc/sys/fs/file-max
4. In /etc/sysctl.conf
fs.file-max=65535
Increasing file descriptor limit did not help much either. Even after increasing this limit, we were still getting this error.And then Sanjeev (another brilliant CF engineer with an amazing sense of humour) cracked it !!
Just before we started getting these errors, there was another error which I had overlooked assuming it was test problem which infact was the clue.
The error was something like
coldfusion.jsp.CompilationFailedException: Errors reported by Java compiler: error: error reading /opt/coldfusionscorpio/lib/./././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././....
at coldfusion.jsp.JavaCompiler.compileClass(JavaCompiler.java:138)
Sanjeev ran it through the debugger and he nailed the culprit. It was something which no one had even suspected. It was javac. :)
We had hit upon a Sun's bug in javac where if the classpath contains a jar which has a manifest and manifest contains classpath entry with relative paths of other jars as well as path to itself, javac goes in an infinite loop. (Sun's bug no 6400872, 6446657, 6456960, 6485027, 6206485)
I dont have the source for javac to pin point what exactly it did, but probably it kept opening all the entries in the classpath and because of the stack overflow it could not close those files - hence reaching the max limit of open file descriptors. This was a third party jar that had this manifest entry. Once we found the problem, the solution was simple - just remove the classpath entry from the manifest! Whew !!!
Immediately after this, it got worse. The exception changed into "SocketException: Too many open files" and all the socket connection started getting rejected. So merely after serving 100 requests, the server was down to its knees.
A quick google search suggested to increase the open file descriptor limit on the machine.
"ulimit -aH" that gives the max limit for number of open files returned 1024.
I added every possible way to increase it. Here are few
1. In /etc/security/limits.conf
* soft nofile 1024
* hard nofile 65535
2. Increase ulimit by "ulimit -n 65535"
3. echo 65535 > /proc/sys/fs/file-max
4. In /etc/sysctl.conf
fs.file-max=65535
Increasing file descriptor limit did not help much either. Even after increasing this limit, we were still getting this error.And then Sanjeev (another brilliant CF engineer with an amazing sense of humour) cracked it !!
Just before we started getting these errors, there was another error which I had overlooked assuming it was test problem which infact was the clue.
The error was something like
coldfusion.jsp.CompilationFailedException: Errors reported by Java compiler: error: error reading /opt/coldfusionscorpio/lib/./././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././././././././././.
/././././././././././././././././././././././././././././././././././././././././././
././././././././././././././././././././././././././././././././././....
at coldfusion.jsp.JavaCompiler.compileClass(JavaCompiler.java:138)
Sanjeev ran it through the debugger and he nailed the culprit. It was something which no one had even suspected. It was javac. :)
We had hit upon a Sun's bug in javac where if the classpath contains a jar which has a manifest and manifest contains classpath entry with relative paths of other jars as well as path to itself, javac goes in an infinite loop. (Sun's bug no 6400872, 6446657, 6456960, 6485027, 6206485)
I dont have the source for javac to pin point what exactly it did, but probably it kept opening all the entries in the classpath and because of the stack overflow it could not close those files - hence reaching the max limit of open file descriptors. This was a third party jar that had this manifest entry. Once we found the problem, the solution was simple - just remove the classpath entry from the manifest! Whew !!!
Subscribe to:
Posts (Atom)
