My team runs ColdFusion (v9) on five separate servers. Three of them are on a load balanced router, and the other two are development and test machines.
Several months ago, we began to have issues where certain page/template requests would fail. When I say fail, I mean that the server would return a 500 error. The error, however, did not trigger our ColdFusion exception handling via cftry/cfcatch and cferror.
After examining the HTTP headers, it looked like some sort of jrun error, so I went into the exception log. Here is an example of an error (this one happened this morning):
"Error","jrpp-1","07/30/12","06:30:02",,"(class: cfezReporting2ecfc400556386, method: runPage signature: ()Ljava/lang/Object;) Incompatible object argument for function call The specific sequence of files included or processed is: X:\docs\ezBuilder\index.cfm'' "
java.lang.VerifyError: (class: cfezReporting2ecfc400556386, method: runPage signature: ()Ljava/lang/Object;) Incompatible object argument for function call
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
at java.lang.Class.getConstructor0(Class.java:2699)
at java.lang.Class.newInstance0(Class.java:326)
at java.lang.Class.newInstance(Class.java:308)
at coldfusion.runtime.TemplateClassLoader.newInstance(TemplateClassLoader.java:552)
at coldfusion.runtime.TemplateClassLoader.newInstance(TemplateClassLoader.java:523)
at coldfusion.runtime.TemplateProxyFactory.getCFCInstance(TemplateProxyFactory.java:270)
at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:173)
at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:158)
at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:148)
at coldfusion.cfc.ComponentProxyFactory.getProxy(ComponentProxyFactory.java:62)
at coldfusion.tagext.lang.InvokeTag.doEndTag(InvokeTag.java:373)
at cfezBuilder2ecfc293785079$funcCREATE_UVN._factor7(X:\docs\ezBuilder\components\ezBuilder.cfc:376)
at cfezBuilder2ecfc293785079$funcCREATE_UVN.runFunction(X:\docs\ezBuilder\components\ezBuilder.cfc:327)
at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472)
at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405)
at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368)
at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55)
at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321)
at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:517)
at coldfusion.runtime.CfJspPage._invokeUDF(CfJspPage.java:2547)
at coldfusion.tagext.lang.InvokeTag.doEndTag(InvokeTag.java:460)
Once this error occurred on a template, it would continue to occur, i.e. it was not sporadic and didn't 'fix itself'.
Another interesting tidbit is that this error seemed to occur on all the servers simultaneously on the same file. By simultaneously, I mean by the time we received notice that one is erroring, we test the other servers and find they are also erroring.
Not being able to find help on this particular error detail anywhere, I made a guess that somehow ColdFusion was unable to find a template or class file. I checked each server and the .cfc file was there (plus, the 'template not found' exception has always been different).
So instead, I made a change to the file (ezReporting.cfc in the above example) and copied it to all the servers, and lo and behold, it worked. The change was simply adding and then removing a space - basically forcing the template to recompile, I guess. I did this each time over the next few weeks each time the error occurred. It was always on that .cfc file.
The next time the error happened, about a week later, it was on the exact same file. This time, I cleared the template cache on the server rather than physically change the template. It worked again.
Since then, this same error occurred on several different files - both CFCs and CFM files. I did not notice a pattern in (a) the "age"/last updated date of the files - everything from a day old to over a year, (b) the content of the files - everything from simple blocks to SQL queries to some basic set/looping, or (c) the names of the files.
This morning, it occurred on yet another file, one that I knew worked two days prior on all servers. The CFC file contents had not been touched on any servers since last year. And, when I went to all five servers, the exact same file failed on all of them. When I cleared the template cache, every server started working again.
I'm giving all these boring details because I'm reaching for anything that can help. Perhaps there is some sort of expiration occurring on the file itself, and that would explain why the file expired on all servers right around the same time -- we change it on the development server and then move it out to the test and production servers with a simple copy/paste. But, there doesn't seem to be any rhyme or reason for expiration, because the file in question is of varying ages as noted above.
I tried the Adobe forums to get help on this particular exception/dump, but have not found anyone who has run into the same thing.
Does anyone else have any ideas? This error troubles me because it doesn't trigger our normal exception handling, so we can't do much about it without human intervention. Thanks for any specific guidance.