I am starting from scratch, putting in a unit testing framework, and working within the existing organization's culture to introduce enough of the tool’s benefits without burdening the organization with too many of the downsides. Everything has a downside, a cost, or whatever you wish to call it. There is no free lunch.
The first steps were to gage the organization's desire for the possible benefits of a unit testing framework, and their willingness to commit to moving forward, and accepting the cost. Having been in an "extreme" programming environment, which was only "extreme" in the sense that projects were "extremely" haphazard and "extremely" mismanaged, making sure that people are clear on what can be expected from implementing some test automation for unit testing and regression testing. Automated does not mean that you do nothing, and the machine does everything. It is difficult to know what people have been sold by the trade journals, and blogs, and common terms are often misappropriated.
The current environment is Flex/Actionscript and HTML, with a ColdFusion backend, with a ColdSpring framework.
The goals are:
1) Automated testing with Ant, with performance metrics logged in a database, and pass/fail results e-mailed nightly to interested parties. (Thorough regression testing and reporting is key).
2) Eclipse Integration or reasonably nice Unit Runner, with easy ability to select individual tests, or test suites. (Easy sell to the engineers is key)
3) Mock Objects, automated test generation, anything to minimize the time impact on engineering, while maximizing the benefit, and keeping the program moving.
Sean Corfield had mentioned MXUnit fairly recently, so it seemed worth a look. Given the higher volume of activity on MXUnit, when compared to CFCUnit and CFUnit, it seemed like a reasonable way to go, for the best of all worlds for future itegrated add-ons. The first downside I will mention is that *MXUnit’s Eclipse Integration is Not Flex Builder 2.01 Compatible, or rather, not compatible with the version of Eclipse (3.11) that ships with Flex Builder 2.01* This wasn’t super clear from the MXUnit website, so it’s possible for an engineer to go through the installation instructions, and end up with a failing plug in. This might not be a problem for some organizations, but may be for others. Most of the team has not yet upgraded to FB 3, so it could be an issue at the moment.
MXUnit's Eclipse integration isn't quite finished, or at least that's how it felt for me. It "worked", but was lacking a few things that I thought were important. An html or Flex based unit runner will do the trick though, so it's not a huge problem.
The things which make the integrated unit runner a little less than usable, for me, are:
1) It seems to have no memory of past tests, so I would have to go through the selection process for the unit test every time; some way to save the available test cases/test suites would be desireable/required,
2) The test case selection process was awkward; I found myself unable to select a Test Case which I knew was present on more than one occasion; This could be forgiven more easily, if it would remember the test cases I had selected in the past, but I was required to select/find the test case with each restart of Eclipse, or any time I wanted to switch tests
On the plus side, the ant unit runner does a great job, and it is clear how to modify the ant task. The availability of the different test result formats, allows me to feed the results into a variety of destinations with ease. The key item was logging test results and performance metrics, and not just logging into a text file, but logging into a database table, so it could be consulted for history. I was able to hack this item in really easily using the cfquery output format for the testResults.
I should at least mention that CFUnit and CFCUnit are both seemingly capable frameworks. I was able to set up integrated Eclipse (with Flex Builder 2.01!) unit tests, and test suites. But the availability of the test Result Formats, and the volume of project activity, were enough to convince me. That does not mean that I am right, nor does it mean it is the best choice for everyone. But the software selection is but one small piece of the puzzlement.
More Information visit ColdFusion Development Services.
Post a Comment