Sunday, May 21, 2006

 

Introduction to Fit and FitNesse

A few years ago, Ward Cunningham gave me an early version of his Fit (Framework for Integrated Tests) system, telling me he thought it was a great way to deal with software acceptance tests. At that point, it was a lot of code, with little documentation, and I just didn't have the time to figure out what it did. I looked at the simple tutorials and the simple examples, but I couldn't figure out exactly what it was or how I would use it.

Over time I've glanced at it a few more times, but still couldn't grasp the big picture. Recently, I discovered that Ward and a co-author had written a book about it: Fit for Developing Software: Framework for Integrated Tests (also known simply as "the Fit book"). Having nothing else to do this weekend, I read the book, and now I get it.

I've decided to write the introduction that I wish I could have read a couple of years ago. Maybe it will help somebody else with the same mental blocks I had to see how useful Fit is.

Unit Testing vs. Acceptance Testing

Most software developers are familiar with some form of unit testing, where one exercises a piece of code and verifies that it generates the expected results. A lot of people just do ad-hoc printf-style tests, or just run the code in the debugger to verify that it does the right thing, but enlightened people use a unit-testing framework that allows programmers to define automated repeatable tests.

While unit testing is useful for verifying that the software works like the programmers think it should, it is not useful for verifying that the software does what the customer wants it to do. This is where acceptance testing comes in. Acceptance tests take many forms:

In short, unit tests represent the developers' idea of what the system is supposed to do, and acceptance tests represent the customers' view of what they need the system to do. Bridging the gulf between the developers and customers is often difficult, because developers don't understand the customers' needs, and customers have trouble expressing their needs in sufficient detail and formality for developers to write code that meets those needs.

Fit's Approach to Acceptance Tests

The Fit framework addresses this problem by providing a means for customers to define their acceptance tests in a way that is formal enough for developers to create automated tests. While the test definitions have some formality to them, they are still simple enough that non-programmers can create the tests and run them themselves. The tests become the common language that customers and developers use to explore the requirements of the system, and become the measure of compliance and completeness.

Basically, tests are defined as tables, meaning in a 2x2 matrix. Test tables can be created in a spreadsheet, or using an HTML editor. There are several standardized formats for the tables, but Fit is generic enough that it can be used on anything that is in table form.

For example, say we are developing a cash-register system for a grocery store, and we want to define tests for a typical sales transaction. We can start by creating a table that sets up the starting inventory for the store:

Initial Inventory
Product CodeDescriptionPriceNumber in StockAdd?
001Tomato0.49200true
002Milk1.7950true
003Beer5.9950true

In Fit, this is known as a column fixture, containing a set of rows containing input data (Product Code, Description, Price, Number in Stock) and expected output. Here, the expected output of the Add() function for each item is "true", meaning we always expect the Add() function to work when this table is used in testing.

Next, the customer creates a table that illustrates a typical grocery-store sales transaction. The initial English description of this test case might be "Customer buys two tomatoes and a quart of milk, paying with a $20 bill and receiving the correct change." After thinking it through for a while, we end up with this:

Action Fixture
startCustomer Sale
enterproduct code001
pressproduct lookup key
checkdisplay descriptionTomato
checkdisplay price$0.49
pressquantity key
press2 key
checkdisplay descriptionTomato
checkdisplay quantityX2
checkdisplay price$0.98
enterproduct code002
pressproduct lookup key
checkdisplay descriptionMilk
checkdisplay price$1.79
presssubtotal key
checkitems total2.77
checktax0.20
checktotal2.97
checkdisplay descriptionSUBTOTAL
checkdisplay price$2.97
entercash tendered2000
presscash sale key
checkcash tendered20.00
checkchange17.23
checkdisplay descriptionCHANGE
checkdisplay price$17.23
checkreceipt printedtrue

Note how much more detailed this is than the original description. The customer has been able to specify which buttons get pressed, what should show up on the LED display at each point in the transaction, internal calculations by the register, and whether a receipt should be printed. We've even captured little details like allowing $20.00 to be entered as "2000" instead of requiring the clerk to enter a decimal point.

The customer wants to verify that the store's inventory data gets updated, so we'll add one more table to check the results:

Check Inventory
Product CodeDescription?Price?Number in Stock?
001Tomato0.49198
002Milk1.7949
003Beer5.9950

So, we now have a description of what the customer wants to see the system do. The customer could write a whole series of such tests, perhaps working with developers, quality-assurance testers, and other people to verify that all the bases are covered. It is written in the language of the customer, but it has enough structure and detail that a programmer might be able to figure out how to parse it and execute it.

Fixtures

The Fit framework knows how to parse tables. So all we need to automate this test is a fixture, which is a piece of code that can take the parsed data, execute operations against the system, and then return the result to Fit.

Fit includes a set of standard fixtures, which can be used as-is or subclassed as needed to meet particular needs. Developers can also write their own fixtures from scratch if necessary.

Writing a fixture can be very easy, especially for the standard fixture types. For example, we have to provide an InitialInventory fixture, with a function Add that will be called for each row in that table. Since our Customer Sale table checks that the "display description" is "Milk", we need to provide a CustomerSale fixture with members such as displayDescription and receiptPrinted to process the actions and checks. Finally, the CheckInventory fixture has to provide a numberInStock function to verify the correct numbers after all the actions have been executed.

Note that while non-programmers can create the tests, a programmer is needed to develop the fixtures. It is likely that there will be some back-and-forth communication between the developers and customers to further refine the meaning of the test steps and to resolve any ambiguities or inconsistencies. Communication is a good thing, and Fit encourages it.

Results

After the fixture associated with a table returns its results, Fit displays them in a web page. Calculated values are displayed in green if the result was as expected, or red if not.

For example, if Fit runs the above test case against the system while it is still in development, the output might look something like this:

Initial Inventory
Product CodeDescriptionPriceNumber in StockAdd?
001Tomato0.49200true
002Milk1.7950true
003Beer5.9950true
Action Fixture
startCustomer Sale
enterproduct code001
pressproduct lookup key
checkdisplay descriptionTomato
checkdisplay price$0.49
pressquantity key
press2 key
checkdisplay descriptionTomato
checkdisplay quantityExpect: X2
Actual: (empty)
checkdisplay price$0.98
enterproduct code002
pressproduct lookup key
checkdisplay descriptionMilk
checkdisplay price$1.79
presssubtotal key
checkitems total2.77
checktax0.20
checktotal2.97
checkdisplay descriptionSUBTOTAL
checkdisplay price$2.97
entercash tendered2000
presscash sale key
checkcash tendered20.00
checkchange17.23
checkdisplay descriptionCHANGE
checkdisplay price$17.23
checkreceipt printedtrue
Check Inventory
Product CodeDescription?Price?Number in Stock?
001Tomato0.49Expect: 198
Actual: 200
002Milk1.79Expect: 49
Actual: 50
003Beer5.9950

From this report, we can see that the item quantity is not being shown on the customer display, and the inventory data is not being updated. So we know what the developers need to be working on to make the customer happy.

Fit provides mechanisms for running multiple tests and providing summary reports. So you can run the entire application test suite at the touch of a button.

FitNesse

Fit is designed to work on HTML tables or spreadsheets. A group of people decided it would be really nice if tests could be created and edited in a wiki, to make it really easy for customers, developers, and everyone else to share in the process of creating and reviewing acceptance tests. And so FitNesse was born. See their web site for details on what FitNesse can do.

The Fit Book doesn't have much to say about FitNesse, but if you like wikis and hate HTML editing as much as I do, you won't want to use Fit without it.

Language Support

One of the reasons I was discouraged from learning about Fit early was the fact that it looked like it supported only Java and .NET, and I don't use either of those software development platforms. In fact, while Fit and FitNesse are written in Java, FitNesse invokes an executable called a "FitServer" to run each test fixture, so the fixtures can be developed in any language you want.

Of course, the job is easier if somebody else has already done the work of developing a FitServer for the language you want to use. I have to use C++ in my job, so I'll be looking into the C++ support for FitNesse. (It's there; I just don't know how good it is yet.)

Other Uses

While Fit and FitNesse were designed for acceptance testing, they are fundamentally just a framework for reading data provided in a wiki page, processing that data, and returning the results back to the user via that wiki page. This is very powerful and would be useful for a lot of automatable tasks other than acceptance tests.

Read the Book

If Fit sounds interesting, I suggest you read the book. It does a good job of separating the "what is this good for" material from the "how do we make it work" material, so customers/analysts/testers can read the non-technical stuff, and the programmers can read the technical stuff.


Comments:
FitNesse is a great tool. I've been using it with python development testing for some time now. Grig Gheorghiu and Titus Brown gave a great presentation at PyCon2006 in Dallas this February that demonstrated it as part of a larger testing framework. Great stuff. You just playing with it on your own or using it at the office now too?
-- Ben
 
I'm just playing with it on my own right now, but I may try to get it in the office.

Right now, I'm trying to get developers to use unit tests. It's slow going. Other people think it's a good thing, but not enough to do any of the work themselves.
 
BTW, I know the only reason you posted that comment was to brag that you got to go to PyCon. :)
 
Hi,
I am using WebLogic with eclipse as IDE. now, i have some problems in running fitness for my java application as given below:

Internal Exception:

java.lang.Exception: FitClient: external process terminated before a connection could be established.
fitnesse.components.CommandRunningFitClient$EarlyTerminationRunnable.run(CommandRunningFitClient.java:149)
java.lang.Thread.run(Unknown Source)
pls help me on this.

u can reach me on my office mail id :
kkachabi@virtusa.com
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?