test framework

Somebody once said that Lispniks have been the first agile developers, doing the stuff people now all go mad about decades ago without thinking twice. So maybe it isn't very surprising that there are quite a lot of tools that help with test driven development.

A quite comprehensive review of various test frameworks is available here.

Known Common Lisp Test Frameworks on CLiki include:

5am - 5am is a nickname for FiveAM, which is a Test Framework by Marco Baringer
ASDF-Install-Tester - ASDF-Install-tester is a development tool and ASDF-Extension that automates the process of checking whether or not ASDF-Installable systems actually install under ASDF
cl-smoketest - cl-smoketest is a small test framework for setting up basic does-it-compile tests
Clickcheck - Clickcheck is a Development tool that provides a test framework
clos-unit - Basically a port of JUnit to Common Lisp, probably deprecated in favor of xlunit
CLUnit - CLUnit is a Test Framework - this Development tool allows a Common Lisp user to create maintain and run tests within a lisp environment
EnclineTest - EnclineTest (etest) is a unit test framework, developed within Encline project
FiveAM - &BODY
FReT - FReT is a unit test framework with interactive testing and authoring of tests
LIFT - The LIsp Framework for Testing is another testing framework in the JUnit / SUnit lineage
MSL-Test - MSL-Test is Paul Foley's Test Framework
NST - NST is a test framework for Common Lisp offering separate and reusable fixture definitions, test groups, extensible test criteria, ASDF integration, and optional JUnit XML output
ptester - ptester is a portable version of Franz's tester library
RT - Short for "regression testing", an older (last update 1995)
rtch - rtch is a Development tool and hackbrary which facilitates organizing tests in the RT test framework in a directory hierarchy
Stefil - Stefil is a young test framework that drops the common idea of *Unit testing and is based on instrumented defun's callable from the repl
tester - tester is the test framework harness that Franz uses for Allegro CL (ACL) development
unit-test - unit-test is a Test Framework by Alan Picard
xlunit - XLUnit is a Test Framework based on XPTest and JUnit
XPTEST - XPTEST is a development tool that provides an automated test framework based on Extreme Programming

There are probably more, because CL makes it really easy to write test code. :)

Like COVER: Common Lisp test case coverage determination tool.

XPTEST and clos-unit ( clos-unit appears to be deprecated in favor of xlunit which has almost the same semantics ) are basically implementations of the *Unit design (like JUnit, SUnit etc), where a test case (and a test fixture, there isn't much differentiation) is a CLOS class, and the individual tests are its methods. Failure of a test is usually indicated by calls to some assertion forms or by signaling an appropriate condition.

FiveAM seems to offer similar functionality, but handles it in another fashion. For example, setting up and tearing down a fixture isn't done with defining methods for a test-case class, but with a WITH-FIXTURE macro which simply wraps one piece of code with another.

rt is an older library from the CMU AI repository. It takes a more simplistic approach, a single test is made up of its name, a form to evaluate, and the expected return values of this form. CLUnit takes a similar approach, but with a lot more sugar, like collecting and classifying tests, giving a comparison function to match the real with the expected output etc.

Clickcheck is a specification based random testing tool.

Status notes

XPTEST seems to be unreachable.

Pages in this topic: 5am ASDF-Install-Tester cl-smoketest Clickcheck clos-unit CLUnit EnclineTest FiveAM FReT LIFT lisp-unit MSL-Test NST ptester RT rtch Stefil tester unit-test xlunit XPTEST

Also linked from: development Test Test Frameworks Testing

CLiki pages can be edited by anyone at any time. Imagine a fearsomely comprehensive disclaimer of liability. Now fear, comprehensively