NST

NST is a test framework for Common Lisp offering separate and reusable fixture definitions, test groups, extensible test criteria, ASDF integration, and optional JUnit XML output.

Download ASDF package from http://www.sift.info/nst/nst-1.3.1.tar.gz

Compatibility.

Platform	Platform version	NST version	Self-test results	Warnings/notes
Allegro	8.1	Through 1.3.1	All tests pass.	Few warnings. Allegro is our primary development platform, and so it's what NST best supports.
Allegro	8.2	1.3.1	All tests pass.
SBCL	1.0.29.11 debian	1.2.7 - 1.3.1	All tests pass.	Many style warning cascade from test definitions.
Clozure	1.4-r13119	1.2.8 - 1.3.1	All tests pass.	Few warnings.
CLisp	2.44.1	1.2.8 - 1.3.1	All tests pass.	Some warnings - more than Closure, many fewer than SBCL.
CMU CL	At one point, NST compiled and loaded without error or warning, but loading NST's self-tests failed completely: "primitive halt called; the party is over." Unfortunately, CMU CL is not available for Ubuntu Karmic for AMD64 (my current work machine), so I haven't had an opportunity to debug this more deeply.
Lispworks	Since the Closer to MOP library supports Lispworks, NST should as well. Unfortunately, the "personal edition" of Lispworks does not seem to include their `asdf-integration` - but they're promising an new release this year, so maybe then.

We'd love to hear of your experience with NST on any other platforms. Out of the box, NST requires the closer-mop package, but should be adaptable to any platform with an implementation of the MOP.

Other resources:

The NST manual and quick reference sheet are separately downloadable (their LaTeX source is in the tarball).
Slides introducing NST to the Twin Cities Lispers' Group (June 9, 2009).

Future development:

We use NST for a number of projects at SIFT, so we expect to continue issuing bug fixes and improvements.
Known bugs: we know about many! We hope to open NST's trac website soon. For now, please email with any problems you encounter.

Recent changes (a possibly-more-complete list of changes is in the tarball):

In 1.3.1 (uploaded February 22):
- ACL 8.2 compatibility - fixed an issue with earlier versions that the new Allegro release didn't like.
- Fixed a minor bug in the JUnit XML output.
In 1.3.0 (uploaded February 2):
- The major fix in this release. The implementation of criteria has been completely rewritten to use method dispatch rather than macro expansion. The def-form-criterion macro is invalidated; the def-values-criterion is deprecated.
- Added the ability to cache fixtures, so that they will be evaluated once.
- Allow group setup/cleanup operations both within and without group fixture bindings.
- Runtime Lisp warnings are now incorporated as NST warnings.
- Added runtime system commands and debugging macros, improved docstring handling.
- Under SBCL, keyboard interrupts are no longer "swallowed" by NST as test errors, but instead stops testing and enters the debugger.
In 1.2.8 (uploaded January 27):
- 1.2.8 clears out a couple of bug fixes in the 1.2 trunk. We've now merged the 1.3 branch into the trunk.
- The package argument to :nst :run-package is now optional, with *package* as default.
- Bug fix for Mac OSX: suppress backtrace collection.
- Download ASDF package from http://www.sift.info/nst/nst-1.2.8.tar.gz
In 1.2.7 (uploaded November 15, 2009):
- Greater compatibility: Clozure CL, clisp.
- Fixed a bug affecting execution of test cleanup.
- Fixed a bug affecting re-definition of fixture sets.
In 1.2.6 (uploaded November 11):
- Reorganized "meta" tests to better examine and regression-test NST's performance in detail.
- Allow tests to be defined separately from their group.
- Replaced check-result with emit-success.
- Improved deprecation warnings.
- Add restarts for use in interactive operation.
- (Start to) capture fixture names when errors arise in fixture application.
- Better failure checking of cleanup operations.
- Additional packaging options.
- Various bug fixes.
In 1.2.0 (uploaded July 22):
- New criterion for quickcheck-style sampling of generated data.
- New do-what-I-mean interactive command :nst :run.
- Code improvements for compile/load-time improvements.
- Deactivated Allegro backtrace harvesting on Mac OS X due to a bug (reported) in Allegro Lisp for Macs.
- Added deprecation warnings on several disused macros.
- Clarified error output message for :eq/:eql/:equal/:equalp.
- Added an ASDF-customizable meaning for ":nst :debug".
- Fixed Makefile for documentation.
- Re-activated links in generated PDF.
- Change to use of "import" for ":nst :open".

NST was written by John Maraist.

test framework development

Page in this topic: John Maraist

CLiki pages can be edited by anyone at any time. Imagine a fearsomely comprehensive disclaimer of liability. Now fear, comprehensively