##
17.2 Testing Changes and Building Regression Tests

Cheetah ships with a regression test suite. To run the built-in tests, execute at the shell prompt:

          cheetah test

Before checking any changes in, run the tests and verify they all pass. That way, users can check out the CVS version of Cheetah at any time with a fairly high confidence that it will work. If you fix a bug or add a feature, please take the time to add a test that exploits the bug/feature. This will help in the future, to prevent somebody else from breaking it again without realizing it. Users can also run the test suite to verify all the features work on their particular platform and computer.

The general procedure for modifying Cheetah is as follows:

1. Write a simple Python program that exploits the bug/feature you're working on. You can either write a regression test (see below), or a separate program that writesthe template output to one file and put the expected output in another file; then you can run `diff` on the two outputs. (`diff` is a utility included on all Unix-like systems. It shows the differences between two files line by line. A precompiled Windows version is at http://gnuwin32.sourceforge.net/packages/diffutils.htm, and MacOS sources at http://perso.wanadoo.fr/gilles.depeyrot/DevTools_en.html.)
2. Make the change in your Cheetah CVS sandbox or in your installed version of Cheetah. If you make it in the sandbox, you'll have to run `python setup.py install` before testing it. If you make it in the installed version, do not run the installer or it will overwrite your changes!
3. Run `cheetah test` to verify you didn't break anything. Then run your little test program.
4. Repeat steps 2-3 until everything is correct.
5. Turn your little program into a regression test as described below.
6. When `cheetah test` runs cleanly with your regression test included, update the `CHANGES` file and check in your changes. If you made the changes in your installed copy of Cheetah, you'll have to copy them back into the CVS sandbox first. If you added any files that must be distributed, be sure to `cvs add` them before committing. Otherwise Cheetah will run fine on your computer but fail on anybody else's, and the test suite can't check for this.
7. Announce the change on the cheetahtemplate-discuss list and provide a tutorial if necessary. The documentation maintainer will update the Users' Guide and Developers' Guide based on this message and on the changelog.

If you add a directory to Cheetah, you have to mention it in `setup.py` or it won't be installed.

The tests are in the `Cheetah.Tests` package, aka the `src/Tests/` directory of your CVS sandbox. Most of the tests are in `SyntaxAndOutput.py`. You can either run all the tests or choose which to run:

  `python Test.py` Run all the tests. (Equivalent to `cheetah test`.)
  `python SyntaxAndOutput.py` Run only the tests in that module.
  `python SyntaxAndOutput.py CGI` Run only the tests in the class `CGI` inside the module. The class must be a direct or indirect subclass of `unittest_local_copy.TestCase`.
  `python SyntaxAndOutput.py CGI Indenter` Run the tests in classes `CGI` and `Indenter`.
  `python SyntaxAndOutput.py CGI.test1` Run only test `test1`, which is a method in the `CGI` class.
  etc...

To make a SyntaxAndOutput test, first see if your test logically fits into one of the existing classes. If so, simply add a method; e.g., `test16`. The method should not require any arguments except `self`, and should call `.verify(source, expectedOutput)`, where the two arguments are a template definition string and a control string. The tester will complain if the template output does not match the control string. You have a wide variety of placeholder variables to choose from, anything that's included in the `defaultTestNameSpace` global dictionary. If that's not enough, add items to the dictionary, but please keep it from being cluttered with wordy esoteric items for a single test).

If your test logically belongs in a separate class, create a subclass of `OutputTest`. You do not need to do anything else; the test suite will automatically find your class in the module. Having a separate class allows you to define state variables needed by your tests (see the `CGI` class) or override `.searchList()` (see the `Indenter` class) to provide your own searchList.

To modify another test module or create your own test module, you'll have to study the existing modules, the `unittest_local_copy` source, and the `unittest` documentation in the Python Library Reference. Note that we are using a hacked version of `unittest` to make a more convenient test structure for Cheetah. The differences between `unittest_local_copy` and Python's standard `unittest` are documented at the top of the module.