The SCHEMESTATION cross-compiler and assembler are used to produce binary byte code objects to be run in the SCHEMESTATION operating system simulator.
This document describes the plans to unit test the compiler/assembler system.
From here on, we see the compiler and the assembler as a conceptual unit that will be discussed together.
The compiler reads in source code (a dialect of Scheme) from a set of source files in a UNIX file system, and outputs one binary object file containing byte code instructions and constants that implement the compiled source code.
The overall structure of the compiler is presented in Fig 1.
The compiler as a whole can be decomposed into the following subparts that can be tested separaterly:
We now describe the testing plans for (i) the subparts 1-6 mentioned above and (ii) the compiler as a whole.
Testing objective: To ensure that the source code parser reads expression according to the R4RS document specifying a Scheme standard, with extensions from the SCHEMESTATION project.
Testing procedure: Test expressions are made by hand that are stored into a file. The expressions are read one by one by the parser. Then the same expressions are read in using the read procedure of the underlying Scheme that is used to implement the compiler, including the parser.
Checking results: A Scheme procedure is used to check that the two methods of reading have resulted in the same result in all test cases.
(Note. The procedure that is used to compare the two expressions is probably much like Scheme's equal?, but must take into account the different representations for the read expression by the two different methods.)
Testing objective: To ensure that (1) the macro expander produces semantically sound expansions of known derived expression types and (2) that expansion hygienicy is maintained.
Testing procedure: Test expressions are made by hand that are stored into a file. The expressions are chosen so that variable shadowing happens much in different ways in order to find hygienicy bugs. The expressions are read and then expanded, resulting in a semantic tree. The semantic tree is then evaluated according to its semantic using a specially built evaluator. The result of this evaluation is stored. Then the same expression is evaluated in the underlying Scheme. This test is dependent on the correct workings of the parser, so parser should be tested first.
Checking results: The results of the two evaluations are compared. The test is succesful if the results are the same or otherwise acceptable together. (Note. The last weakening might be necessary due to the fact that the underlying Scheme possibly cannot evaluate all constructs implemented in the SCHEMESTATION project.)
Testing objective: To ensure that the semantic optimizations applied to the semantic trees in the compiler preserve the semantics of the compiled expressions.
Testing procedure: As in "Macro expansion", but with optimizations performed.
Checking results: As in "Macro expansion".
Testing objective: To ensure that the linearizer transform semantic trees into byte code instruction sequences that implement the semantics of the trees.
Testing procedure: Test expressions are generated. Some of the expressions are made by hand and others generated automatically. The expressions are compiled and then run using the implemented SCHEMESTATION virtual machine. The resulting values are stored. The same expressions are then evaluated using an existing Scheme interpreter. Peephole optimization is disabled during this test. This test is dependent on the correct workings of the assembler, so assembler should be tested first.
Checking results: The two results of an expression are compared for equality, but see the note at the end of "Macro expansion".
Testing objective: To ensure that the peephole optimizer does not make unsound optimizations.
Testing procedure: As in "Linearizer", but with peephole optimization turned on.
Checking results: As in "Linearizer".
Testing objective: To ensure that the assembler creates byte code objects compliant with the SCHEMESTATION byte code object specifications.
Testing procedure: Test assembly files are assembled and then deassembled. Assembling is done using the assembler part of the compiler, and deassembling using an independently implemented deassembler.
Checking results: It is checked manually (or possibly using a script) that the deassembled code corresponds to the original assembly code.
After the subparts have been tested, the whole compiler is tested by compiling and running programs with known assumed behaviour. The same programs are compiled using all possible combinations of, or a reasonable subset of, parameters affecting the compilation. The results from running the programs are compared with the known behaviour.
Programs with errors generated by hand are compiled and the resulting error messages are checked for consistency. The compiler is also tested by a user that has not implemented the compiler as a user-friendliness check. Random rubbish is sent to the compiler in order to check that it does not choke on completely unpredictable input.