|
|
| 1 Purpose |
|
This document describes the testing phases inside the integration testing
phase. It divides integration tests to subsystem tests that can be
performed either one at a time during integration development or on
bigger chunks when integration advances fast. The target of these
tests is to pinpoint the bugs to as small a scope as possible.
To support this, even finer-grained methods can be used to pinpoint
faults in system. These methods can be implemented with options system
implemented in schemestation C includes. These options can be used to
turn certain features (for example, heap expansion or garbage
collection, or certain optimizations) on and off.
|
| 2 Heap-VM-Scheduler-Compiler Tests |
|
The heap, VM, scheduler and the compiler form the very core of the
system. Thus, the first tests performed to the system are:
- The agent linearisations can be loaded into the virtual machine,
- Incorrect linearisations are detected and refused (system
maintains stability),
- Linearisations can be run on the system. Behaviour of the system
under type errors, domain errors and all other
exception-generating errors must act according to the VM
specifications.
- Functionality of the system is tested using different Scheme test
programs (fibo, qsort, data structures test) and the validity of
results is checked by comparing them to results in other Scheme
implementations.
- The Scheduler is tested ; Addition and "concurrent" scheduling of
agents is tested. Exception behavior is checked with
multiple agents.
- Termination of agents (HALT) is tested.
|
| 3 Addressing-Messaging Tests |
|
The Addressing-messaging tests test that the heap, VM and compiler
co-operate with the address and messaging systems as specified,
messages can be sent and received to self and other agents, the
message queues function as specified, and the (internal) messaging
system doesn't drop the messages or route them to wrong agents.
Thses are mostly checked at the unit testing phase, but following
tests are performed to find potential errors in the integration phase:
- Different messages are sent between one (to self), two and several
agents, it is measured that all messages are correct and reach
the target agents.
- Behavior of messages sent to nonexistant agents are tested. These
messages should have no effect.
- System behavior with dying agents measured ; this should appear as
just ignoring the messages after death of the destination agent.
- The order relations of the messages is measured, if it is
predictable.
|
| 4 Networking Tests |
|
The networking tests are performed to test networking interface
functionality, system stability and potential limits of functionality
with all the systems tested above. Tests include:
- Addressing-messaging tests extended to the network environment
(which should be transparent operation involving only domain and
router network setup),
- Cross-architectural compability testing (Alpha vs. Intel
vs. HPPA),
|
| 5 Domain kernel agent |
|
The domain kernel agent is the topmost authority in a SCHEMESTATION
domain. After domain kernel agent implementation, following tests are
performed:
- The operations defined at OS definition 4.10 are tested.
- The operations are tested with illegal values.
- Migration of agents from domain to another is tested. Migration
rules defined in OS definition are tested and the erratic
conditions are tested (migration to nonexistent
domain). Migration is tested between different architectures to
ensure platform independence.
- As always, stress and stability tests are performed.
|
| 6 Scheduler and addressing agents |
|
The scheduler and addressing agents are tested the same way as the
domain kernel agent.
|
| 7 Results |
|
The results map one-to-one with the plans above.
|
| 7.1 Totals |
|
| Name |
Done |
Succesful |
Total |
| Heap-VM-Scheduler-Compiler Tests |
6 (100%) |
5 (83%) |
6 |
| Addressing-Messaging Tests |
4 (100%) |
4 (100%) |
4 |
| Networking Tests |
2 (100%) |
2 (100%) |
2 |
| Domain kernel agent |
4 (100%) |
4 (100%) |
4 |
|
| 7.2 Heap-VM-Scheduler-Compiler Tests |
|
Test cases (first part: (D)one, (U)ndone, (I)ncomplete,
second part: (S)uccess, (U)nsuccess, (I)ncomplete (F)atal):
| Name |
Status |
Result |
Comments |
| The agent linearisations can be loaded into the virtual machine |
D |
S |
| Incorrect linearisations are detected and refused |
D |
I/U/F |
Incorrect linearisations can crash the heap. |
| Linearisations can be run on the system. |
D |
S |
| Functionality of the system is tested using different Scheme test programs. |
D |
S |
| The Scheduler is tested |
D |
S |
| Termination of agents (HALT) is tested. |
D |
S |
|
| 7.3 Addressing-Messaging Tests |
|
| Name |
Status |
Result |
Comments |
| Different messages are sent between one (to self), two and several
agents. |
D |
S |
| Behavior of messages sent to nonexistant agents are tested. |
D |
S |
| System behavior with dying agents measured |
D |
S |
| The order relations of the messages is measured |
D |
S |
|
| 7.4 Networking Tests |
|
| Name |
Status |
Result |
Comments |
| Addressing-messaging tests extended to the network environment |
D |
S |
| Cross-architectural compability testing (Alpha vs. Intel), |
D |
S |
(intel, alpha) |
|
| 7.5 Domain kernel agent |
|
| Name |
Status |
Result |
Comments |
| The operations defined at OS definition 4.10 are tested. |
D |
S |
| The operations are tested with illegal values. |
D |
S |
| Migration of agents from domain to another is tested. |
D |
S |
| As always, stress and stability tests are performed. |
D |
S |
|