External Agent Interface Testing Plan

Document formatted by vherva at Fri Apr 24 11:23:39 1998 on the host schemestation. This document is produced by the SchemeStation project during the Tik-76.115 course.

1 Purpose
	This document describes the unit testing of the SCHEMESTATION external agent system. The purpose of the unit testing is to ensure, that the implemented external agent system system corresponds to the specified external agent protocol [External Agent Interface Specification] and that messages can be passed between the SCHEMESTATION domain and the external agents with reasonable performance.
2 Module testing
	Testing the overall functionality of the module is dicided in to following categories: The ssextagent library complies with the implemented external agent system. The external agent system is able to receive packet simultaneously from multiple external agent connections The external agent system is able to send packets. The external agent system is capable of listening the socket and accepting connections. It also has to be able to drop connection to peers not sustaining the SCHEMESTATION external agent protocol.
3 Function Testing
	Each function is tested separately. For each function the following is tested: Illegal arguments Functionality Capability to deal with malfunctional connection
4 Other tests
	The external agent system will be flooded with maximum amount of packets that the a test external agent can produce, and with packets that contain illegal headers.
5 The testing program
	A separate testing program will be written to test all the external agent operations. The testing involves having two test programs to act as minimal domains and communicate with each other via one test router. The two domains and the router are three instances of one executable, that acts as router, if a specific command line argument is supplied. The two client programs also get their own domain address, the address of the router and the address of the other client program as a command line argument.
5.1 The test program phases
5.1.1 External Agent system initialization
	The external agent system is initialized by calling extagent_initialize. A packet callback is supplied for the system using extagent_registerpacketcallback. Tehe callback dumps the gotten packet on to the screen. The separate client executable that uses the ssextagent lib is started to test connecting to a SCHEMESTATION domain. The result of the hand shake initiated between the server and the client will be checked using gdb. Also, a perl script will be written to try to connect to the router with false protocol. The router may neither accept the connections nor stop serving the valid connections.
5.1.2 Sending and receiving the packets
	In the sending phase, the client programbasicly executes a thight loop sending packets to the server program. The client program can be set in to this mode with eg. typing a suitable key combination. The result is verified by counting number of received messages and sent messages. The sending party can be instructed to dump the sent messages to stdout, too, so that the possible errors can be caught using diff(1).
5.1.3 Invalid packet flood
	Another of the client programs will be set to "malfunction" mode - it will flood the server with random packets. It will be checked that none of these packets cause denial of service neither for the malfunctional client nor the intact client. All invalid packets have to be ignored.
6 The results
	As the external agent system was both spceified and written quite late compared to the rest of the system, we chose to make the testing programs with care in the beginning. The tests begun right after all the functionality was coded. The approach was iterative; we executed the tests knowing that there would be bugs, located the bugs, fixed them and started over. This iteration stopped when the tested were succesfully passed. Due to the chosen approach, a huge number of bugs was found and terminated. The most serious and most difficult ones concerned having several high traffic external agents connected to the system in parallel. To summarize: the external agent system now has functions that have been tested to satisfy the following criteria: The functions de follow the specified semantics with legal arguments. The external agent is able to survive malfunctional connections. The external agent system handles flooding traffic without crashing. Connections not conforming to the external agent protocol are silently dropped.
6.1 Discussion
	The executed test were not even nearly perfect. Some bugs are likely to have passed our test cases. However, we expect to to find them in the integration testing, when we begin testing the tcp server and the file server.
6.2 Summary
	subject  status  result  comments  Do the functions implement the semantics defined in the spec?  done  passed   -  Do the functions survive calls with illegal arguments?  partly done  -   -  Stress test - flood the external agent system with maximum amount of packets   done  passed  -  Stress test - connect to the system with very many external agents in parallel   done  passed  -  Does the system cleanly drop invalid connections   done  passed  -
6.3 TCP server
6.4 Functions
	name  status  result  comments  extag_acceptcbk  done   success     extag_acceptconnection  done   success     extag_begindelivery  done   success     extag_closeconnection  done   success     extag_delete_packets  done   success     extag_destroypacket  done   success     extag_eloopcallback  done   success     extag_finishhandshake  done   success     extag_getport  done   success     extag_initconnection  done   success     extag_initialize  done   success     extag_packettoasc  done   success     extag_recvpacket  done   success     extag_send  done   success     extag_sendpacket  done   success     extag_seteventloopreqs  done   success     extag_shutdown  done   success     ssexact_blockingread  done   success     ssexact_close  done   success     ssexact_fd  done   success     ssexact_initialize  done   success     ssexact_read  done   success     ssexact_send  done   success