Tik-76.115 Test Report, v5.1
Group: Hayabusa, Work: Monrovia, Responsible: Jarmo Mäki
Last modified 2001-04-23 - Jarmo Mäki
Change log
- 2001-04-23 Proofread the report
- 2001-04-21 Added several reports of performed tests
- 2001-04-20 Added several reports of performed tests
- 2001-03-19 Added log
- 2001-03-16 Added several reports of performed tests
- 2001-03-11 Added usability and performance tests
- 2001-02-13 Added the separate module testing reports
- 2001-02-11 modified the layout
- 2001-02-25 added test cases
- 2000-01-07 First version
1. Introduction
The purpose of this report is to describe the test cases made in T3 phase,
when the tests were carried out, who performed the tests, how long did it
take to perform the tests, what the result of the tests was and if the
test was passed or not. All bugs and change requests must also be reported
into Burana so that the responsible persons will be notified by email. In
chapter 7 one can find the result of all tests. The tests are divided into
module, integration and system testing. The test report follows the
definitions specified in the test plan.
1.1 About module tests
Module testing is divided into server, client and protocol tests. These
tests will be performed only by code writers. Each module test is approved
by code writers. The whole module testing is approved by the project manager
who scans the test reports carefully and verifies that the tests were
performed precisely enough.
1.2 About integration tests
Integration testing assures that the server-protocol-client combination
works without problems. The integration of the whole system into the demo
environment is also reported here. The server-protocol-client combination
will be approved by code writers and the integration of the whole system
into the demo environment will be approved by the dedicated test personnel.
The whole integration testing will be approved by the project manager who
scans the test reports carefully and verifies that the tests were performed
precisely enough.
1.3 About system tests
System testing includes testing the functional features, usability and
performance. The functional testing is divided into testing the movement of
the character, player commands, administrator commands. System testings
will be verified by the project manager.
2. Module testing
2.1 Server
Test: 2.1.1
Test Case: Server module test
Date: 2001-01-31
Participants: Ilpo Nyyssönen
Execution: The module compiles and starts up
Duration: 20 minutes
Result: It compiled without errors or warnings and starts up
cleanly. It also seems to listen the tcp and udp ports.
Grade: A
Passed: Yes
2.2 Client
Test: 2.2.1
Test Case: Compilation and running of the code
Date: 2001-02-10
Participants: Anssi Kanninen
Execution: -
Duration: Several hours
Result: Compiling is successfull and there are no warnings. The game
starts with no error messages.
Grade: A+
Passed: Yes
Test: 2.2.2
Test Case: Connecting user interface
Date: 2001-02-10
Participants: Anssi Kanninen
Execution: -
Duration: 10 times
Result: Connecting is successfull and the game starts.
For some reason it takes longer with Windows-client than with the Linux KVM.
Grade: A
Passed: Yes
Test: 2.2.3
Test Case: Game UI updating
Date: 2001-02-10
Participants: Anssi Kanninen
Execution: -
Duration: 10 times
Result: Debug messages show that once the map has been drawn,
the updating works well. Map is updated and messages are written if
there are some.
Grade: A+
Passed: Yes
Test: 2.2.4
Test Case: Executing commands
Date: 2001-02-10
Participants: Anssi Kanninen
Execution: -
Duration: 10 times
Result: When a command has been executed, the debug messages
show that they are sent successfully. There's still some misbehaviour with
the menu commands.
Grade: A
Passed: Yes
Test: 2.2.5
Test Case: Quitting the game
Date: 2001-02-10
Participants: Anssi Kanninen
Execution: -
Duration: 10 times
Result: The game quits well without error messages.
Grade: A
Passed: Yes
Result of module testing:
3. Integration testing
Test: 3.1.1
Test Case: Server/Client
Date: 2001-02-01
Participants: Jarmo Mäki
Performance: This test was done using the log files which both
modules write. All commands were sent by the client and recieved by the
server properly. The server sent updates to the client every 1 sec or so
and the client updated the GUI as rapidly.
Duration: 1h
Result: The server module co-operated with the client module
perfectly
Passed: Yes
Test: 3.1.2
Test Case: Integrating the system with equipment used in demo
(Not yet carried out)
Date:
Participants:
Performance:
Duration:
Result:
Passed:
Result of integration testing: Passed
4. Functional testing
4.1. Movement
Test: 4.1.1a
Test Case: Common movement
Date: 2001-02-05
Participants: Jarmo Mäki
Performance: The test was performed with the Palm emulator. I
moved around on the map and tried to find a bug in the movement, but no
bug was found. Moving worked fine and the buttons used for moving worked
once in a turn.
Duration: 10min
Result: Worked fine
Grade: A+
Passed: Yes
Test: 4.1.1b
Test Case: Common movement
Date: 2001-02-08
Participants: Juha T. Vainio
Performance: Common movement works well and is easy to use. It
is a bit lagging sometimes when a slight pause is taken between issuing
movement commands. When it usually takes about a second for the dude to
move, sometimes it takes even three. Otherwise all is as it should be, and
the lagging is not that disturbing.
Duration: 5 min
Result: Ok
Grade: A
Passed: Yes
Test: 4.1.1c
Test Case: Common movement
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: The test was performed with Windows Palm emulator. I moved around and everything seemed to be OK.
Duration: 15min
Result: OK
Grade: A+
Passed: Yes
Test: 4.1.1d
Test Case: Common movement
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: The test was performed with Windows Palm emulator.
Moving around works nicely and new arrow buttons are handy.
Duration: 10min
Result: OK
Grade: A+
Passed: Yes
Test: 4.1.1e
Test Case: Common movement
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux Palm emulator.
I walked around the arena and there were no problems at all.
Duration: 5 min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.1.1f
Test Case: Common movement
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: The test was performed with the Palm emulator. I
moved around on the map and tried to find a bug in the movement, but no
bug was found. Moving worked fine and the buttons used for moving worked
once in a turn.
Duration: 10min
Result: Worked fine
Grade: A+
Passed: Yes
Test: 4.1.2a
Test Case: Moving to a square where already is a static/dynamic
object
Date: 2001-02-08
Participants: Juha T. Vainio
Performance: At this point there are no dynamic objects other
than the player itself, so this is tested purely on static objects. This
works very well. When the dude is trying to work its way to a square which
already occupies an object, server notices this and acts accordingly.
Duration: 5 min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.1.2b
Test Case: Moving to a square where already is a static/dynamic
object
Date: 2001-02-10
Participants: Jarmo Mäki
Performance: This test was done using the Linux and Windows Palm
emulators. The Linux emulator was used locally on the server and the Windows
emulator was used on my home computer. I played the game with two players at
the same time. Players were not able to move to the same square that already
has a static or dynamic object (i.e. tree, mountain, other player) and the
message "You meet client1" etc was displayed. If I tried to walk out of the
map "You won't jump" was displayed and the character didn't leave the map
area.
Duration: 15min
Result: Worked ok
Grade: A+
Passed: Yes
Test: 4.1.2c
Test Case: Moving to a square where already is a static/dynamic
object
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: This test was done using windows palm emulator. When moving to a square where is a static object a message "You meet " is displayed and the character doesn't move to that square. When moving to a square where is a dynamic character (npc or pc) the character attacks the other one.
Duration: 10 min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.1.2d
Test Case: Moving to a square where already is a static/dynamic
object
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: This test was done using windows palm emulator.
Worked nicely. You meet mountain, you meet tree or something appears
when you try to move to a square which already occupies a static object.
When moving to a dynamic object, attacking begins accordingly. No problems.
Duration: 10 min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.1.2e
Test Case: Moving to a square where already is a static/dynamic object
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux Palm emulator.
This worked as it was supposed to so that when trying to move to a square with
a mountain or a fire ant it did something else than just basic movement.
Duration: 5 min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.1.2f
Test Case: Moving to a square where already is a static/dynamic
object
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: This test was done using the Linux and Windows Palm
emulators. The Linux emulator was used locally on the server and the Windows
emulator was used on my home computer. I played the game with two players at
the same time. Players were not able to move to the same square that already
has a static or dynamic object (i.e. tree, mountain, other player) and the
message "You meet client1" etc was displayed. If I tried to walk out of the
map "You won't jump" was displayed and the character didn't leave the map
area.
Duration: 15min
Result: Worked ok
Grade: A+
Passed: Yes
Test: 4.1.3a
Test Case: Many players or NPCs move to the same square
Date: 2001-02-10
Participants: Jarmo Mäki
Performance:This test was done using the Linux and Windows Palm
emulators. The Linux emulator was used locally on the server and the Windows
emulator was used on my home computer. I played the game with two players at
the same time. When two players tried to move to the same square in the same
turn I got "You hesitate" message for one player and only the other player
moved to the square.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.1.3b
Test Case: Many players or NPCs move to the same square
Date: 2001-03-15
Participants: Jarmo Mäki
Performance:This test was done with windows and linux emulators. I tried to move two players to the same square but I got an answer "you hesitate" for one player and only the other player moved to that square. When I tried to move to the same square where a npc was just moving to it seemed that the only the pc could move to that square.
Duration: 15min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.1.3c
Test Case: Many players or NPCs move to the same square
Date: 2001-03-16
Participants: Juha T. Vainio
Performance:This test was done with two windows palm emulators.
For this test, I used two player characters and on the same turn moved them
towards the same square. Another one moved correctly to the square, the other
one hesitated and did not move. Nicely designed and implemented and worked
flawlessly.
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.1.3d
Test Case: Many players or NPCs move to the same square
Date: 2001-04-20
Participants: Oskari Mertalo
Performance:The test was performed with the Linux Palm emulator.
I tried to move to a square already occupied by a fire ant and got a message
'You attack Fire ant'.
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.1.3e
Test Case: Many players or NPCs move to the same square
Date: 2001-04-21
Participants: Jarmo Mäki
Performance:This test was done using the Linux and Windows Palm
emulators. The Linux emulator was used locally on the server and the Windows
emulator was used on my home computer. I played the game with two players at
the same time. When two players tried to move to the same square in the same
turn I got "You hesitate" message for one player and only the other player
moved to the square.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Movement Result: Passed
Acceptor: Juha Vainio
4.2. Player Commands
Test: 4.2.1a
Test Case: War Cry
Date: 2001-02-05
Participants: Jarmo Mäki
Performance: War Cry is a rather simple command and there wasn't
much to test. I tried to push the button as fast as I could, but the button
still sent each time a message to the server. The output was "You cry 'I'll
kill you all!" which was received once in a turn. If there was a player
somewhere near then the other player could hear the first player cry
"client2 cries 'I'll kill you all'".
Duration: 5min
Result: worked fine
Grade: A+
Passed: Yes
Test: 4.2.1b
Test Case: War Cry
Date: 2001-02-08
Participants: Juha T. Vainio
Performance: War cry functionality works well. When the button
is pushed, server notices this and sends the proper reply, in this case a
remark "You cry 'I'll kill you all!'". The button is clear and so is the
text appearing to the text screen.
Duration: 5 min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.1c
Test Case: War Cry
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: Test was done the same way as in 4.2.1a. Same results.
Duration: 5min
Result: worked fine
Grade: A+
Passed: Yes
Test: 4.2.1d
Test Case: War Cry
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: Still fails to disappoint every time
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.1e
Test Case: War Cry
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux Palm emulator.
Clicking on this displayed the hideous war cry in the message window. Doing
this several times during one turn resulted in only one war cry so it works
perfectly.
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.1f
Test Case: War Cry
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: Test was done the same way as in 4.2.1a. Same results.
Duration: 5min
Result: worked fine
Grade: A+
Passed: Yes
Test: 4.2.2a
Test Case: Say
Date: 2001-02-05
Participants: Jarmo Mäki
Performance: This test was performed with the Palm emulator.
The command works fine execpt in the following situation: if you push say
button and if after that the server disconnects you so that you did not
have time to push "OK" button in the "say" menu then the cursor keeps
blinking on the screen. I don't know how this looks on a real Palm, but on
the emulator it's annoying. Also there should be a way to exit the "Say"
dialog box without saying anything. The Say command didn't accept any
scandinavian characters (sniff). A player nearby could hear if someone said
something.
Duration: 15min
Result: A little problem with the cursor. No cancel button.
Grade: A
Passed: Yes
Test: 4.2.2b
Test Case: Say
Date: 2001-02-08
Participants: Juha T. Vainio
Performance: Say works well at first, but after issuing the
command four times in a row, the game gets into a strange state showing
a cursor and part of the say box in the game area. This may be a bug in
the emulator. In addition, it may be considered as a non-wanted feature
that the contents of the last message remains in the say-box. With Palm,
it is not very easy to erase the text, so the cleaning of the text box
should at least be considered carefully. The absence of 'cancel' button
is not good.
Duration: 10 min
Result: Cancel button missing, last message should be erased,
the screen is not refreshed properly
Grade: A-
Passed: Yes, but not very well
Test: 4.2.2c
Test Case: Say
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: This test was performed with linux and windows Palm emulators. I tried with several techniques (shutting down the server, unexpected things) to make the say command fail, but didn't succeed. This command seems to work fine now.
Duration: 15min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.2d
Test Case: Say
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: This test was performed with windows Palm emulator.
Say works very well and the minor bugs have been fixed. Now the dialog
box is empty and saying can be canceled. You can also say nothing,
which probably is not very useful. On the other hand, it is not very fatal,
either.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.2e
Test Case: Say
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux Palm emulator.
This feature was very nice and worked good. Only sometimes the message
'You say:...' appeared twice in the message window but when you scrolled back
and forth you could only see one of those messages. Strange?
Duration: 5 min
Result: Ok
Grade: A
Passed: Yes
Test: 4.2.2f
Test Case: Say
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: see 4.2.2c
Duration: 15min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.3a
Test Case: Log in
Date: 2001-02-15
Participants: Jarmo Mäki
Performance: I logged in 20 times and there was no problem. I
also tried to use wrong passwords and username and in those cases I got
"authentication failed" which was expectable. I performed the test on
localhost using Palm emulator. If the player had already logged in then
nobody else could log in using the same name.
Duration: 10min
Result: OK
Grade: A+
Passed: Yes
Test: 4.2.3b
Test Case: Log in
Date: 2001-02-08
Participants: Juha T. Vainio
Performance: When logging in the first time, it works well.
Although typing in the server's name could be considered difficult with
Palm's Graffiti system. Cannot be avoided though, if there are no
possibilities to save the server information on Palm. When logging in the
second time, without closing the Palm emulator, a dialog box appears on the
screen with an OK-button. When pushed, the OK-button sends you back to to
the connecting screen. You have to exit the emulator in order to log in
again.
Duration: 10 min
Result: The emulator had to be quitted before new connection was
possible.
Grade: B
Passed: No
Test: 4.2.3c
Test Case: Log in
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: This test was done with linux and windows palm emulators. When logging in the first time, everything is fine. The game stores player information such as location and HP. Logging after death is also possible, the player just has negative HP so that he has to regenerate HP before been attacked by anyone or else he will die again. Some times the windows emulator doesn't let the player to log in, but I think that bug is caused by the windows emulator and not by us.
Duration: 15min
Result: OK
Grade: A+
Passed: Yes
Test: 4.2.3d
Test Case: Log in
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: This test was done with windows palm emulators.
There are no problems with login. Player identity is checked correctly,
no refresh problems, etc. The windows emulator however seems to lose the
connection to the internet after you have restarted the game several times.
Not the problem of ours, though.
Duration: 10min
Result: OK
Grade: A+
Passed: Yes
Test: 4.2.3e
Test Case: Log in
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux Palm emulator.
Logging in worked like logging in to any system. Didn't have any problems.
Duration: 5 min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.3f
Test Case: Log in
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: see 4.2.3c
Duration: 15min
Result: OK
Grade: A+
Passed: Yes
Test: 4.2.4a
Test Case: Log out
Date: 2001-02-05
Participants: Jarmo Mäki
Performance: The test was done with Palm emulator. I tried to
log out by using the "Disconnect" option in the menu and by closing the
emulator. If the server was shutdown (i.e. kill -9) the client did react
properly to the lost connection.
Duration: 15min
Result: OK
Grade: A+
Passed: Yes
Test: 4.2.4b
Test Case: Log out
Date: 2001-02-08
Participants: Juha T. Vainio
Performance: Logging out is done by choosing another program to
use in Palm, as the 'quit' feature is usually implemented in Palm, or by
choosing 'disconnect' in the menu bar. Both work well, but the game cannot
be logged in again after this (as explained in the 'log in' section). From
the server's point of view, the player has not quit the game when he chooses
to leave without 'disconnect' from the menu. Server is up and running,
sending updates to the player that exists no more. There could be a 'cancel'
button also in 'disconnect' dialog.
Duration: 10min
Result:
Grade: A-
Passed: Yes
Test: 4.2.4c
Test Case: Log out
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: Done with Windows and linux palm emulators.
Log out using disconnect works fine. Server has disconnected works also fine. Test was basicly done the same way as 4.2.4a
Duration: 15min
Result: OK
Grade: A+
Passed: Yes
Test: 4.2.4d
Test Case: Log out
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: Done with Windows and linux palm emulators.
No problems in logging out with disconnect. However, when the game is
'left to the background', eg. left without disconnecting, the session
is not disconnected and the player is still in the game.
Duration: 10min
Result: OK
Grade: A-
Passed: Yes
Test: 4.2.4e
Test Case: Log out
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: see 4.2.4a
Duration: 15min
Result: OK
Grade: A+
Passed: Yes
Test: 4.2.5a
Test Case: Menu Bar
Date: 2001-02-02
Participants: Jarmo Mäki
Performance: The test was done with Palm Emulator. The Menu bar
has only few options so there wasn't much to test. Though the menu was
simple, there were some features that I didn't like. First of all, the menu
seemed to remember the last status (I open the menu -> choose "About" -> the
menu closes automatically -> when I open the menu the next time, "About" is
highlighted). There's nothing wrong with that except in the following
situation: I open the menu -> choose disconnect -> log in -> open the menu
again -> disconnect is automatically highlighted -> server i.e. crashes in
the meantime or closes connection -> you are thrown out of the game -> I
log in once more when server is up & running again -> push anywhere with
the pencil and I'm thrown out again. Sniff. (The menu never closed when I
was thrown out). It's not a nice feature, that when the menu is open you can
push with the pencil anywhere on the screen (except the other options) and
it executes the one which is highlighted. Last, but not least: I wasn't able
to close the menu without choosing something.
Duration: 15min
Result: minor problems with the menu
Grade: A-
Passed: Yes
Test: 4.2.5b
Test Case: Menu Bar
Date: 2000-02-08
Participants: Juha T. Vainio
Performance: Menu bar seems to work appropriately at first. Then
problems arise. The menu bar appears only partly on the screen and when East
button is pushed, it acts as the player wants to disconnect. Could be at
least partly the fault of the emulator. Only partly, because even when the
menu is shown correctly, it cannot be exited by tapping something outside
the menu. This should be corrected, so that disconnection is not always
imminent.
Duration: 10min
Grade: A-
Passed: Yes
Test: 4.2.5c
Test Case: Menu Bar
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: Test done using linux emulator. The bugs found in the last testings are fixed. The only lousy feature that I could find was that if you have the menu bar open when the server disconnects you then the menu doesn't actually close automatically. So, when you login the next time you have to push one time on the screen to be able to interact. The effect of this feature is very small.
Duration: 10min
Grade: A
Passed: Yes
Test: 4.2.5d
Test Case: Menu Bar
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: Test done using windows emulator.
Menu bar works well and does what it is told. All is well.
Duration: 5min
Grade: A+
Passed: Yes
Test: 4.2.5e
Test Case: Menu Bar
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: Same results as in 4.2.5c
Duration: 10min
Grade: A
Passed: Yes
Test: 4.2.6a
Test Case: Quit button
Date: 2001-02-05
Participants: Jarmo Mäki
Performance: Test was done with Palm Emulator. The quit button
in the login screen quitted the game as expected. The Windows Palm Emulator
had some problems to reconnect, but that was only a problem caused by the
Emulator.
Duration: 2min
Result: worked fine
Grade: A+
Passed: Yes
Test: 4.2.6b
Test Case: Quit button
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: Test was done with Palm Emulator. The quit button
in the login screen quitted the game as expected. The Windows Palm Emulator
had some problems to reconnect, but that was only a problem caused by the
Emulator.
Duration: 3min
Result: worked fine
Grade: A+
Passed: Yes
Test: 4.2.6c
Test Case: Quit button
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: see 4.2.6b
Duration: 3min
Result: worked fine
Grade: A+
Passed: Yes
Test: 4.2.7a
Test Case: Stats
Date: 2001-02-05
Participants: Jarmo Mäki
Performance: There were several problems with the Stats menu.
The fields in the stats menu seemed to increase everytime I logged in to
the game. If I had logged in twice (log in - > log out -> log in) I got
twice both ST and WISDOM, if I had logged in three times I got ST and
WISDOM also three times. Another problem was that when I first opened the
menu bar and had "disconnect" highlighted and then opened the stats and
pressed OK the GUI got broken (open menu and make sure disconnect is
highlighted -> then open the stats -> press OK -> GUI gets broken). When
the GUI gets broken I'm not anymore able to see the map, the text area is
not updated and "disconnect" in the menu doesn't work which means that
I'm not able to quit the game properly.
Duration: 20min
Result: worked but not perfect
Grade: A-
Passed: Yes
Test: 4.2.7b
Test Case: Stats
Date: 2001-02-08
Participants: Juha T. Vainio
Performance: Stats button works nicely.
Duration: 1 min
Result: A+
Passed: Yes
Test: 4.2.7c
Test Case: Stats
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: Stats button works nicely.. I'm level 9 already... :)
Duration: 8,5 min
Result: A+
Passed: Yes
Test: 4.2.7d
Test Case: Stats
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: Stats button works nicely... Still at level 6, but
I have removed and added myself several times. :)
Duration: 5 min
Result: A+
Passed: Yes
Test: 4.2.7e
Test Case: Stats
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux palm emulator.
I was able to scroll the stats normally and close the stats window after
that. Good work.
Duration: 5 min
Result: A+
Passed: Yes
Test: 4.2.7f
Test Case: Stats
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: Stats button works (I used Windows Emulator)
Duration: 5 min
Result: A+
Passed: Yes
Test: 4.2.8a
Test Case: Text Area
Date: 2001-02-05
Participants: Jarmo Mäki
Performance: I used all commands to see if the text area was
updated properly. The commands seemed to work fine (everytime I pushed a
button, the client log showed that one command was send and the server log
showed that one command was received). The problem was that after a while
the text area started to show some commands twice. First when I log in to
the game, everything works fine. When I use lots of commands in a row and
make a lot of spam, the text area starts to show some commands twice (after
5 min or so). When the text area is broken, the only way to fix it is to
disconnect the game and log in again.
Duration: 20min
Result: Ok
Grade: A-
Passed: Yes
Test: 4.2.8b
Test Case: Text Area
Date: 2001-03-15
Participants: Jarmo Mäki
Performance: This test was done using windows palm emulator. Some of the bugs found in the last testing are fixed, but still the text scrolls wierd so that some messages are shown twice. Sometimes some messages are not shown, but that's because the server cannot send all messages to the client (that's something which has been included in the server package on purpose). It would be nice to see all important messages (such as you die)
Duration: 10min
Result: Not all important messages are shown.
Grade: A
Passed: Yes
Test: 4.2.8c
Test Case: Text Area
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: This test was done using windows palm emulator.
Text area works well, but there is a problem with the scrollbar. The location
indicator is not always at the bottom of the bar when it should be. Probably
a bug in the UI library functions.
Duration: 10min
Result: Ok
Grade: A
Passed: Yes
Test: 4.2.8d
Test Case: Text Area
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux Palm emulator.
The text area was pretty ok if you take into consideration how little space
there was for it. Some messages seemed to appear there twice (at least Say)
but other than that it worked and was informative.
Duration: 10 min
Result: Ok
Grade: A
Passed: Yes
Test: 4.2.8d
Test Case: Text Area
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: The test was performed with the Windows Palm emulator.
The text area was pretty ok if you take into consideration how little space
there was for it. Some messages seemed to appear there twice (at least Say)
but other than that it worked and was informative.
Duration: 10 min
Result: Ok
Grade: A
Passed: Yes
Test: 4.2.9a
Test Case: Look
Date: 2001-03-15
Participants:Jarmo Mäki
Performance: This test was done using Windows and Linux emulators. The command works nicely and you can even use it to check in what condition your enemy is. I tested this command many times while I played the game. If there are no dynamic objects nearby then the command shows a "You see nobody" message.
Duration: 10min
Result: works nicely
Grade: A+
Passed: Yes
Test: 4.2.9b
Test Case: Look
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: This test was done using Windows emulator.
Look seems to function well, but the coordinates given for dynamic objects
are not the standard XY coordinates. The X axis is correctly from west
to east, but the Y axis is from north to south. Not a big problem, but
may be confusing if several dynamic objects are seen.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.9c
Test Case: Look
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux Palm emulator.
Look noticed all interesting objects on the screen but the Y coordinates were
confusing because they were not what people have used to.
Duration: 10 min
Result: Ok
Grade: A
Passed: Yes
Test: 4.2.9d
Test Case: Look
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: The test was performed with Windows Palm Emulator. Look seems to work, but the coordinates are a bit confusing.
Duration: 10 min
Result: Ok
Grade: A
Passed: Yes
Test: 4.2.10a
Test Case: Changing Arena
Date: 2001-03-15
Participants: Jarmo M&aulm;ki
Performance: This test was done using Windows Emulators. Changing the arena seemed first to work fine, but when I added 9 zimbas to toilet arena (there were 9 empty squares) and then tried to enter that arena, there was no empty square and Palm failed to draw the map. I was still able to disconnect the game and after checking the stats Palm updated the map, but I couldn't move or do anything.
Duration: 15min
Result: if no empty square -> bug
Grade: B
Passed: No
Test: 4.2.10b
Test Case: Changing Arena
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: This test was done using Windows Emulator.
Changing the arena works well with the standard game (without adding
zimbas :)), although the "disconnected from the server" message is a bit
distracting.
Duration: 10min
Result: Ok
Grade: A
Passed: Yes
Test: 4.2.10c
Test Case: Changing Arena
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux Palm emulator.
Nothing at all to complain about.
Duration: 5 min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.10d
Test Case: Changing Arena
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: The test was performed with the Linux Palm emulator.
Could find anything strange.
Duration: 5 min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.11a
Test Case: Combat
Date: 2001-03-15
Participants: Jarmo Mäki
Performance:This test was done using Windows and Linux Emulators. I was able to fight npc's and pc's and everything seemed to be ok. The HP's were regenerated properly and my stats advanced. I played the game for about 4h and during that time I couldn't find any bug in the combat. The fact, that not all messages are shown is quite disturbing if you die and if the server doesn't send "You die" message, but that problem is already described in 4.2.8b.
Duration: 4h
Result: ok
Grade: A+
Passed: Yes
Test: 4.2.11b
Test Case: Combat
Date: 2001-03-16
Participants: Juha T. Vainio
Performance:This test was done using Windows Emulator.
Combat works very well every time. At least once I succeeded in killing
both of the participants of the fight, which was nice.
Duration: 30min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.2.11c
Test Case: Combat
Date: 2001-04-20
Participants: Oskari Mertalo
Performance: The test was performed with the Linux Palm emulator.
Every aspect of the combat worked; attacking, running away, winning, losing.
The only thing that bothered me was when finally killing a nasty Fire ant I
got these messages (and in this order): 'Osku killed Fire ant', ' You attack
Fire ant'. A minor thing but still confusing. Might get more confusing while
in combat with multiple opponents.
Duration: 20 min
Result: Ok
Grade: A
Passed: Yes
Player Commands Result: Accepted
Acceptor:Juha Vainio
4.3. Administrator Commands
Test: 4.3.1a
Test Case: Starting the Server
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: No problems whatsoever in starting the server. If
a server was already running in the port in question, the server informed
with an error message. If no server was running, the server started
successfully every time.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.3.2a
Test Case: Shutting down the Server
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: No problems in shutting the server down, either.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.3.2b
Test Case: Shutting down the Server
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: No problems in shutting the server down.
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.3.3a
Test Case: Adding player characters
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: Adding player characters with the command line
tool was very easy and worked well. If a wrong type of data was entered,
the tool gave an error message and aborted. Of course it would be better
if the tool would ask again instead of aborting the whole process.
Checking of existing characters and arenas worked well, too.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.3.3b
Test Case: Adding player characters
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: see 4.3.3a
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 4.3.4a
Test Case: Removing player characters
Date: 2001-03-16
Participants: Juha T. Vainio
Performance: Removal of player characters was also very easy.
But I think there should be an error message if a non-existent player
is tried to remove. If there is a typo in the name, the administrator
does not necessarily notice that the removal fails.
Duration: 10min
Result: Ok
Grade: A
Passed: Yes
Test: 4.3.4b
Test Case: Removing player characters
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: see 4.3.3a for futher information
Duration: 8min
Result: Ok
Grade: A
Passed: Yes
Test: 4.3.5a
Test Case: Kicking player characters out of the game
Date: 2001-04-21
Participants: Jarmo Mäki
Performance: Works fine and you cannot kick players that are not playing the game.
Duration: 8min
Result: Ok
Grade: A
Passed: Yes
5. Usability testing
Test personnel:
Sebastian Friberg:
Studying computer science at Arcada Polytechnic
Software specialist at Tele1 Europe
Very good skills in computer science
No previous knowledge of Monrovia project, but has played Mud games
Elina Salonen:
Studying dentistry at Helsinki University
Average skills in Windows Environment
No previous knowledge of Monrovia project or mud games
Jarmo Mäki:
Member of Hayabusa group, supervised the tests
Juha Vainio:
Member of Hayabusa group, verified that the tasks were sensible
5.1 General impression of the game
Test: 5.1.1a
Test Case: Have the limitations of the Palm taken into account?
Date: 2001-03-17
Participants: Juha T. Vainio (unfortunately has previous
experience and inside knowledge of the game)
Performance: Limitations such as the size of the display and
playing the game using the pen-like stylus have been taken into account
nicely. The fighting is done without constant interaction (hit-hit-hit),
which is especially good thing.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.1.1a
Test Case: Have the limitations of the Palm taken into account?
Date: 2001-03-18
Participants: Elina Salonen and Sebastian Friberg
Performance: There were some features which the test persons
didn't like, such as too short messages. They would have wanted to have
more messages, but the Palm display is rather small and therefore only
short messages can be displayed. This limitation could have been taken
better into account.
The navigation bar was easy to use and the commands were very handy.
They liked the way the display was divided into several smaller boxes.
Writing couldn't be tested properly, because only an emulator was
available.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.1.2a
Test Case: Intelligibility of the Game/ is the GUI easy to adopt
Date: 2001-03-17
Participants: Juha T. Vainio (unfortunately has previous
experience and inside knowledge of the game)
Performance: The GUI
is very easy to adopt, as it is simple and functions are well illustrated.
The purpose in the game is also very clear (kill everything that moves and
some of the things that do not move :))
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.1.2b
Test Case: Intelligibility of the Game/ is the GUI easy to adopt
Date: 2001-03-18
Participants: Elina Salonen and Sebastian Friberg
Performance: The GUI was very easy to adopt. Sebastian, who is
very familiar with games like Monrovia, would have been able to play the
game without any help. Elina had never played Mud games before so I first
had to tell her some background information, but she too learned to use
the GUI very quickly.
Duration: 15min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.1.3a
Test Case: Is the game illustrative (are the objects clear etc)?
Date: 2001-03-17
Participants: Juha T. Vainio (unfortunately has previous
experience and inside knowledge of the game)
Performance: Objects are very clear. Except perhaps Zimba, but
as she represents an ugly orc, it is forgivable. Trees, mountains, etc, are
nicely done and there is no possibility of confusion
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.1.3b
Test Case: Is the game illustrative (are the objects clear etc)?
Date: 2001-03-18
Participants: Elina Salonen and Sebastian Friberg
Performance: The objects are very descriptive. Comments: The
area changing square should have an icon or else it's hard to understand
what exactly happens when you change area the first time. Elina said that
Zimba didn't look ugly but cute. Mountains and trees were easy to identify.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.1.4a
Test Case: Games plot / does the game advance meaningfully
Date: 2001-03-17
Participants: Juha T. Vainio (unfortunately has previous
experience and inside knowledge of the game)
Performance: There is no clear plot, and game does not advance
at all. It's just killing fire ants and other players.
Duration: 10min
Result: Ok
Grade: A
Passed: Yes
Test: 5.1.4b
Test Case: Games plot / does the game advance meaningfully
Date: 2001-03-18
Participants: Sebastian Friberg and Elina Salonen
Performance: Well, the game doesn't actually have a sensible
plot. Walking around and killing ants is after a while boring.
Duration: 10min
Result: Ok
Grade: A
Passed: Yes
Test: 5.1.5a
Test Case: Does the game provide motivation?
Date: 2001-03-17
Participants: Juha T. Vainio (unfortunately has previous
experience and inside knowledge of the game)
Performance: Killing your friends in a game is always fun
and motivating. :)
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.1.5b
Test Case: Does the game provide motivation?
Date: 2001-03-18
Participants: Sebastian Friberg and Elina Salonen
Performance: Multiplaying is always motivating. Elina
actually excited a bit when someone else (someone from Hayabusa who
was testing the game) by accident came near her. Chatting with
someone who you never have met before is also fun. It's a bit
disappointing that the game it self is so compact.
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
5.2 Tasks for test persons
Test: 5.2.1a
Test Case: Log into the game (tell him the name of the server,
username, password and port)
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Writing the whole server name, player name and
password are slow to write with the stylus, but otherwise all went well
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.1b
Test Case: Log into the game (tell him the name of the server,
username, password and port)
Date: 2001-03-16
Participants: Sebastian Friberg, a person with average
knowledge of computers
Performance: The test was done using windows emulator. Test
person entered the information quickly and then logged into the game.
Test person had no problems in logging into the game.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.1c
Test Case: Log into the game (tell him the name of the server,
username, password and port)
Date: 2001-03-18
Participants: Elina Salonen
Performance: Test person entered the information quickly and
then logged into the game. Test person had no problems in logging into
the game.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.2a
Test Case: Find information of the game (version and copyright)
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Cannot be found unless prior knowledge of Palm
usage (menu bar usage).
Duration: 5min
Result: Ok
Grade: A
Passed: Yes
Test: 5.2.2b
Test Case: Find information of the game (version and copyright)
Date: 2001-03-16
Participants: Sebastian Friberg, a person with average
knowledge of computers
Performance: Test was done using windows palm emulator. Test
person didn't know which was the right button so he had to try all buttons.
After he had pushed four wrong buttons he found the correct one.
Duration: 2min
Result: Ok
Grade: A
Passed: Yes
Test: 5.2.2c
Test Case: Find information of the game (version and copyright)
Date: 2001-03-18
Participants: Elina Salonen
Performance: Cannot be found unless prior knowledge of Palm
usage
Duration: 2min
Result: Ok
Grade: A
Passed: Yes
Test: 5.2.3a
Test Case: Solve:What is your dexterity and armour class?
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Easily found under stats
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.3b
Test Case: Solve:What is your dexterity and armour class?
Date: 2001-03-16
Participants: Sebastian Friberg, a person with average
knowledge of computers
Performance: Very easily found
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.3c
Test Case: Solve:What is your dexterity and armour class?
Date: 2001-03-18
Participants: Elina Salonen
Performance: Elina first tried to use look command, but then
noticed the stats button. This task was pretty easily performed.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.4a
Test Case: Solve:Is it possible to climb a tree?
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Not possible. No such command
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.4b
Test Case: Solve:Is it possible to climb a tree?
Date: 2001-03-16
Participants: Sebastian Friberg, a person with average
knowledge of computers
Performance: Test person went through all commands quickly
and understood how this game was supposed to be played. He understood
that this game doesn't provide such a command.
Duration: 2min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.4c
Test Case: Solve:Is it possible to climb a tree?
Date: 2001-03-18
Participants: Elina Salonen
Performance: Test person went through all commands quickly
and understood how this game was supposed to be played. She understood
that this game doesn't provide such a command in a rather short time.
Duration: 2min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.5a
Test Case: Find mountains
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Easily found and recognized
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.5b
Test Case: Find mountains
Date: 2001-03-16
Participants: Sebastian Friberg, a person with average
knowledge of computers
Performance: Easily found and recognized
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.5c
Test Case: Find mountains
Date: 2001-03-16
Participants: Elina Salonen
Performance: Elina was at north edge of the map when I asked
her to performed this test so she first thought that the black empty
squares were supposed to be mountains. Maybe the empty squares should
have also a icon, (for example ocean)? After I had told her that the
empty squares were supposed to be just empty squares, it took her about
15sec to find the mountains. The mountains were easily identified.
Duration: 2min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.6a
Test Case: Solve:Is it possible to climb mountains?
Date: 2001-03-17
Participants:
Performance: Not possible. No such command.
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.6b
Test Case: Solve:Is it possible to climb mountains?
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: Test person went through all commands quickly
and understood how this game was supposed to be played. He understood
that this game doesn't provide such a command.
Duration: 2min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.6c
Test Case: Solve:Is it possible to climb mountains?
Date: 2001-03-18
Participants: Elina Salonen
Performance: Test person went through all commands quickly
and understood how this game was supposed to be played. She understood
that this game doesn't provide such a command.
Duration: 2min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.7a
Test Case: Say:"This is a great game" (or something else)
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Easy and clear.
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.7b
Test Case: Say:"This is a great game" (or something else)
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: Easy and clear
Duration: 2min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.7c
Test Case: Say:"This is a great game" (or something else)
Date: 2001-03-18
Participants: Elina Salonen
Performance: Easy and clear. The letter "q" looks like "g"..
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.8a
Test Case: Find the east end of the map
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Which direction is east? :)
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.8b
Test Case: Find the east end of the map
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: No problems moving around on the map. Found east
end easily and didn't even try to jump from the map. The map borders
were easily recongnized.
Duration: 2min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.8c
Test Case: Find the east end of the map
Date: 2001-03-18
Participants: Elina Salonen
Performance: No problems moving around on the map. Found east
end easily and didn't even try to jump from the map. Moving was slow and
she got a little annoyed. She was impatient and pushed the moving button
several times during a turn.
Duration: 2min
Result: Ok
Grade: A-
Passed: Yes
Test: 5.2.9a
Test Case: Fight fire ant
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Couldn't avoid it, as those bastards attack on
sight.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.9b
Test Case: Fight fire ant
Date: 2001-03-17
Participants: Sebastian Friberg
Performance: Found easily the only fire ant, which I had put
on the map and fought it. He killed the ant quickly and didn't try to
use wrong commands etc.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.9c
Test Case: Fight fire ant
Date: 2001-03-18
Participants: Elina Salonen
Performance: It took a while to find the only fire ant, which
I had put on the map and fought it. She killed the ant quickly and didn't
try to use wrong commands etc.
Duration: 3min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.10a
Test Case: Attack Ugly orc
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Easy, but took some time to find the orc
Duration: 10min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.10b
Test Case: Attack Ugly orc
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: Test person attacked the orc properly.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.10c
Test Case: Attack Ugly orc
Date: 2001-03-18
Participants: Elina Salonen
Performance: Test person attacked the orc properly.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.11a
Test Case: Solve: Is it possible to stop fighting with Ugly orc?
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Depends on your strength. If you do not kill the
orc on the first blow, escaping is possible
Duration: 5min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.11b
Test Case: Solve: Is it possible to stop fighting with Ugly orc?
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: Test person just walked away from the orc. When I
asked if there was some other way to stop a fight, he guessed no.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.11c
Test Case: Solve: Is it possible to stop fighting with Ugly orc?
Date: 2001-03-18
Participants: Elina Salonen
Performance: Test person just walked away from the orc. When I
asked if there was some other way to stop a fight, she guessed no. She
mentioned that the name of the ugly orc was a bit odd, because the ugly
orc looked quite cute.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.12a
Test Case: Solve: Condition of Ugly orc
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Not very clear at first, but the condition can
be found out by looking
Duration: 5min
Result: Ok
Grade: A
Passed: Yes
Test: 5.2.12b
Test Case: Solve: Condition of Ugly orc
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: He had no problems to check the condition of
ugly orc. (He immediately tried to check orc's condition with look
command)
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.12c
Test Case: Solve: Condition of Ugly orc
Date: 2001-03-18
Participants: Elina Salonen
Performance: She had no problems to check the condition of
ugly orc. (She immediately tried to check orc's condition with look
command)
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.13a
Test Case: Solve: Your condition
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Hitpoints can be seen on the main screen
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.13b
Test Case: Solve: Your condition
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: He pointed to the HitPoints right away
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.13c
Test Case: Solve: Your condition
Date: 2001-03-18
Participants: Elina Salonen
Performance: She checked her condition from the stats bar and
guessed after a while that HP means Health Points. She said, that the HP
should have been Health so that it would have been easier to recognize.
She didn't notice the HP which is displayed all the time during the game,
but that because she had no clue what HP actually means.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.14a
Test Case: Solve: What other ways are there to fight?
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Could not find other ways to fight, except verbal
attacks (especially from ugly orc)
Duration: 5min
Result: ok
Grade: A+
Passed: Yes
Test: 5.2.14b
Test Case: Solve: What other ways are there to fight?
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: He tried to annoy the orc with "War Cry", but
didn't succeed. At this point he said that, the War Cry command was a
bit confusing and useless.
Duration: 2min
Result: ok
Grade: A+
Passed: Yes
Test: 5.2.14c
Test Case: Solve: What other ways are there to fight?
Date: 2001-03-18
Participants: Elina Salonen
Performance: She tried to use war cry, but understood quite
quickly that you cannot start a fight with that command. She commented
that it would be a good idea if you could start a fight with that
command etc.
Duration: 2min
Result: ok
Grade: A+
Passed: Yes
Test: 5.2.15a
Test Case: Change area
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Could not find the portal until was told that
the square with nothing on it is the portal. :)
Duration: 20min
Result: Portal should have its own icon
Grade: B
Passed: No
Test: 5.2.15b
Test Case: Change area
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: Changing the area was easy after I had told him
where it can be found. The portal should have its own icon.
Duration: 3min
Result: Portal should have its own icon
Grade: B
Passed: No
Test: 5.2.15c
Test Case: Change area
Date: 2001-03-18
Participants: Elina Salonen
Performance: Changing the area was easy after I had told her
where it can be found. The portal should have its own icon.
Duration: 2min
Result: Portal should have its own icon
Grade: B
Passed: No
Test: 5.2.16a
Test Case: Shout a war cry
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: Easy piece.
Duration: 1min
Result: Ok
Grade: A+
Passed: Yes
Test: 5.2.16b
Test Case: Shout a war cry
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: Easy piece, but useless command. Comment: There
shouldn't be useless commands if the game is so compact.
Duration: 1min
Result: Ok
Grade: A-
Passed: Yes
Test: 5.2.16c
Test Case: Shout a war cry
Date: 2001-03-18
Participants: Elina Salonen
Performance: Easy piece, but why can't you start a fight with it?
Duration: 1min
Result: Ok
Grade: A
Passed: Yes
Test: 5.2.17a
Test Case: Return to the Palm OS view (quit the game properly)
Date: 2001-03-17
Participants: Juha T. Vainio, acting as an average man from
the street, who has never heard of Monrovia
Performance: First reaction is to leave the game as the normal
Palm programs are quitted, only by leaving. But this way the connection
does not disconnect.
Duration: 5min
Result: Disconnecting is not clear
Grade: B
Passed: No
Test: 5.2.17b
Test Case: Return to the Palm OS view (quit the game properly)
Date: 2001-03-16
Participants: Sebastian Friberg
Performance: He pushed wrong buttons and exited the game in a improper way leaving the connection open. When I told him to first disconnect the game, he remembered that disconnection can be done from the manu bar.
Duration: 2min
Result: Disconnecting is not clear
Grade: B
Passed: No
Test: 5.2.17b
Test Case: Return to the Palm OS view (quit the game properly)
Date: 2001-03-18
Participants: Elina Salonen
Performance: She pushed wrong buttons and exited the game in
a improper way leaving the connection open. When I told her to first
disconnect the game, she disconnected using the option in the menu bar
and then quitted in a proper way.
Duration: 2min
Result: Disconnecting is not clear
Grade: B
Passed: No
6. Performance testing
Test: 6.1
Test Case: Maximum players
Date: 2001-03-18
Participants: Jarmo Mäki
Performance: I was able to connect the server with 12 players
at the same time. I made the test with 11 Windows Palm emulators and 1
Linux Palm emulators. The Windows Palm emulator caused much lag on my
computer, but I was able to move each character a bit.
Duration: 30min
Passed: Yes
Test: 6.2
Test Case: Maximum characters
Date: 2001-03-18
Participants: Jarmo Mäki
Performance: The maximum characters in the game is not limited
by the server nor the Palm client. The map size is the only limiting
factor. There shouldn't be too many creatures, which the computer has to
move, on one map because they bounce against each other which causes lag.
If there is a map of size 30x20 and 300 ants (which move randomly) on the
map, then this would mean that nearly every ant bounces against someone
else and in that case the lag is terrible. The amount of computer
controlled creatures should be in maximum 1/10 of the amount of empty
squares.
Duration: 30min
Passed: Yes
Test: 6.4
Test Case: Server memory
Date: 2001-03-18
Participants: Jarmo Mäki
Performance: I wasn't able to cause Memory leak nor did the
game in any point use much memory. The memory use was about 5-10MB
depending how much there were players and npc in the game.
Duration: 30min
Passed: Yes
Test: 6.5
Test Case: Server hard disk space/How much log is written
Date: 2001-03-18
Participants: Jarmo Mäki
Performance: How much log is written depends on what log level
the server has been started. Using log level 1 prints too much log and
therefore it's recommended to have at least loglevel 2. Loglevel 1 wrote
about 400 kB log in 15min when I had 20 ants in the game and there were
no players.
Duration: 20min
Passed: Yes
7. Customer testing
Test: 7.1.1
Test Case: Customer test
Date: 2001-03-27
Participants: Antti Ilvessuo / Jarmo Mäki
Comment:
Antti had no problems to learn to play the game. He logged in to the game
easily and before I had told him to do anything he was already going
through the commands. Antti had no troubles finding the stats and he
understood the terms used in the stats window. The objects were easily identified, but he didn't like the graphics we had used
and mentioned that the icons should be made finer.
Fighting in the game wasn't clear to him. First he tried to hit the enemy
every round with the direction buttons. Later when he had relized that he
didn't have to do anything when a fight had started he said that it was
boring. He commented that if this game was to be taken into real use there should be more
choices to be made during fight so that the players interest is kept up.
He said that there could be for example different attack methods which a
player could use during a fight. These attack methods can be displayed on
top of the character on the map. During the fight he hoped that it would
be possible to see how much damage you have caused to the enemy. Actually
there are different messages shown for some hits (if you score a critical
hit then a "you brutally ..." message is shown), but during the fight lots
more messages should be shown so that it would be easier to see how the
fight is going.
He noticed that when while moving around the map area there are places where you cannot make out
if the character has really moved, because the displayed map doesn't change.
He suggested that there should be little marks on the ground so that the player can be sure
that the character really has moved. Also the direction arrows should be
made black when they are pushed.
He said also that an inventory is mandatory if you want to make players addicted to the game.
His overall impression about the game was that the game is easy to play, but it lacks some functionality which makes the game really interesting.
Test: 7.1.2
Test Case: Customer test
Date: 2001-3-17
Participants: Mari Lättilä / Jarmo Mäki
Comment:
Mari had problems to understand the GUI. She
didn't consider the player to be in the middle of the map, because she
thought that the whole screen was meant by the map. When I asked her to
find stats of her character she didn't first see the stats button.
It seemed that dividing the GUI into several smaller boxes the way we did
didn't please her.
The player stats were not clear to her. She didn't know what hp was and
she couldn't make out to what the level corresponds. She said that the values for the stats should be more understandable (what does size 3 mean).
Mari hoped that moving was a bit faster. She was unsure if
anything was to happen because sometimes the game lagged. She commented that the character was slow to react to commands.
Fighting in the game was confusing to her because she didn't relize that the fight was still going on although she did nothing.
She said also that just fighting in the game is not enough and she recommended that we create a better plot for the game.
Her overall impression about the game was that the game seemed to work fine, but it wasn't interesting and the gui wasn't easy enough to adopt.
8. Result of the entire testing
Passed
Acceptor: Juha Vainio
9. Test Log
| Test |
Passed (Grade) |
Bug state |
BugID |
| 2.1.1 |
A |
|
|
| 2.2.1 |
A+ |
|
|
| 2.2.2 |
A |
|
|
| 2.2.3 |
A+ |
|
|
| 2.2.4 |
A |
|
|
| 2.2.5 |
A |
|
|
| 3.1.1 |
Yes |
|
|
| 3.1.2 |
Yes |
|
|
| 4.1.1 |
A+ |
|
|
| 4.1.2 |
A+ |
|
|
| 4.1.3 |
A+ |
|
|
| 4.2.1 |
A+ |
|
|
| 4.2.2 |
A+ |
closed |
1107 |
| 4.2.3 |
A+ |
suspended |
1125 |
| 4.2.4 |
A+ |
suspended |
1125 |
| 4.2.5 |
A+ |
closed |
1103 |
| 4.2.6 |
A+ |
|
|
| 4.2.7 |
A+ |
closed |
1108 |
| 4.2.8 |
A |
closed |
1109 |
| 4.2.9a |
A+ |
|
|
| 4.2.10a |
A+ |
suspended,closed |
1248,1251 |
| 4.2.11 |
A |
|
|
| 4.3.1 |
A+ |
|
|
| 4.3.2 |
A+ |
|
|
| 4.3.3 |
A+ |
|
|
| 4.3.4 |
A |
closed |
1250 |
| 5.1.1a |
A+ |
|
|
| 5.1.1b |
A+ |
|
|
| 5.1.2a |
A+ |
|
|
| 5.1.2b |
A+ |
|
|
| 5.2.3a |
A+ |
|
|
| 5.1.3b |
A+ |
|
|
| 5.1.4a |
A |
|
|
| 5.1.4b |
A |
|
|
| 5.1.5a |
A+ |
|
|
| 5.1.5b |
A+ |
|
|
| 5.2.1a |
A+ |
|
|
| 5.2.1b |
A+ |
|
|
| 5.2.1c |
A+ |
|
|
| 5.2.2a |
A |
|
|
| 5.2.2b |
A |
|
|
| 5.2.1c |
A+ |
|
|
| 5.2.2c |
A |
|
|
| 5.2.3a |
A+ |
|
|
| 5.2.3v |
A+ |
|
|
| 5.2.3c |
A+ |
|
|
| 5.2.4a |
A+ |
|
|
| 5.2.4b |
A+ |
|
|
| 5.2.4c |
A+ |
|
|
| 5.2.5a |
A+ |
|
|
| 5.2.5b |
A+ |
|
|
| 5.2.5c |
A+ |
|
|
| 5.2.6a |
A+ |
|
|
| 5.2.6b |
A+ |
|
|
| 5.2.6c |
A+ |
|
|
| 5.2.7a |
A+ |
|
|
| 5.2.7b |
A+ |
|
|
| 5.2.7c |
A+ |
|
|
| 5.2.8a |
A+ |
|
|
| 5.2.8b |
A+ |
|
|
| 5.2.8c |
A- |
|
|
| 5.2.9a |
A+ |
|
|
| 5.2.9b |
A+ |
|
|
| 5.2.9c |
A+ |
|
|
| 5.2.10a |
A+ |
|
|
| 5.2.10b |
A+ |
|
|
| 5.2.10c |
A+ |
|
|
| 5.2.11a |
A+ |
|
|
| 5.2.11b |
A+ |
|
|
| 5.2.11c |
A+ |
|
|
| 5.2.12a |
A |
|
|
| 5.2.12b |
A+ |
|
|
| 5.2.12c |
A+ |
|
|
| 5.2.13a |
A+ |
|
|
| 5.2.13b |
A+ |
|
|
| 5.2.13c |
A+ |
|
|
| 5.2.14a |
A+ |
|
|
| 5.2.14b |
A+ |
|
|
| 5.2.14c |
A+ |
|
|
| 5.2.15a |
B |
|
|
| 5.2.15b |
B |
|
|
| 5.2.15c |
B |
suspended |
1125 |
| 5.2.16c |
A+ |
|
|
| 5.2.16b |
A- |
|
|
| 5.2.16c |
A |
|
|
| 5.2.17a |
B |
closed |
1283 |
| 5.2.17b |
B |
|
|
| 5.2.17c |
B |
|
|