Usability Evaluation Plan Team 33k!
Mark B
Amy Chang
Ibrahim Moreno
Hernan Liatis
Sean Hussey
Team 33k! will be evaluating the usability of the game Oregon Trail, designed by Aqua Team Hunger Force. Because the game has not been fully implemented by Aqua Team Hunger Force, the evaluation will evaluate pre-made mockups of the game as well as the functional part of the user interface as of Saturday, November 15, 2008. The user interface will be version 3.1 retrieved from the Cincom Repository.
The evaluation will have to be done in a short amount of time, so an observational approach would not be feasible. This type of evaluation would require much more careful planning, a gathering of participants, and equipment such as cameras. Also, the materials given to us would hardly be sufficient for a participant to grasp the idea behind the game, and, therefore, will have trouble finding problems. A cognitive walkthrough would also be unfeasible since the prototype is lacking in many respects. This result is unfortunate since a cognitive walkthrough would be the best one to use for a computer game. After all, people who play games tend to learn it through experimentation in the game.
Because observation and cognitive walkthrough are not feasible, the evaluation would be handled using a heuristics approach. Since Team 33k! also created an Oregon Trail game, the evaluators will have sufficient knowledge of the domain. This familiarity helps us overcome any major obstacles in evaluation screen mock-ups. In other words, we can try to follow the mock-ups according to our knowledge of the game. Furthermore, many heuristics, such as the font used, do not require a functional prototype.
The heuristic evaluation will be done with 4-5 people, all from Team 33k!. Therefore, the evaluation should encounter around 60-75% of the problems with the game. All evaluations will be done independently using a checklist form from the Xerox Corporation. Once each member is complete with their forms, the problems will be compiled and then categorized. The group will then give a severity rating by listing what they believe to be the top five problems with the game. After listing out the top five problems, the group will create and send a usability report to Aqua Team Hunger Force.
Hopefully, this evaluation will reveal major usability bugs in Aqua Team Hunger Force’s incomplete prototype. Therefore, errors can be fixed before the final product is released. If the evaluation was done after release, the developers might have to spend massive amount of time and money in maintenance to fix these problems. Team 33k!’s evaluation should lower that cost.
Usability Report on
“Oregon Trail”
Developed by Aqua Team Hunger Force
Report by:
Team 33k!
Mark B, Amy Chang, Ibrahim Moreno, Hernan Liatis, and Sean Hussey
Team 33k! performed an evaluation of Aqua Team Hunger Force’s “Oregon Trail” based on good practice principles developed Jakob Nielson. The major usability problems found fall under the categories of information organization, error handling and messaging, and help and documentation. For each category, Team 33k! will provide at least one recommendation to solve the problem.
Organization of Information
The program makes a good use of “chunking” related information and actions. For instance, the list of people is completely separated from the list of items. Furthermore, the radio buttons for pace and ration are grouped and bordered, making sure the user know the relations between each button. In other words, the user can tell that clicking a button in the pace grouping has no effect on the button in the ration grouping.
The sub-menus themselves are mostly well-organized. Most items are grouped vertically as opposed to horizontally. Most technical communicators follow a general rule to avoid a horizontal placement of items since up/down eye and mouse movement is easier than left/right movement. The same reasoning explains why most applications and websites avoid horizontal scrolling.
Problem Areas
While the information chunks do not overlap, they have a tendency to be very close together. For instance, the main screen clumps all the information at the very bottom. Furthermore, the clumped information seems fairly unorganized since there is no real upper-bound between the information and whatever is supposed to be above it. From the mock-ups, the white-space above the information will be covered by what appears to be an animation of the oxen walking. While visually nice to look at, this animation is not as important as the people, items, map, rations, etc. The user should not be distracted by unnecessary information.
The sub-menus also have issue in organization. First, the first screen displays the radio buttons for profession horizontally. Second, the radio buttons for the profession do not directly match with the descriptions of the professions. Third, many areas have issues with alignment. Numbers with decimals should have all the decimals aligned in one column. Also, numbers are not right justified, so the ones digit do not all appear in one column. Having right justification for numbers allow users to easily compare the values of numbers since each place value will have its own individual column.
Recommendations
For the main screen, we recommend spacing out the information so that it doesn’t appear so crowded at the bottom. Put the most important piece of information in the center, and make use of white space to separate chunks out. Any auxiliary or aesthetic information should be placed at the very top, very bottom, or at the sides.
All items in the sub-menus should be listed vertically. Specifically, the choice of profession needs to be aligned from top to bottom. Also, the descriptions of each profession should be right next to each radio button. With this arrangement, the user doesn’t need to constantly switch back and forth between the buttons and the description. He or she simply needs to follow the descriptions vertically and pick a profession once one sounds decent for him or her. Finally, align the numbers correctly so as to limit confusion when reading and comparing the numbers.
Error Handling and Messaging
The game handles some fairly basic, but important, errors. For instance, the player can not exit the setup screen without choosing a profession. In the store, a calculate button takes in input and relays the purchase information to the user. In order to exit the store screen, the user has to calculate his or her purchases. If the player tries to buy something it can not afford, the store sends an error message saying that the player can not afford the purchases.
Problem Areas
However, the game does not handle all major errors correctly. For example, if the user clicks trade when there are no items available, the game crashes with a divide-by-zero exception. Also, the user can easily cancel out of the screens, creating a wagon with no leader, no passengers, and no profession. Finally, incorrect input can be entered into the store screen, such as ‘y’ oxen and ‘2.5’ wheels.
Also, the error messages sound like they are blaming the user for their problems. A major reason for this problem is the use of exclamation points in the messages themselves. Fortunately, the messages do tell the user what is the problem and the remedy.
The buttons in the game are arranged in such a manner that the cancel button appears before the equivalent of the okay button. Such a placement can be confusing for the majority of users who have grown accustomed to ‘okay’ appearing before ‘cancel.’ Also, some radio buttons, like pace, do not act as expected. They tend to be all selected when the user selects one.
Recommendations
Whenever the game requires numerical inputs, the program should check to find the common mathematical errors and exceptions encountered, such as divide-by-zero. Furthermore, the program should check to see if there is any valid input even when canceling. In other words, the first time the wagon setup screen is up, there should be no cancel button. This way, the game is ensured to have a correct configuration. It is also recommended to employ the use of default input.
The game’s error messages should be worded so that it does not appear to be the user’s fault that the error occurred. The wording should avoid using exclamation points and any hostile sounding language. The error message should simply tell what happened and how to fix it.
Finally, the game should display the buttons in such a manner that they will create fewer errors. For instance, the ‘okay’ button should appear before any ‘cancel’ button. This placement will ease users who play through the game more quickly. Also, provide a way to undo any mistake the user made. A good way to do this is to provide save/load option or provide the option to undo a purchase before moving. Finally, no more than one radio button should be selected.
Help and Documentation
Most game screens provide information for the user to figure out what is going on in that screen. For example, the store screen tells the user to input the quantity into the text fields. The setup, store, and trade screens are all labeled and, at least, tell the user what is going on.
Problem Areas
Unfortunately, the game lacks in a lot of help and documentation. The game assumes the player is familiar with the mechanics of the game. Therefore, new players will not know what to do. This problem becomes more apparent in the main screen, where there are many buttons that do not have obvious links to what they do. A player may not know what ration corresponds to what number. In fact, the player may not know where the rations are because there is no label for rations. Furthermore, the game does not state the requirements and affect of a hunt.
Recommendations
The game should have an about page, help screen, or manual. Before, the game starts, the game could tell the user the basic premise of the game as well as the goal. The help documentation should be accessible no matter what. It is recommended that help be an option in the top menu bar and that the help screen display information about each screen and each action. Finally, all groupings should be labeled to avoid any ambiguity.
Conclusion
The game has problems in organization, error handling, and documentation. For each problem area, Team 33K! provides at least recommendation for improvement. With the identification of these problems as well as the aid of these recommendations, Aqua Team Hunger Force will make a better product. A better user experience should lead to more customers buying the game by word-of-mouth as well as a user-base for future products.