Hotspots: Admin Pages | Turn-in Site |
Current Links: Case Final Project Summer 2007
M8 - UI Evaluation (Peer critiques)
We chose to do the heuristic evaluation because we liked the idea of a checklist and ratings and felt like it had the most structured approach. This method also had the added benefit of being able to do a bulk of the project independently, which boded well for our conflicting schedules.
To begin, we independently evaluated the team’s UIs using Jakob Nielsen’s ten guiding principles found at
http://www.useit.com/papers/heuristic/heuristic_list.html. After playing around with the UIs for a bit, we wrote down if we thought the UIs met the criteria or not. If they didn’t, then we tried to be very explicit in what we thought was lacking or what we thought needed to be improved.
Our next step was to come together as a group and go over everyone’s comments. We compiled a list and rated the errors based on the following scale (found on Dr. Waters’ slides):
0 – I don’t agree that this is a usability problem at all
1 – Cosmetic problem only, only fix if extra time available
2 – Minor problem, Fixing giving low priority
3 – Major problem, Important to fix, give high priority
4 – Catastrophe, imperative to fix this before product released
Then, we wrote our report to share our findings. We got points taken off for not including the impact and frequency of the bugs as well as explaining how we arrived at the ratings we did. Our total score was 96/100. When in doubt, explain everything!
M8, Part One
We were given Team Apathy’s project to evaluate. We chose to use the heuristic evaluation to accomplish this task because we felt as though it pertained to this assignment better than the other methods for numerous reasons. In order to explain our rationale, we will first go through why we didn’t pick the other methods.
First of all, we felt as though the cognitive walkthrough wasn’t a good idea because much of what the system is supposed to do has been laid out for us in the assignment requirements. Therefore, it would be hard to say whether or not their design is intuitive since we have mulled over the program specifications extensively and could not impartially view their design as though we were first-time users. Secondly, we rejected the think aloud method because we thought it would be awkward to record ourselves. In addition, it would be tedious to have to transcribe our thoughts and we would probably end up with repetitive results anyway.
Thus, we ended up with the heuristic approach. We felt this method seemed to be the most empirical solution due to its checklist, common set of criteria and grading scale. Every member of our team will act as an individual evaluator. We will use Jakob Nielsen’s Ten Usability Heuristics to evaluate Team Apathy’s design. Then, we will write down whatever usability bugs we think there might be and rate their severity according to the scale in the lecture slides.
We hope this method will teach us how to become better evaluators of design and usability as well as provide us the tools to make Team Apathy’s project better. Furthermore, we hope doing this assignment will make us more aware of the decisions we make on our own project.
M8, Evaluation Report
Overall, we think Team Apathy did a good job in complying with the assignment. Their system is easy to navigate. Their UIs are not cluttered and for the most part easy to understand. However, we wish they would change the layout a little bit as it seems a little too bare. For example, the windows have too much negative space and the default gray is lackluster. Also, the edit and launch buttons seem awkward since they are skewed only a little to the left. They need to have a more distinct, balanced separation from the rest of the buttons.
We are also confused on what the “restore” function does. We are thinking that it brings back an accidentally deleted supplier or POS location. If this is the case, that is a good way to prevent a potentially grave error. However, we do not know how useful that button would be since the user would have to push it right after the mistake was made unless there was a copy being made of every action the user makes. Anyway, a clarification on what the button is restoring would be nice.
Another major design criticism we have is that there aren’t any exit buttons; although, there are cancel buttons on the edit UIs. Moreover, there is no “send order” button for the POS location. Also, there needs to be some error handling for when you push a button and nothing is selected because nothing happens. It might be confusing to the user as to why nothing is happening.
As a side note, the cancel button doesn’t actually cancel anything when you add items to the inventory. In other words, the items still get added. We aren’t sure if that was an oversight or something that is just in the process of being implemented.
We also thought the default value should be changed to reflect “1” instead of “0” when you are adding something to the inventory. In addition, there needs to be some sort of default entry when you are doing things like adding suppliers and inventories. As it is now, the last thing you copied and pasted is showing up. Furthermore, there is a “0” that shows up in between the name and location of the POS location. This also happens when adding a new supplier.
As for the sell policy function, we like the radio button format. However, it would be nice to see the sell policy displayed in the UI. It would be tedious to have to push the button each time to see what the current sell policy is. This idea could also be expanded to the total cost and total market value buttons. As another side note, the sell policy window is currently labeled “unlabeled canvas.”
We have just a couple of minor errors that need to be fixed as well. For example, privilege is misspelled in the Edit Employee App Model. It currently reads “priviledge.” Also, the dollar sign and price need to be inverted as it is now showing up as “0$” in the total cost and total market value.
We do like how you can edit things in one window. That is a nice touch. The tabbing toggle is also a good feature. However, we would prefer less of a border around the top and left sides and maybe more of one on the right and bottom sides.
The assigned ratings for most of these comments were a “1” or “2” with only few “3s” and “4s.” Therefore, we believe Team Apathy did a fantastic job overall. They also went above and beyond with the employee information and fields such as username and password. They just need a few tweaks to make their program remarkable.
M8 Response
We agree with the Green Monkeys’ comments for the most part. Our lack of error prevention and handling is definitely our biggest problem right now. Without such safeguards and precautions, our UIs are designed for the experienced user. They do not take in the new employee who is likely to push buttons and get errors without knowing what they mean.
For example, we need to have a dialog box pop up for when the user pushes a button without having anything selected. It would state something like, “Make sure a supplier is selected,” or whatever the error is. Currently, as the Green Monkeys pointed out, the user just gets a VisualWorks error which definitely does not let the user know what the problem is. Instead of making the easy fix of simply making sure something is selected, the user could go through many different and extremely frustrating forms of troubleshooting. In a business situation, this scenario could result in a lot of time, money and personnel being wasted.
Similarly, we need to make sure that fields that are meant for integers do not accept letters or symbols. We appreciate the Green Monkeys pointing this out because such a mistake would create another unintelligible VisualWorks error message. We could solve this problem by checking the inputted information and making sure it is compliant with what is required. Therefore, if the user accidentally inserts a symbol while trying to change the selling price, our program will detect the error as well as display a dialog box indicating the problem.
Another valid point the Green Monkeys made was that we need to make sure the user can’t make more than one supplier or POS location with the same ID. This problem could lead to lost or duplicate orders as well as general confusion. A solution to this problem would entail having our program automatically check for existing suppliers or POS locations with the same ID as the new one being entered. If there is an existing one already in the system, then something like a merge option or create a new ID option should pop up.
In response to the Green Monkeys’ comments about our UIs being non-uniform and distracting, we have changed the colors of our UIs to be the same. Initially, we thought it would be better for the user if they were different as a visual indicator of viewing alternate screens. However, we agree with the Green Monkeys that our UIs did look like they were from separate programs. We could further enforce the unity of our UIs by having the exit buttons in the same location as well as making sure the button designs are all the same. Furthermore, a logo of some sorts would probably help as well.
Finally, we have changed our graphs to Green Monkey’s specifications, meaning labeling the x and y axes as well as adding legends that explain what the different parts of the graph represent. We agree that the graphs are much clearer now as a result. Furthermore, we have changed the background colors of the graphs to be the same as the UIs’ background color, so they are more uniform with the rest of the UIs.
Coding with OO definitely makes things easier to change. Right now, all of these fixes would be pretty simple because they would entail changing something in a particular class without having to alter other classes. For example, adding a dialog box that lets the user know he or she needs to input an integer for the edit price button in the POS Location UI does not require additional code in any other class. Likewise, the change would have absolutely no effect on any of the other UIs.
In addition, finding where the code needs to be modified is simpler as well with OO design. If we were to change the POS UI, we would just look for the POS UI class. There is no need to look through a program that could be thousands of lines to find the place that is controlling the POS UI.
Other benefits of using OO design are that we could add and delete classes to the ones we have now without making major adjustments. For example, we worked on some classes separately and then made them cohesive later on. Moreover, we have the ability to reuse our existing code.
Ratings
Usability Bugs for Team Apathy:
total cost and market value show "0$", needs to be inverted rating: 2
should have default value inside of input fields otherwise last pasted item gets defaulted in there rating: 2
name sell policy window rating: 1
doesn't have button to send orders rating: 4
don't like design rating: 1
clarify what restore does rating: 2
clicking cancel doesn't cancel the entire add, just the one dialog (it still adds the POS location) rating: 3
need an error message for when you click on edit but have nothing selected b/c it will just do nothing rating: 4
windows have too much white space rating: 1
default assignment for variables should be better rating: 2
need to correct spelling of "priviledge" in Edit Employee App Model rating: 2
need exit buttons on UIs rating: 3
display sell policy so you don't need to click to see what it is rating: 1
change edit and launch buttons rating: 1
One Example of Using the Checklist
Ten Usability Heuristics
Visibility of system status
I don’t know if this is really applicable since the program is not running in real time.
Match between system and the real world
Team Apathy used terminology that would be understandable by everyone. Furthermore, their edit fields were very clear. However, they do need to correct the spelling of “priviledge” in their Edit Employee App Model. Also, I’m not quite sure if “restore” means that you can get back a supplier or POS location that has been deleted. (I’m guessing that is what that function does.) The order of the UIs makes perfect sense.
User control and freedom
If that is what the restore function does, then Team Apathy did a good job of providing a way to correct a possible mistake. However, they do not have exit buttons on their UIs. I do like their use of ok and cancel functions in their edit UIs.
Consistency and standards
Team Apathy used consistent terminology that does not confuse the user.
Error prevention
It would be better if the sell policy was indicated on the screen so that the user didn’t have to click the button to see what it is. Also, it would be better to have a default value (even if it is blank) set in the input field so that the last pasted item doesn’t become the default value.
Recognition rather than recall
I like the tabbed layout which facilitates easy toggling between different screens. They also have the edit screens nicely tucked away so as not to clutter their UIs. In addition, there is no need for recall.
Flexibility and efficiency of use
Accelerators were not used.
Aesthetic and minimalist design
They used minimalist design but perhaps too sparse. I would like to see something other than the default gray. Also, the edit and launch buttons look odd as they are only slightly to the left. Therefore, they look more misaligned than purposely sectioned off. I would either decrease the width of the window or put them closer together and then move them more to the left. Also, I would label the sell policy window so it doesn’t say “unlabeled canvas.” Lastly, their total cost and market cost show “0$” instead of “$0.”
Help users recognize, diagnose, and recover from errors
No error messages were used.
Link to this Page