P4 - PROJECT TESTING

In addition to individual interviews and observations we have conducted a survey to collect quantitative user feedback. We tried asking non leading questions so that we would not direct the user in any way. The survey starts by asking the users age so that we can filter out potential difficulties connected to specific age groups. Next we ask the user to state their browser and computer type. This is useful as we want our website to work on multiple devices. The survey then asks the user to rate the overall usability of the site on a scale of one to six. Further on we give the user a specific navigation exercise to conduct and then rate the experience. This way we ensure that the user is actually navigating the website and we are able to evaluate the usability of this specific path on the site. Finally the survey asks if the user had a favourite part and a least favourite part, followed by a last question where we ask for suggested improvements. Click here to see our google form survey.

As we are working for a fictitious client we could not get our client to test the website.

The target audience of the page includes anyone with access to a computer or a phone, who desires to learn a new language. The page targets beginners, with little to no previous experience with their chosen language. To make sure the page works well for all of our audience we have tried to test it on people in different life situations and ages. In P1 we presented two widely different personas to represent the variety of our target audience. We have used our personas as a base when collecting test audiences, since a great variety of people could fit our target. Hence, we have reached out to fellow students and siblings as well as parents, aunts and uncles. The diagram below shows the age distribution of the testers taking the digital survey. We can see that testers from 18 to 25 years of age are the highest represented group from this survey. This was an expected outcome as it is easier for us to access the typical ‘James’ (22) rather than the typical ‘Turid’ (65).

For our live testing we have testet roommates, friends and family. We conducted ten live tests in total, most of them were in person but we also did some digital ones. Of the test audience that we interviewed, about 90% were between 23 and 26 years of age. The remaining 10% were around 60 years old. This uneven distribution is a result of us being students and mostly being surrounded by other students and people of our own age. In other words, older test audiences were not as easily available to us, unfortunately.

The table below shows the different browsers and computers that our testers used, and the combination of these.

In this summary we will present some of the numbers from our survey and the general feedback on each topic before presenting some of the more specific comments from the survey and the live interviews and our responses to them.

Navigation

Our page generally got a high score on navigation. From our survey 71.4% of the testers gave the page a 5 out of 6 when rating the page based on overall arrangement and how easy the site is to navigate. When the testers were asked to navigate to where they could practice Spanish on the page, 71.4% said it was really easy (6/6) to navigate here. 64,3% said it was really easy (6/6) to find the page where they could test themselves in the words they had just learned. In the last part of the survey, when asked to write what they potentially liked about the page, people generally described the page as clean and easy to navigate. The page was also perceived as well arranged and intuitive. All over only positive feedback regarding navigation.

Button to practiceMarket when you get a bad score in testMarket.html

One user commented on how there should have been a button leading directly to practiceMarket.html when you get a score sending out the message “you should practice more”. This was not changed before delivery, but we see how this would make sense and be useful. We way we would have solved this is by making an extra button that would be visible when you get a bad score using the existing JavaScript file, and linking it directly to the corresponding practice arena.

Getting back to the homepage

Most users found it very intuitive to press the logo to get back to the home page. Some of the testers over 50 years also managed to find it, but hesitated a few seconds first. This shows how important it is to test the group of the audience that is different from ourselves.

Design

Our survey shows that 50% of the testers find the Idiomas por Idiotas website very appealing (rating 6 out of 6) and 42,9% find page 5 out of 6 appealing. 92,8% of the survey testers gave the page a 5 or better when asked if the design choices of the page corresponds with the purpose of the site - which is to learn spanish. These are very good results. When asked if there was anything they particularly liked about the page the testers mostly mentioned design related matters. They liked the layout, graphics, illustrations, color palette, and theme.

The colors of the image carousel in testMarket.html

Before our first round with tests the colors here were the same as in the mockup. Some users pointed out how the red background of the image representing the current word to translate (we will call it “current image”) made them confused. Did red mean that I got it wrong? And others did not get which one they were supposed to translate. As an attempt to make the page less confusing we changed the background colors so that the background of the current image is white, and the previous and next are light blue, with black and white colored images. In the later tests we got no more remarks on the design being confusing on this page.

Usability

In our survey we asked if the users understood what they were supposed to do in the arena ‘the market’. 50% said they understood the task very well (rating 6 out of 6). The rest of the users were distributed between the ratings 3, 4 and 5. This is a good result, but lower than the previous rankings which suggests that there is room for improvement. When reading the feedback on what the users potentially disliked about the page some users describe the market arena as somewhat difficult to understand. It is also mentioned that some of the illustrations in the market are too small. This makes them hard to hoover and find. Another feedback was that the market test should accept both big and small letters, as for now it does not evaluate ‘H’ and ‘h’ as the same letter.

What to do in practiceMarket.html

In the first rounds of testing some users commented that it would be nice with a short text explaining what you should do when you get to the practice arena, even though they found out reasonably fast what to do. Since one of our main goals is to make the site easy for everyone, we wanted to make this change. We decided to put the text right under the title, so it would be one of the first things a user would see when entering the page.

Feedback message in testMarket.html

From the first rounds of testing we got several comments about how the feedback message when the user presses the “check” button could improve. For instance the color of the text was white, and they suggested it should be green for correct and red for wrong. Also, some commented that it would be nice to know what the correct answer is, when you get it wrong. We all agreed that these were good suggestions that would improve the page and would be easy to implement. Both these changes were then fixed by changing, and adding a few lines of code in testPointsAndInput.js.

Bigger font size in the navbar

Some of our older testers pointed out that it was great we had large text, but that the navigation bar was too small. This has been changed before delivery.

Content

We did not have any specific questions regarding our content, but we got a lot of positive comments about content in the question where they could write what they liked about the page. Some of these elements are listed below:

• Our choice of different arenas were suitable for the intended audience.
• An interactive game is very fun.
• The different illustrations were suitable and fun.

One thing we have learned from this project is that it is very smart for the progress of the project to have several deadlines and milestones. When we started to do the coding part, it was very efficient that all the design issues had been taken care of in the previous submission. Since P3 had a late due date, we made our own milestones and deadlines for P3 in order to motivate us and promote progress.

We have had regular meetings and worksessions twice a week which has been important for our progress and communication throughout the project. We feel like we found a good balance between working individually and sitting together as a group.

We have had some delays because of sickness amongst some of the group members as well as misunderstandings of what tasks were to be done. We usually did not know which tasks had not been done prior to the check-ups meetings. This might have been avoided if we had made a better system for communication or follow-ups between meetings. Besides our meetings we had all communication via messenger which turned out somewhat messy. In later projects we will therefore consider to do regular follow ups between meetings via a suitable social platform such as Slack.

We chose to make this particular site because it allowed us to be creative in design and in what kind of functionalities we wanted. The fact that we were able to work with a colorful and creative site kept our spirit up and made it more fun to sit down and learn about new coding laguages and coding in general. We gave ourselves a challenge choosing such a complex idea for our project, and we have spent a lot of time in order to pull this off. Overall we are very pleased with our final project delivery.