RATINGS

Screen Shot 2019-06-10 at 8.10.15 PM.png

EMPLOYER

Crossing Minds

TEAM

3 People, Visual Designer, Product Manager, Myself

CONTRIBUTION

User Research, Usability Testing, Wireframes.

DURATION

January 2019 - March 2019

MY CONTRIBUTION

I created and conducted user research and moderated usability tests to understand the user base and their expectations for the onboarding, rating and recommendations experience. Based on the feedback and insights, I made proposals for product strategy informing the user flow and integration of onboarding and item ratings with the overall app experience. I then created user flows and wireframes for the redesign of the Hai app onboarding experience as well as interaction mockups for the item rating experience.

ABOUT HAI

Hai iOS App is a recommendation platform created by Crossing Minds, an AI startup using machine learning to help users discover experiences across domains, like movies, book, and even restaurants.  The recommendation engine itself is as unbiased as possible, has no ads, is platform agnostic, and always free to use.  The goal is to collect the most pure taste correlations from each user.  Crossing Minds' business model is to take learnings, not data, from users to create an API that other businesses can implement and improve their recommendations.

 

THE CHALLENGE

A first version of the on-boarding experience for Hai iOS App already existed when I joined Crossing Minds. In early 2019, the product design team, comprised of a visual product designer, the product manager, and myself, was tasked with redesigning the on-boarding and tutorial experience for users. 

 

On-boarding is the user's first introduction to the app.  It was an opportunity to introduce users to features and educate them about essential functions and features. Since many features were added and improved since the original on-boarding was created, we also wanted to address the consistency of experience from on-boarding throughout the rest of the app.

ORIGINAL DESIGN

 

PRODUCT STRATEGY

Integrated and Consistent UX:
  • In the existing version of the on-boarding experience, users rated with a single tap to "like" and a double tap to "dislike".  Tapping on items revealed heart icons where a "dislike" is equivalent to 1 star out of 5, and a "like" is equivalent to 4 stars. This was a different rating experience than the Train feature within the app where users swiped left for 1 star and right for 4 stars, or press and slide on stars to rate.

  • We also wanted to better understand how the on-boarding experience affected the user's expectations of the app experience itself, and adjust the designs accordingly.

 
Analytics and Business Goals:
  • Another factor was that the ratings lacked diversity. In Amplitude, our analytics platform, 1 and 4 star ratings represented more than 2/3 of the ratings when, within the app, users have the option to select from half a star to 5 stars, 10 star rating choices total. 

  • Users often spent 10 minutes during on-boarding liking and disliking hundreds of items, sometimes even half an hour to an hour.  We were concerned that users will rate most of the items that they knew and would not adjust or add to their ratings later on.

  • If that were the case, the recommendation engine would not be able to make the best recommendations since it would not receive the most accurate ratings. Then, from a business standpoint, the algorithm would not be as robust when our API is applied to B2B, the revenue generating part of our business model.

ORIGINAL RATING METHODS

3 Ways to Rate:
 
(From left to right)
1. On-boarding: Tap once (4 stars) Tap twice (1 star)

2. Discovery: Press and slide to select specific star rating

3. Train: Swipe left (1 stars) Swipe right (4 stars) Press and slide to select specific star rating 

There were too many methods we were asking user's to learn, we needed to merge these options or remove some options.

 

USER FEEDBACK

User Interviews and Usability Testing for the Original Design: 

 

Insuring an Integrated UX:
  • Feedback from first-time users during usability testing was that there were some screens that felt unnecessary, and the item cards were a little small.

    • We needed to condense some parts of the flow, and would need a solution for displaying item cards that provided a good selection of items for users to choose from, that was also large enough for more of the image and title to be displayed.​

  • Users liked the ease of rating with tap and double tap, which was the original intention, but were unaware that their "likes" and "dislikes" were translated into stars once they reviewed their ratings in their Profile, and when they were introduced to rating items with stars upon entering the app. The users who noticed the difference did not like the inconsistency.

    • We decided to design alternative rating methods and gestures to bring the on-boarding rating methods closer to the Train experience including potentially redesigning the train experience​ itself.

  • Users expected that the interests they selected, movies, books, games etc (a minimum of 3), would be the interests they are shown once they reach the app itself, the current app showed all 6 available domains despite what users selected during on-boarding.

    • We changed the discovery feed format to display only the domains that users selected once users finished on-boarding.

 

Customizing the Recommendation Experience:

  • During user interviews, we also learned that the interests with which users decide to rate and train Hai is sometimes different from the interests in which they want recommendations.  For example getting plenty of book recommendations from word-of-mouth, but needing better movie recommendations from Hai.

    • After all, that is the beauty of cross-domain recommendations, the kind of input (train) can differ from the desired output (discovery).

  • Additionally, many users return to the app to get recommendations for only 1 or 2 domains, like movies, or books.

    • From this insight, we gave users the option to change and rearrange the interests in their discovery feeds while keeping all 6 interests available for the Train feature.

 

Address Analytics and Business Goals:

  • User interviews also revealed that most users inherently understood their ratings contributed to better recommendations, and noticed that Hai was already making recommendations as they rated during on-boarding, and therefore are willing to spend time rating.

    • We can ask more from users during on-boarding, like rating specific star ratings and even including the bookmark feature to save items as they rate and as they discover.

References for the Rating Experience:
 
We referenced Tinder, Bumble, and Letterboxd. 

The Train feature originally referenced Tinder, and allowed users to rate items with a swiping feature, swipe left for 2 stars, right for 4 stars, as well as pressing down and sliding on the stars. 

 

We created mockups to explore experiences with binary decisions, like Tinder with X and heart icons for like and dislike, and Bumble Beeline where users have a lineup of potential matches in two columns that they can scroll through and swipe. 

 

Letterboxd, a movie discovery and review app, had other labels that allowed users to indicate that they have watched the movie and/or like the movie, and can rate or add the movie to a watchlist.

RATING METHOD MOCKUPS

Alternative ideas for the rating method gathered from myself and the product team. We incorporate bookmarking in a few versions as well, since bookmarks also provide a better understanding of the user's tastes, although it carries less weight than the user's ratings. 

 

Although we liked the different rating directions, the best version was also one that was easier to implement.  The version we chose (below) is a combination of tapping then swiping to adjust the star rating.

 

RATING METHOD FINAL DESIGNS

The user can still tap to rate 4 stars and double tap to rate 1 star, the difference is that the user is now prompted to adjust their ratings by sliding their finger on the stars that appear.  The cards are in two columns which allows them to be larger yet still have enough of a selection that users will have more variety in what they are rating and therefore a diversity of items that are rated as well.

We also replaced the full screen with the Hai Avatar encouraging the user to keep rating with a toast alert instead.

We would also go on to refine our rating method throughout the app to tapping on the item which reveals 2.5 stars which the user can adjust later on.

IMG_3536_edited.png

ON-BOARDING MOCKUPS

 

Additions to the On-boarding Flow:

  • Visually we wanted to offset the two columns of item cards to create continuous vertical flow that would encourage users to continue scrolling, and hopefully rating and discovering new things.

  • We also wanted to introduce the ability to edit their domains, initially within on-boarding itself.  We ultimately opted to show users the actual discovery feed and to call out the ability to customize their recommendation interests in the subsequent pop-up tutorials.

ON-BOARDING FINAL DESIGNS

 
 

USER FLOW COMPARISON

Screen Shot 2019-07-17 at 10.14.34 PM.pn

With the redesign, we reduced the overall number of screens/steps while introducing more relevant information for both the user and more relevant information gathering for the AI.

The overall flow was also smoother, less tedious and more engaging.

ANALYTICS

 
Screen Shot 2019-07-01 at 4.57.09 PM.png

Before the redesign: The average rating, skip, and bookmarks per user within on-boarding and the app. (all actions that inform the AI about a user's preferences).

 

We anticipated that there would be fewer ratings overall after we redesigned the rating system which encourages users to put in more effort and choose from 10 different star ratings.  With fewer ratings our goal was that there would be more diverse and therefore higher quality ratings.

Screen Shot 2019-07-01 at 4.59.10 PM.png

After the redesign: We did indeed see fewer ratings on average.  And because we introduced bookmarking during on-boarding we also saw that we almost doubled the average number of bookmarks for each user.

Screen Shot 2019-07-01 at 5.28.19 PM.png

Before the redesign: users on average rated most items 4 stars (the default tap once) and 1 star (the default double tap) with fewer ratings at the extremes of 0.5 and 5 stars.

Screen Shot 2019-07-01 at 5.28.41 PM.png

After the redesign: users on average rated many more items 5 stars and 0.5 stars which are much more informative ratings for the AI.  Even with fewer ratings and while there still many more 4 star ratings, the more diverse distribution of ratings is better quality of ratings to predict a user's taste preferences. 

Conclusion

 

We incorporated updates to the design that we knew needed to be made, aimed for a better understanding of our user's contexts through user interviews, as well as observed their behavior while using the app to draw insights.

We used the original tapping gesture as a lead-in to adjusting ratings, which we also integrated into the rating method in-app. We added bookmarking, and replaced intermediary screens with toast notifications.  We introduced a way to edit their recommendation interests once the user finished on-boarding and entered the app.  Finally we rewarded users with a badge for On-boarding once they finished the complete experience.

What started as an initiative to redesign the on-boarding experience also lead to redesigning the in-app experience, particularly when it came to customizing the recommendation experience and consistent rating methods throughout. Our analytics validated our assumptions and ultimately our redesign improved the experience for our users as well as addressed technical and business requirements.