Nevo example image
Caleb Kingcott
Nevo example image



Designing an AI feature for Spotify mobile

Spotify currently uses AI to recommend music to its users. However, the use of AI for music exploration is expanding and continuously improving so I'm setting out to discover if the services use of the technology can be taken further!

ROLE:

Strategy, Research, UX Design

MENTOR:

Julien Deveaux

PROJECT SCOPE & TOOLS:

80 hours: Pen & paper, Sketch, Invision


Phase 1: Discover

Secondary Research

Spotify does an excellent job of helping its users discover new music. Despite this, music taste is wildly specific to an individual which means that there is an exciting opportunity here to improve upon music exploration within the Spotify app. Before designing, I needed to understand specifically what opportunities there were and how we could deliver on those to offer a better product.

I discovered that AI is used in different ways by various streaming services. Some brands used AI to help users browse for music using voice search, others used it to recommend tracks. I then looked into how Spotify recommended music technically as this would give me greater insight into how the proposed feature could integrate into the existing infrastructure. I discovered that Spotify uses three recommendation models in combination to recommend music to users. These include collaborative filtering, Natural language Processing (NLP) and raw audio models.

View a full summary of Spotify recommendation models

Spotify Recommendation Models Diagram

A technical diagram of the Spotify recommendation system

Contextual Enquiry

At this point, I had a good understanding of the market but needed to understand the typical Spotify user and what their goals and constraints were. I conducted a contextual enquiry with three regular users of the Spotify Android app and learnt the following:

  • Spotify users rely heavily on AI suggested playlists
  • Environment impacts how the user uses the Spotify app (earphones in public, Bluetooth speakers in the car).
  • The type of music listened to is heavily influenced by the user's mood and activity. For example, one participant listened to upbeat music when tired and driving long distances.
  • Phase 2: Define and Ideation

    It was then essential to define the user, the goals (user, business and technical) and the direction of the AI feature.

    User Persona

    The user persona painted the picture of a general Spotify mobile user who relied on the app to listen to music on the go.

    User Persona

    Goals

    Research had revealed numerous user, business and technical goals and I set these out as they would all influence the new AI feature, both directly and indirectly.

    User, Business and Technical Goals diagram

    Ideation

    I ideated on various AI features, making sure to design for user goals and constraints. I could see potential in using voice search to help users explore music but concluded that users tended to scroll through recommended playlists rather than search. Instead, I decided to expand on the music recommendation AI by adding real-time user data such as current location and accelerometer feedback. Practically, this would mean that music could be played/suggested to the user which is suitable for the activity (e.g. walking, working) and the environment (e.g. hot weather, car journey).

    Rough sketches were drawn in the initial ideation process

    Affinity Diagram

    I considered various versions of the feature including an auto play button and a more integrated design.

    Affinity diagram laying out potential implementations

    Task Flow / diagram

    I also considered how the feature would integrate technically into the existing Spotify recommendation models.

    Phase 3: Design and Prototyping

    Invision Prototypes

    I prototyped two different versions of the AI feature, designing the screens in Sketch and then building these into functioning prototypes with Invision. This workflow would allow ‘lifelike’ testing of the app feature on a physical mobile.

    Prototype 1: Spotify AI feature

    Prototype 2: Spotify AI feature

    Phase 4: Testing

    Testing with context

    I then tested the prototypes with three participants on an Android mobile in the appropriate environment. It was very important to me that there was context so if a user tended to listen to Spotify in their car, that’s where the test would be. The format was relaxed and the participants were given space to explore the app as well as discuss their thoughts and actions.

    Initial testing revealed that users could potentially find the feature useful and didn’t seem to have a problem with the use of their data. Spotify users trusted the brand and could see how the feature would be valuable. However, it became apparent that the current design blended in too much with the existing interface which essentially hid the feature.

    A participant, Mary, using the prototype before a long car journey

    Iteration

    In a process of iteration, I took the testing feedback on board and designed / redesigned the following:

  • A popup that presents a customised playlist when a new environment / activity is detected
  • New iconography, graphics
  • The creation of new playlists to suit mood / environment rather than suggesting existing playlists
  • Iterated AI feature

    Conclusion

    Reflection

    I discovered that Spotify users are trusting of the service and are happy for their data to be used in order to improve music suggestions. While users want to feel that music suggested on the app is fully personalised to them, they also want this process to happen almost automatically and don’t feel the need for further controls. For that reason, the proposed feature uses multiple sources of data to further personalise music suggestions in an expected and unobstrusive fashion.

    Next Steps

    It is worth bearing in mind that the research conducted was mostly of a qualitative nature. While it’s been very effective for discovering opportunities, quantitative data will be needed to prove the effectiveness of this feature and it’s design. To take this project further, I believe that a functioning prototype would need to be developed which could be released to active users via A/B testing. Over time, user data and feedback would give an accurate idea of the success of the AI feature and would be an ideal place to move forward.