I spent last Friday at Refresh, a conference for Product, Design and Front End people that takes place every year in Tallinn since 2015.
I don’t go to conferences often, but when I do I like to come back with the feeling that I learned something new and that I interacted with fine, like-minded people. I like going home with a new vision on an old topic, a new topic to dig more into, or a new idea to try out.
Taking handwritten notes is my way of making sure I can focus even when there’s a difficult or boring presentation, and it is how I can take out specific bits of useful information. I tweet some of that during the event to share the love with the online community; and when I return to work, I also write a quick summary of these things to share with my work mates (like this post). This last bit is something many of us are putting into practice in the Product & Design department at Pipedrive as a way of sharing knowledge.
I had been to the first edition of Refresh in 2015 with very little to no expectations and I left with a positive attitude. Presentations, content, venue, food and people were great. I couldn’t make it to the 2016 one, but this time around they managed to deliver again. While in 2015 the presentations were in one single stage mixing up all the topics, this time around they had two parallel tracks: one for Product & Desing; another for Front End. I spent the full day at the… guess? Product & Design one, of course.
Below is a summary of the key things I learned during my three favorite presentations. Things I assessed in these:
- The topic: was it of my interest and did the content deliver on it?
- The presenter: did she/he did a good job delivering the message?
- The slides: where they digestible and well-prepared?
- Applicability: Can I do something with this when I get back to work?
Here they are, from OK to the best:
Number 3: How to design Experiments that Don’t Suck
Willie is a very dynamic presenter (spoke too fast at times) and I personally like that style. I am interested in experimenting and research in general, since this gives great direction to my work as a Product Manager. Willie delivered 3 specific lessons all contextualized in his own experience running experiments at Dropbox. He started from emphasizing the importance of being methodic when running experiments, and he did so by quoting the Mythbusters:
— Maria Lasprilla (@marialasprilla) September 8, 2017
He dissected the definition of the hypothesis and how getting it right is basically getting your experiment design right, since a hypothesis contains in a single paragraph the [execution] of your experiment, the [metric] you think will be affected by it and the [assumption(s)] you have. If you get those wrong, your experiment will not work.
While there were three lessons, I caught only two. I get easily distracted (which is why handwriting helps me as well) and I seemed to have been mentally elsewhere when he mentioned the second one. But here are the ones I got:
Lesson #1: Do not blindly follow successful experiments by others. Just because it worked in their context does not mean it is the right experiment for you to run. Basically, all those out-of-the-box experiment tools are not real sh*t.
Lesson #2: (dreaming of the next coffee break…).
Lesson #3: When do you give up experimenting? When you have stopped learning.
The main message of Willie is that experiments are useful not necessarily if they pass, but if you learn from them and, my own addition: if you apply those learnings. Experiment, learn, iterate, rinse and repeat. Quit when there’s no learning.
Number 2: Designing for Learning Applications
Jonn was a very articulate and clean presenter. His presentation was neatly organized and went from one point to the next in a very logical way. This was great, but maybe because his tone of voice was too even all along or because it was the second part of the day, towards the end of his turn, it got hard to follow.
I chose to pay attention to this presentation because teaching is very dear topic to me (I was a teacher for five years). I could also relate with his approach and the work we do at Pipedrive.
Jonn talked about the importance of understanding what the process of teaching was like when building a learning application. He broke it down into steps that he then correlated with the process of product design. He also talked about the app as a teacher, and the user as a student. I could see the common elements with the work we do at Pipedrive, and I cannot imagine a product manager or a designer doing a good job without understanding the process of the work done by the professional or the person for which they are trying to solve a problem, or the role they play when using the app, or the app in itself. It is that bit of knowing the area of business in which you are that make some product managers or designer strongers than others.
I am not strong at all in sales, but if there’s a topic I have learned a lot about in the last two years that’s the one. The same happened when I was working for fintech software companies, and I’ll repeat it in whatever business area comes next.
Jonn’s bottom line was that to have a great foundation to build a (winning educational) app you need to have:
- A product concept
- Strong back-end, data,…
- A great team
Number 1: Machine Learning from the Product Perspective
Inga was by far my favorite. She was dynamic, with a perfect pace (not too fast, not too slow), delivered clearly applicable information with beautifully designed and digestible slides.
Although many presenters are great at making Machine Learning sound like a couple of buzzwords that get randomly squeezed into their presentations, Inga delivered on what she promised. She covered:
- What Machine Learning is
- What problems are good to solve with it
- What can product managers do when considering to work on a product with ML
- Why we should care about ML
She quoted Arthur Samuel’s definition from 1959:
Machine Learning: Field of study that gives computers the ability to learn without being explicitly programmed.
Data is critical to building a Machine Learning Model, and this data should be ideally labelled. There are several problems that ML can help solve, but mainly:
- Decisions with many outputs
- Decisions with many inputs
- Decisions at scale
One example she brought and that I noticed myself while using the app a few weeks ago were the Slack Highlights:
My favorite part was how she made clear what product managers needed to bring into the table when considering using machine learning in a product:
- The problems: we bring the problems that we think can be solved with the help of ML
- The data. We need to make sure we have the data needed to feed the model
- Alternatives: we need to consider alternative non-ML solutions, because this is a time consuming expensive approach. In most cases we will be able to solve the same problems with simpler tools / techniques
- This is what Inga highlighted as the most important bit if we went down the ML path: Designing for when the ML Model fails.
She also covered how a product development cycle was different when it was being built using ML and a bit that I much wanted to bring with me but could not capture (my mind was boiling with ideas at that point) was what were the converging trends that indicated why we should care about ML.
It was a lovely event and I feel I came with ideas and refreshed knowledge. I’ll see about next year, but first I need to get these new ideas and knowledge rolling.