5½ Lessons From One Year as a Yammer PM
5 January 2017 ‧ 8 min read
It is hard to believe, but it has been more than a year since I joined Yammer as a Product Manager. Looking back over this year, I realize that I have learned a ton. I knew when I joined that the Yammer way of doing product management is very thoughtful and methodical, and that the PM team is amazing. Those two factors meant for me that I was able to hit the ground running, while also developing rapidly every day. In this article, I want to share some of the things I learned in my first year at Yammer, and I am excited about everything that I am still going to learn!
1. Working with designers was tricky at first for me
I’m not going to lie — for me personally, finding the right way of working with the designers on the awesome Yammer design team was the biggest challenge in the beginning. I truly admire our designers for all those skills that I don’t have, and their unique contribution to the projects I work on. I think, part of why I found it hardest to find the right mode of working with designers at first is that I could least put myself in their shoes. I was an engineer myself, and the work of researchers and analysts is similar to some of the activities in my previous job in consulting, but the creative work processes of designers are somewhat harder for me to truly relate to.
In the beginning, I found it hard to align with the designers on my project about when to think divergently vs. convergently, and had difficulty getting full buy-in from the design side into the features we were building. Since those initial days, I have learned to spend a lot more time in the beginning of the project building context with designers: Sharing not only the goals and hypotheses, but also all the research and analysis that had gone into why the feature was prioritized. That also entails discussing where I am more certain about what should be done, and where I have fewer ideas (but still being open to talking about everything). It also of course means that I need to remain open in my interpretation of the information until the designer and I have found a common way of thinking about the project.
The second lesson was being very explicit at all times about who is doing what, when the next touch points are, and by when what phase of the work will be completed. That has helped tremendously in not having different expectations about what stage of thinking we are in. Based on some common templates for project setup that the design team has developed, I now like to first align with the designer who is working on a project on the following points: What are we building, why are we building it, what are we NOT building (i.e., what is out of scope), what constraints should we keep in mind, what are some initial considerations that come to mind, and by when do we want the design (or the next milestone).
2. Being data-informed is awesome
One of the great things about being a PM at Yammer is the amount of data we have available to base our decisions on. When we are planning the next product improvements, we can get very precise numbers on the percentage of users who use various features of the product, and in which way. The way we do product management at Yammer is data informed: We A/B test almost everything, and features that don’t improve our key product metrics are rarely shipped (we kill about half of the features we build based on the A/B test results).
Especially right after I joined, I spent a ton of time looking through data and trying to understand the makeup of our user base. What features do they use, how often do they come back, are there any patterns in their behavior? Building up this picture helps tremendously in prioritizing what to build and which areas of the product to improve. Our great analyst team at Yammer analyzes A/B test results and does more in-depth analyses (like running regressions to find leading indicators for “good” user behavior), but basic querying of the data is available to anyone on the team, and I make a lot of use of that.
Working in consulting has taught me to always have data to back up decisions (but in many consulting projects, we often did not have the best quality data we wanted). In my product management job, I have (almost) all the data I could possibly want, and if it’s not there, it’s often a very quick thing to add some logging for the thing I’m interested in. I believe that having a robust logging and analytics pipeline is one of the most important non-product features to build for any SaaS application.
3. There is no substitute for talking to users
Having talked so much about the quantitative side of informing product decisions, I have to do a bit of a 180 and add that without talking to users, the data can only tell you so much. I love watching real people engage with our product, or ask them questions about how they use it. Every additional user conversation I have, I learn something new that I didn’t know before. One area in which I keep learning is around use cases: Yammer is made for companies and other organizations, and every organization is different in what they do, but also their culture, their environment, etc. This means that they have different requirements and wishes in terms of what groups they create, how Yammer conversations unfold, and what their challenges are.
The people using the product of course differ on a more personal level as well, which is hard to tease out from just looking at the data. In a recent example, we were testing a new version of our groups discovery page. The A/B test results were somewhat inconclusive — the new version fared a bit worse in terms of getting users to join groups, but that was partly explained by the fact that the join action was de-emphasized in the new design. Within the team at Yammer, however, many people liked the design of the new version better, so in order to make a ship/kill decision, the user researcher assigned to the product and I decided to run a usability study with about 8 people. In those sessions, we observe an individual user navigate through the product, trying to do certain tasks we ask of them, and interview them about why they were doing what they were doing. What I learned from these sessions was that there was one group of people that preferred information more in a quickly scannable and condensed “list” form, whereas others were more visually oriented and preferred a more spacious, tiled layout. This difference seems a completely obvious potential distinction in retrospect, but just by looking at the metrics from the A/B test, it would have been impossible to get to this nuance.
4. The cost of complexity is hard to underestimate
Much has been written already about the need to avoid feature bloat and the value of simplicity. Over time, however, it is almost impossible to not have a product that is increasing in complexity. This complexity slows things down tremendously. In a way similar to how adding features to a complex system becomes harder and harder from an engineering perspective, the same is true from the product perspective.
One example in which this complexity not just slowed down a project, but essentially stopped it in its tracks: I was working on a workflow feature to mark conversations to be “read later” — in a way, a flagging feature to follow up on a conversation. What made the conceptual design of the feature hard was that each conversation in Yammer already has two related but different states for each user: A conversation can be “read” or “unread”, and “seen” or “unseen”. I won’t go into the details what these mean (since that is indeed very complex), but for the “read later” project it meant that we had to think through all the various interactions that these different dimensions would have, and what a transition from one state to the other in one dimension would mean to the others. This discussion not only took a very long time, but also led in the end to the decision to not continue the project — mostly since we realized that two dimensions were already too complex, and adding a third dimension would make things exponentially worse.
There is only one remedy to product complexity: Regularly simplifying by culling unnecessary, rarely-used features and concepts from the product. This is always hard to do, because it has no immediate value to existing users — the value only comes later, when new, more valuable features can be added more quickly and easily. There are probably three ways to kill features: Just remove and hope no one notices, remove something but ship something more valuable at the same time (and explain that the removal was necessary to ship the improvement), or try to explicitly explain why the removal was necessary — but don’t hope for much understanding from those users who had built their workflow around your obscure feature!
5. I changed how I prepare for meetings (somewhat)
One of the strengths I bring to product management from my former job as a management consultant is meeting preparation: Making sure there are clear objectives to get out of the meeting, all the information to make a decision is collected, the arguments are carefully laid out, and there’s a compelling story. All of this preparation is critical in consulting: You might only have one shot at the meeting with the important senior executive, and they’re also paying you tens, if not hundreds of thousands of dollars for the project, so you better make sure everything is perfect.
I had to adjust a little bit from that mode of operation in my PM job. I still strongly believe in meeting preparation, since it makes the meeting more effective and reduces the amount of collective time wasted. I don’t make slides for every meeting anymore, but at the very least I will have a clear idea of the objectives I want to achieve and how to talk through them. However, I have started adding more informal alignment and preparation. For example, before a product review I will have mentioned the general direction the project is taking to our head of product and gotten his reactions, so that I know that the path the project is on will neither be a complete surprise nor something that seems unlikely to get everyone’s buy-in. Similarly, I might drop by the designer’s or lead engineer’s desk and mention an idea that I had to see whether they even think it’s worth and feasible to pursue.
Finding the right balance between this formal and informal preparation of discussions is something that I keep working on, in order to maximize the effectiveness of my own time, and minimize time wasted — both by myself, if I go down too far a wrong path, and by others, when discussing half-baked concepts that could have been improved simply by spending more time thinking about it.
5 ½. Bonus: What seems easy can be really hard
This is not really something I had to learn, but I might not have been fully aware of the extent. One of the projects I am working on is giving users the ability to edit messages after they posted them. This is the single most requested Yammer feature, so building it seemed like a no-brainer. Multiple times, we had gotten user feedback along the lines of “why don’t you have this feature yet? It’s so easy, I can even tell you where the button could be!”.
Of course, what makes this feature hard is not deciding where the button should sit. It’s not even the trickier product decisions, like the interactions with all the various parts and states that a conversation on Yammer could have. The enormous difficulty in this project came from the fact that it fundamentally changed one of the assumptions that a lot of the Yammer architecture had been built on: A message, once posted, can never change. The project has already taken way longer than originally estimated while the scope has been reduced. This is of course a typical problem of technology projects, but doesn’t really happen often at Yammer since we try to run very small, iterative projects that are easier to understand completely and estimate the effort of. For the message editing project, however, there wasn’t really a good way to break it up into pieces that would still be valuable to the end user: It is simply not acceptable that by editing, a thread can get into an inconsistent state, or show up differently on different devices, or disappear from search etc.
We are getting closer to shipping this project (fingers crossed!), but next time I embark on a project like this, I will try even harder to break down big chunks into smaller pieces, while also gathering more information on complexities and interdependencies early on, and reserving the decision to stop a project like this if from that initial research it becomes evident that the return-on-investment calculation has fundamentally changed, or that due to dependencies, now might not be the right time to run the project.
I hope you enjoyed these points, even though they might be a bit of a random and personal collection. If you did, feel free to follow me on Twitter where I share interesting product management articles I come across daily.
This article was originally published on the Yammer Medium blog