My manager had asked to talk to me. The conversation we had was one of those conversations that you dream about as a product manager. When I say “dream”, I mean wake up early in the morning wondering what on earth you’ve let yourself in for.

She started with  “you’re not doing much at the moment are you, Matt?” and then swiftly followed up with “I’ve got something really interesting for you”. I knew I was in trouble. When she said “I’d like you to be the product manager for a procurement exercise” I realised that I was somewhere beyond the limits of my digital product comfort zone.

She finished with “Oh, it’s one of the largest contracts for the Census, which is the biggest peacetime operation in this country, and you’ve got six weeks”. I think I said something like “sure, could be interesting” [1].

“Product managing beyond products” is the title of the recent national #ProductPeople get together in Manchester. It’s also the best summary of my experience being the product manager for a procurement exercise. I was lucky to be given the chance to talk about this in the Lightning Talks section of the event in the afternoon: thanks to Jon, Arfah and everyone else involved for organising the event (and for not doing it in London). Read on to find out what happened, how many existential product questions I confronted, and what my experience means for other product people.

1. Product skills can be useful outside product land

One of the first things that I learned was that, even outside of the context of a digital service, I had some product skills that I could put to use. The two examples I want to talk about are the backlog and scored questions.

The backlog

Lots of good work had taken place and there was a big pile of requirements for the devices that we were procuring. The problem was that this pile of requirements looked a bit like my attic: piles of stuff left by different people at different times for different reasons.

Many post-its gave their lives to allow me to get the requirements down into one list; eliminating duplicates, grouping them together, identifying areas that needed more information, and even identifying a few gaps that became obvious when you put everything together. Once this was done, I created a new list where I removed everything that wasn’t critical, including anything I wasn’t sure about. That produced a list of about 20 requirements, rather than the original three figure number.

I feel bad even writing about this: it’s an embarrassingly basic product skill, and one that any product person who has ever wrestled with an unruly backlog – and that’s almost any product person – knows. I think this is the point. In a different context, that basic product skill was critical. What we did next was go out with an RFI, or Request for Information: a procurement option that allows you to go to the market and ask suppliers for indicative costs, without any commitment to purchase.

Had we gone with the original requirements list we’d have either got no responses, or some completely unrealistic ones. With the new list, we had enough response to know that – for the first time on this piece of work – what we wanted might be possible. And at the volumes and timescales we were looking at, that was important. We subsequently added to the list, but it was our baseline to work from – our procurement MVP, if you like.

The scored questions

The Evaluation Questions, the Scoring Matrix, the weighting… in a procurement exercise, you spend a significant amount of time working on and discussing how you’re going to assess the bids (or tenders) that you get in response to your requirements. This happens at a fine level of detail, and it takes more time than you expect. A lot more time than you expect.

Seen at DVLA - Definition of Done by GDSTeam, CC-BY-2.0

After I’d stopped obsessing about the Cobra Effect, I realised that this wasn’t as strange as I had initially thought. What is the single aspect of backlog management that I spend more time on as a product manager than any other? [2] It’s making sure that there are good acceptance criteria on each story. Acceptance criteria checks that the work can achieve the outcomes specified for it, just like a good evaluation question. (Scored questions are a little more like test criteria, where an answer has to succeed at more than one test in order to deliver value.)

2. A product perspective has power

As time went on, I realised that it wasn’t simply skills, like the ability to prioritise a backlog or determine a scored question, that could help. My most significant contributions to the project were to make use of some perspectives that will be familiar to any product people who’ve worked on digital services. I want to talk about two of these perspectives, because using them in a different context helped me realise how important they are.

(I think of perspectives as approaches that are a little more flexible than “mindsets” or “templates” and that can be used in different circumstances. These examples both use lean/agile approaches that will be familiar to product people but less familiar in the world of procurement, a world sometimes caricatured as inflexibly waterfall by definition.)

https://xkcd.com/198/

The thing you thought you wanted turns out not to be the thing you need

What do you think is the most important skill that a product manager can have? [3] I’ve thought a lot about this, and my answer is one that you’ll never find on a job spec. I think the most important skill that a product manager can have is the ability to openly, honestly and with humility admit that they were wrong.

If you are serious about user research, then at some point you are going to find out something about the way your users interact with your product that goes entirely against what you were expecting to find. (If this doesn’t happen, then there’s probably something wrong with your user research.) When this happens, you have to confront the fact that the thing that you thought you wanted has turned out not to be the thing you need.

This leaves you with two options. You can pivot to what users need or you can batten down the hatches and double-down on the importance of your original vision, whilst suggesting that what the users actually need is – or should be – the responsibility of another project. The first option can often look and feel like making something beautiful or perfect, because you have a long time to spend on polishing it; the second option can feel rough and ready, even if it does deliver the maximum value to the user. [4]

via GIPHY

The procurement exercise we were engaged in was to purchase large quantities of physical devices – phones, laptops, tablets, that sort of thing. When we started, we were fairly obsessed with the devices that we would be ordering. What size of screen would we need? What operating system? At one point, I got so fed up of sarcastic quips from people wandering past my desk that I suggested the next person who asked me to order them an extra iPhone would have to take over from me.

As time went on, I realised that we might need some minimum specs in a few areas but otherwise the device specs were the easiest bit. Census is a time limited exercise. Almost all of these devices would be distributed on the same day to be used by people for a short period of time, without a lot of prior training or experience and no guarantee of digital skills. I started to realise that it wasn’t the difference between a 5.5″ and a 6″ screen that would make the difference, it was what happened when someone forgot their password; it wasn’t the operating system that made this exercise work, it was whether the device build setup could be automated across standard wifi or whether it had to be manually set up by a technician over several hours.

We could have doubled-down on our initial vision, insisted that we didn’t have much time, were successfully delivering to our original scope, and committed to order significant numbers of devices while leaving the big problems (like “how do we get these devices set up for users?”) for someone to retrospectively fix later on.

No-one on the team wanted to duck problems that we knew about so we pivoted from looking at devices to looking at the system(s) that managed the devices. This meant formulating requirements that covered automated password resets, contact centre resolution times and the remote management of devices. The value of this approach became apparent late on in the exercise when we were able to accommodate a change, because all of our work could scale.

Discovery over false certainty

When working with an iterative approach, what is the greatest temptation that a product manager faces [5]? I’ve thought a bit about this one as well, and I think the answer is “false certainty”. That temptation to step beyond the prototype or experiment and tell your stakeholders that you’ll definitely be making “one of those”, the time you tell someone that their request will be delivered in exactly nine sprints from now, or when you say “our users need” rather than “this might help our users with…”. No-one likes admitting that they don’t know everything. It will always feel difficult to say “I don’t know”, even if you add “… yet” to the end of that statement, and this is how we fall into the trap of false certainty.

How do high performing proddev teams "talk product". I went back through notes from the last couple wks. What stands out most is how they confidently (yet humbly) embrace uncertainty

The difference between order takers and true product teams is stark #ux #prodmgmt #design pic.twitter.com/R4tQiLiKHJ

— John Cutler (@johncutlefish) December 28, 2018

I think False Certainty is such an ever present danger that I made a sticker to remind me. I ordered some to take along to the event, and they arrived in Cardiff on the day that I was talking in Manchester. That means there’s a pile of them on my desk: please get in touch if you’d like one.

Certainty was what people were asking for in an expensive and complex procurement process, where everything had to be scrutinised and approved by legal people. And there were a few areas where we didn’t know the answers, and where we worked out that we wouldn’t be able to find out the answers in the timeframe that we had. We had to put some detail in, and the temptation was to take an (educated) guess.

We didn’t guess, and that was hard, because it meant owning up to the fact that we didn’t know. It was also hard because this meant we had to build in some flexibility to the requirements, and flexibility in requirements means a lot more complexity because it can affect other requirements and they all have to be clear and fair. If you can’t specify every process running on a device, how can you specify battery length? You can’t, so you have to specify a minimum level, and that’s harder.

We knew that users of the devices were going to need to perform some different tasks on the devices; we didn’t know exactly how they were going to perform all of these tasks. We could have given our suppliers some false certainty with a list of applications and websites. What we did was tell the truth and give potential suppliers details about the applications and websites we were considering. We did this along with a requirement for centrally managed IP blacklist and whitelists and the ability to specify any app available from an official appstore as part of a device build a fixed amount of time in advance.

These were difficult discussions: we were (entirely correctly) challenged over every single aspect of the exercise that wasn’t locked down and described in as much detail as possible. Having these discussions made much more work for us in the short-term. But we avoided providing false certainty, we found a way to be honest about where we were whilst ensuring a fair process, and we didn’t lock our future selves and colleagues into a constrained landscape where change – even one additional or different app – would have come with a cost. We opted for discovery over false certainty, and we made space for it to happen in a way that was consistent with the context we were working in.

3. You can learn a lot working outside product land

I’ve talked a lot about what I was able to add to the procurement exercise, mainly because it was a surprise to me and I hope my experience might encourage other product people to try product management somewhere beyond their comfort zone. I need to add two important qualifiers before I finish

Firstly, please don’t think that I’m suggesting anything more than that I was happy to play a part or that I added more than any of the other members of the team did. I didn’t, and they were all brilliant. I said earlier that no-one in the team wanted to accept throwing a problem over the wall like a dead cat for  someone to discover and deal with later on. That commitment to deal with every problem we uncovered despite the challenges was only part of what made it such a great team to be part of. I also have to thank my manager for having the faith in me to be able to adapt and contribute.

Secondly, I gained more than I contributed. Working in a different context doesn’t just force you to look at things differently, it makes you develop skills differently. In particular, this experience made me think about developing a product narrative, the story about why we’re doing what we’re doing, and how we’re going to do it. Working on a narrative that is given out to people representing some big organisations in a London hotel suite to try and persuade them that it’s worth them bidding for this work: that’s a one shot scenario that has to work. It makes all the time we have to spend with our teams and stakeholders to talk about vision and value seem like a luxury, and some of the chances we pass up to establish a narrative seem careless.

The End. So what does this mean for me?

The story had a happy ending. As part of a great team, I helped ensure that we completed the procurement successfully despite complexities, tight timescales and more appendices than you can shake a duplex printer at [6].

If you’re a product person, you should know that you have lots more to contribute to work outside of your comfort zone than you think. It also means that you have lots more to gain from it than you think. If you get a chance, give it a try.

[1] There were a lot more words than this and it was a lot more pleasant and less abrupt than this makes it sound, but these were the bits I thought about when I woke up the next morning and wondered if I had dreamed it all, like the dream segment from a Mog story.

[2] This is an honest reflection and not an ideal scenario. I know that we should all be working towards the point where the product manager doesn’t have to do this and the team all understand how to write perfect stories, and that the fact that we haven’t got there is at least partially my fault. I promise to try harder.

[3] For those keeping count, this is the first existential product question in this blog post.

[4] Like Kanban boards, I am always suspicious of visions and directions that look too easy or clean cut.

[5] And that’s the second existential product question.

[6] Reader, we almost ran out of alphabet. Appendix Z really did happen.