Chris Butler talking about the future of decision-making in an era of AI.
Chris Butler gave a talk on Decision-Making for Product Teams to the Chicago ‘Product Camp’ folks.
It’s worth a watch/listen.
[00:00:00] Ben Senior Product Manager at McDonald’s: There you go. All right. So let me, let me kick things off for today. So just to introduce myself, I’m Ben Edmiston. I’m a senior product manager at McDonald’s and also the, uh, the, the president and, uh, grand guba, I suppose, for product camp Chicago. I have been for about the last four or five years a little bit about product camp real quick, and I’ll be quiet about it.
[00:00:26] We’re a nonprofit organization within the Chicago area. That’s all about teaching, learning, discussing, sharing great ideas around the best practice of product management. So to that end, we We host a number of events. Uh, a lot of in person activities. Uh, floodgates. Here comes everybody. We usually have one big events in the fall each year.
[00:00:51] There will be one coming up toward the end of the year here and what we’re experimenting with today and would love to get your feedback in a follow up survey. We want to not just have it be a one time interaction across the Chicago, uh, and extended product community, but also want to have, uh, webinars kind of interspersed throughout the year, too.
[00:01:12] So, Chris Butler is joining us here today. Very happy to have you back. Chris virtually. Um, Chris was actually our keynote speaker at the at the product cam Chicago event. We posted. We hosted here in MHQ last fall and by popular demand. He’s back with us today to talk about decision making for product teams.
[00:01:36] So, Chris, I’m gonna go ahead and turn turn things over to you. Go
[00:01:40] ahead and take it away.
[00:01:41] Chris Butler: Yeah, great. Thank you everybody, um, for being here today, and, um, I, I’m really excited to kind of share this discussion, and I do want to make it a discussion, by the way, so as much as I’m going to be speaking, um, right now, I will have a couple different points where I’d love for people to just, like, join in the discussion, and that can either be through chat if you’d want to just, like, uh, post something, As an answer to my question, um, and then there’ll be a few times I’d love for people to come off mute and maybe give me a little bit of, uh, more information and just talk to other people, uh, within this group.
[00:02:17] And I do want to leave some time for maybe kind of partial mastermind, like, how are people dealing with decision making inside their orgs at the end of this as well as Q and A. And so, um, yeah, anyways, I’m excited to be here. We’re calling this kind of deliberate decision making process. And, um, you know, uh, I, this is some information if you want to reach out to me.
[00:02:38] I mean, I’m always happy to connect on LinkedIn, so, so please let’s do that if, if you find this. And then I, I’d love to hear feedback. Uh, like as I start to talk about these concepts, this is really meant to be from like end to end. How do we talk about decision making within organizations, especially for product people?
[00:02:51] So with that, uh, let’s start with the first question here and you can post this in chat. How many decisions have you made? This week so far only Thursday, so I’d love to hear from people where we’re at so no one’s made a decision yet So far this week Oh hundreds too many infinity simple Every moment of our lives is filled with decision making apparently which okay and not enough I think that’s that is actually a very interesting way to put it.
[00:03:21] Yeah, I think like When I think about decision making, we’ll talk about what we mean when we talk about like decision making in this, this context, but this is great. So thank you everybody for, for posting that. Um, I’d love to hear maybe from one or two people right now. What was a decision that you made this week and kind of how did you do that?
[00:03:40] And if you give me like maybe a 30 second overview. So if people want to come off mute and just. Maybe offer something, something more than deciding to come to this meetup. Anybody, anybody
[00:04:00] feel comfortable coming off mute just for a second.
[00:04:06] Sure. I’ll go. Yeah, please. Hey Chris. Uh, so I think the biggest decision that I’ve had to work through, it’s actually probably a series of decisions this week is starting a new project, getting a set of requirements from the previous stakeholder and trying to figure out. How I want to organize a product backlog and kind of just not just the structure and logistics in JIRA, but just how to provide a framework to prioritize everything.
[00:04:37] Um, beyond just here’s a flat list of requirements. And how’d that go? I’m still in the middle of it, man, but so far it’s, uh, it’s not great. Okay. It just isn’t fun. These aren’t the fun decisions. Yeah, that’s fair. What else? What’s another type of decision? Maybe something different that people have done this week.
[00:05:01] Hey, this is Garrett. I’ll, uh, I’ll try to, uh, if you don’t mind, I’ll try to come off. I’m having some audio issues, some, uh, zoom issues, the classic, but maybe my decision should have been to, uh, update my zoom, uh, or something, uh, we, we, we had to make a decision around, um, I won’t go all the way into the weeds on this, but more or less the best way to, to, um, to quantify and qualify CSAT, um, as it relates to, um, uh, I’m just going to call it a release that we’re working on, um, and some of the nuances that go into it.
[00:05:33] So the decision was around whether or not to include, um, like a certain level of bugs in, um, and satisfaction or whether or not the, you know, that would show through in just the overall satisfaction itself, as opposed to having to itemize it. So without getting into weeds on it, we had like a really good conversation around, um, what it means to, Okay.
[00:05:54] Discern customer satisfaction and how to, how to kind of quantify that. So, okay, great. How’d that go? It went fine. Uh, I mean, I, I think it, it’s gone well. It’s an ongoing dialogue. So there’s, uh, engagement with, you know, some of our research teams, but. All right. Well, thank you. So, I mean, these are all the types of decisions that people have to make in product roles.
[00:06:18] And there’s actually millions of more that, that we have to do. I think the way I want to start, um, you know, with this discussion is I, There’s a bunch of things that we kind of think of as decision making, but I want to maybe call them as, like, we’re not going to talk about them today. They’re related in some way, um, but the first thing is, I guess, like, I usually hear people talk a lot about, like, Eisenhower matrix and the way that, like, how do we decide whether something’s important and urgent and whether we should take action on it or not.
[00:06:44] Um, that’s not what we’re going to talk about today. Um, and actually we’re not going to talk about any other real prioritization methodology. Um, I think prioritization is a type of decision, but it’s a very narrow one, in my opinion. We’re also not going to talk about one way or two way decisions. Um, you know, one way door, two way door type of decisions that is very popularized by Amazon.
[00:07:03] Um, and while I think that’s interesting, it talks more about the stakes of a decision rather than the decision process itself. Right, just because you identified whether it was a one way or two way door doesn’t mean that you’re actually going to now make the decision in a very particular way or not.
[00:07:16] Like, there’s many ways to make decisions for either of those types of decisions. I’m also not going to talk about, uh, uh, RACI or RACI, or any other types of, like, um, matrix, uh, kind of ownership things. Um, and if you go to the Wikipedia page, there’s like 50 or 60 different versions of this. I’m not going to talk about that because this just really talks more about, like, the people in your team and what is the role there, rather than how is it that we actually make that decision.
[00:07:41] Um, and then I’m not going to talk about Spader Rapid, which are, I think, you know, get an honorable mention here as far as decision making techniques. But I would say that they’re probably just like Rossi by another name. Um, and they do include a little bit of process. They do include a little bit more about that.
[00:07:58] Like, how should you start to set up the decision making process? Um, but it really mostly focused on at least who should be making this decision, which is really, to me, just like a very narrow part of that. Um, so now that we’ve set kind of some ground rules about what I want to talk about, um, we’re going to talk about really this like meta technique.
[00:08:15] Um, and this is something that I’ve been writing about on the Uncertainty Project, which is, I’ll talk about that at the very end of this, as like a community for decision making and strategy. But I want to talk about like the way that we end up creating an overall process for decision making. That is not specific to certain types of decisions necessarily.
[00:08:32] Um, You can of course create processes that are specific to a decision. Um, But I want to talk about like a framework that I end up using that is like about all types of decisions that we could potentially make. And so I break it down really into kind of five areas. Right? So one is identification. Do we need to make, do we need to actually enact this process or not?
[00:08:51] Um, there’s a discourse, which I think is very important, and something that people tend to muddy an awful lot between discourse and decision. They kind of jam those two things together, rather than making space for discourse, and then having a decision making point as like a single point in time. Um, we need to talk about like how we communicate it out, um, and what ways we communicate it, who we communicate it to, things like that.
[00:09:11] Um, and then finally, there should be this. Kind of loop of learning, um, and I’ll talk about what I, what I mean by that, um, in a little bit. So these are the main ways I’m going to take you through a decision making, uh, process or meta framework. I don’t, I don’t know what the right way to put it quite yet, um, but that’s what we’re going to be going through, is these, really these five steps.
[00:09:29] And then I want to talk a little bit about how the future of decision making may be impacted by some of the tools that are being built today. So with that, let’s start with identification. And I think when we think about identification, at least from my perspective, I wonder an awful lot about whether we should use some type of heavyweight process.
[00:09:48] Um, and what I mean by that is that we actually make You know, as, as people said in the chat, like we make decisions all the time, like every time that you type something, you’re kind of making a decision about what you should include or not include. Um, but what’s important there is that you’re not necessarily worried about, you’re, you’re worried about the impact potentially, but there’s like a difference in that it’s impacting the way you’re making the decision right now, um, rather than it may impact a bunch of other people.
[00:10:13] And so that’s where I think something like blast radius is a terminology that I’ve started to think about, which is. Who is actually going to not only be impacted by this, but how should they be involved in the decision making as it takes place? And so, you know, doesn’t everything include this blast radius?
[00:10:29] And I would, I would argue that, that no, like the idea of contributing to a decision is one that if you are the sole owner and you’re the person that makes all the decisions about this thing and you’re writing it yourself or Um, you’re going to go, go off and do it yourself. You’re the one that’s actionable.
[00:10:46] And the, the fact is, is that it’s kind of assumed that you will just be doing this by yourself. This is not the process for that, right? Like you should, you should kind of just assume that you’re going to do this. But you should consider like, is someone else going to be impacted by this? And so, you know, I guess this leads to why we should actually make decisions with other people.
[00:11:04] And, um, it’s not just because necessarily that we want to, like, communicate that, right? That is an important step of the decision. And you may actually do something by yourself and immediately communicate it. Um, but there’s a bunch of reasons why we actually should make decisions with other people. And this is related to, uh, thinking, you know, Daniel Kahneman wrote this book, Thinking Fast and Slow, which is a lot about kind of cognitive biases and the way that people create shortcuts to be able to make decisions faster with less information, kind of preserve our resources.
[00:11:35] But that issue about like kind of cognitive biases is is one that I think I think a lot of people try to say we should map all these cognitive biases and then we should be aware of them and we should stop ourselves from having these cognitive biases. Now, I think when we get into the realm of group decision making, this is a task, uh, as an example, that is used in kind of cognitive psychology studies to think about, How do people make decisions when it comes to, uh, the biases that they have?
[00:12:06] And then how might this be impacted, um, by groups of people? And so what I’m going to do is, I’m going to actually ask everybody in a moment to take a look at four cards. And on each one of these cards, there’s a letter on one side and a number on the other. But you’ll only be able to see one side. And so the question will be Which cards do you need to turn to test this particular rule?
[00:12:28] All cards with vowels on one side have an even number on the other. And I want you to only select the cards you think are required to do this. Um, now, I am going to say most people get this wrong, but I just want you to now look at these cards. I want you to try to decide which cards would you flip over specifically, um, to prove the rule that all cards with vowels on one side have an even number on the other.
[00:12:52] Okay, and we can do this by just putting things into chat. So here’s the cards that I’m going to show you right now. So I want you to think about this, right? For these cards, and we’ll go, we’ll go back, um, what cards do you need to turn over to understand all the cards with vowels on one side have an even number on the other?
[00:13:15] So I’ll give everybody a moment here. Which cards would you turn over to prove that rule?
[00:13:22] So I’ll wait in the chat. Just you? Okay.
[00:13:28] Who else? 7 and 2. Okay. U and B. U and 2. Alright, so we have some consensus around U and 2, it looks like. Yeah, that’s great. Okay. So, if you do this, uh, the actual answer is U and 7. Um, so Faith, you got that one right. Eric, that’s maybe too many. But, um, looks like Ben and And faith got it correct in this case.
[00:14:00] Now what’s interesting is that, as I said, most people get this wrong. And the reason why is because they think about things in a way that is maybe the opposite of the way they should when they’re trying to do the solution to this. And usually about 20 percent of the time people get this right. And so, that 20 percent of the time It’s mostly because of the fact that there’s a lot of kind of cognitive biases rather than being like hyper logical that show up here.
[00:14:25] Now, what’s interesting is that when you start to actually have like a group of people working and talking with each other on this particular task, the success rate goes up to about 80%. And the reason why is because of this particular thing, which is the discourse of the discussion about decision making that needs to take place.
[00:14:44] And so I think this is where what I like to do is when we’re talking about identification here, we want to start to understand, like, we need to actually make a decision as a group of people because we want to reduce individual bias in the decision making process around that. And so, the This leads to the last step of identification and kind of before we jump into discourse, which is really this concept of like deciding how to decide.
[00:15:09] And so I’ll talk about how this relates to the discourse side, but it really comes down to a couple different key things. And so when we talk about deciding how to decide, I mean, you know, a couple different like aspects. So one, what the discourse requires, right? Like, so. We realize that this is going to impact other people.
[00:15:25] There’s a blast radius that we probably could do better by actually having a discussion with other people. We want to then figure out who should be in that discourse. And it could be a lot of people, right? But like, who is actually going to help us reduce the bias of like a bad solution and who’s going to bring us additional things?
[00:15:44] We want to then talk about, like, when we should decide, right? So, is there a time frame that this is, uh, appropriate for? Um, is there a moment by which it’s no longer valuable to make a decision? And do we want to timebox this in some way? And then who should actually decide, which is very different than who should be part of the discourse.
[00:16:05] And then how we actually decide that itself, which I’ll talk about during the decision step. Like, there’s a bunch of different ways that you can actually decide that is more than just we all agree, um, which happens to be a default mode. So, once we’ve decided how to decide, we then move into this discourse, which is really about a discussion between all of the people that are appropriate to, uh, to do things.
[00:16:26] And this is really what I’ll talk about are as containers. Um, and I say containers because specifically I mean anything. It could be asynchronous, it could be synchronous, it could be in person, it could be virtual. It could really be anything, but what you’re trying to do is allow for people to engage in some way.
[00:16:42] Um, and There’s a few things that happen inside of this discourse that I think are really important. So one, not only have we already kind of set the ground rules for how we do this, we give people time to think, right? So discourse is not just about the idea that we’re, we’re all arguing and discussing, but it’s also about space to be able to be expansive about the way we consider the problem that we’re trying to solve or the decision we’re trying to make.
[00:17:07] We allow for disagreement. Um, because those tensions inside of the team actually are most valuable in creating better decisions that if there’s only one person making a decision, it’s not more valuable than because we’re still going to suffer from the same biases or limits of information or knowledge or experience that that one person has, we’re going to then create options.
[00:17:26] Right. So in every one of these discourses, we’re building up to this decision point. And so we need to, like, allow for different options. And it could just be two, but it could be three. And it could be more than that, um, if it makes sense. Um, and then we’re also refining options. So we may take certain options off of the table kind of immediately.
[00:17:41] Right. And this is where prioritization could come in. You could use something like scoring methodologies of different options to say, which one’s the best in the case of like, uh, when I was at Facebook. We used a lot of red, uh, like, traffic light tables, which would be Columns for different, like, values or variables, and then rows for different options, and then red, yellow, green type of coloring, and I did start to see that, like, whatever the person originally wanted ended up being the thing that was most green, usually, um, in this decision making process.
[00:18:09] But there’s lots of different ways to kind of, like, refine and talk about the options that are there. Um, and I think the that one of the key aspects here is how do we actually have better containers in some way and um, That that ends up being I think a really hard part when we talk about like time management within Organizations a lot of the time the reason why people no one has time Is because they haven’t actually deliberately created these containers to have really good discourse and separated that discourse from the decision making Um, and so the first thing I want to say is that meetings are not a bug I saw somewhere that there was like a post about this and I know that, I think it’s Shopify, um, earlier, the beginning of last year ended up just cancelling all meetings.
[00:18:49] Of course, then the calendars filled back up again. But meetings are not a bug. Meetings are a reason and actually have been the traditional way by which we get together to talk about things. And maybe people will rail against the idea of like a lot of different time blocks being put on people’s calendars for future meetings.
[00:19:05] Um, and we can talk about like, what does it mean to actually create a good meeting? Um, but meetings are not the problem here, right? Meet, the fact that we for a very long time have used our verbal capabilities to discuss and argue with each other and then make a decision is just the default mode for humanity.
[00:19:22] So meetings are not the problem here. Bad meetings are the problem. Um, and I think the, the, that, that’s something I just want to say out here first. Um, so if you want to create helpful discourse containers, here’s a few things that I’ve found to be really valuable. One is to allow for people to think, which is something that we talked about for the, the, the, the, um, kind of discussion, uh, discourse, right?
[00:19:44] We allow people to kind of collect options, discuss controversy, but give them time to think and give them the space to do that, but have a place to like collect notes and do things like that. Um, you as the person, and I would say that most product managers, part of their job is to actually facilitate decision making.
[00:20:01] And so your job is actually not to sell anything. You can have an opinion, absolutely. But your job right now in this discourse container is to gather as many options as possible and explore those options in the best way possible. You want to time box these types of things. Right? Um, having a meeting where people are just sitting in a room for hours on end, um, doesn’t really create the urgency.
[00:20:22] It doesn’t allow for people to understand, like, how they should start to gather their thoughts in a way that is more specific. And I would say that this is related to the overall discourse container, not just, like, individual discourse containers. Um, When you get feedback during this discourse, I think a mistake that a lot of people make, and that then leads to consensus based decision making, which we’ll talk about in a moment, is that they feel like they have to actually address all the feedback and combine it with the various options.
[00:20:49] There can be options where the feedback is actually taken into account, but there’s also times where feedback is helpful, but can be basically put in the we’re not going to do anything about it bucket. And so that I think is really important. Um, and then finally, I think, like, overall, like, you should start to, like, log some of these things, or if you end up saying that you’re not going to do something, you should log that somewhere.
[00:21:12] And there is something about, like, the idea of, like, how documents or notes end up relating here. I don’t have a lot of guidance on that because I think that it’s, it’s something that is, it tends to be a little bit more per organization. But I think it is important to start to then, like, create an actual log of what this decision is about.
[00:21:28] And there’s a lot of different ways to do that. The last thing I’ll talk about for discourse is really around methods for disagreement. Um, and so, you know, methods for disagreement I think are one of the most important things for us as PMs to actually foster inside of cross functional teams. And so, I’m gonna just like breeze through a bunch of these.
[00:21:47] I will send out links later if that’s helpful, but, um, these are things that actually create disagreement within the organization. So premortems, it’s something I do almost every kickoff. Ritual dissent, which is like a way where you actually sit in a certain way so that people are not like face to face in disagreement, but allows for better feedback mechanisms.
[00:22:04] Double Crux, which is really about the key value that is in difference of opinion. Um, Principle Disagreement, which is kind of partially a methodology and a stance for the way that we actually disagree with the ideas rather than the people. Um, Delegation Poker, and actually all types of poker, um, within this world allow for kind of initial disagreement, but then discussion when we find out that we have different understandings of things.
[00:22:26] Boris, which is like a very weird name for a workshop. But talks about kind of trade offs and the way that we try to now compare different types of trade offs between, say, product and engineering and sales. Um, Lean Coffee like Disagreement. So, uh, one of the things I’ve done here is if we have an, uh, like a set of proposals, I want people to actually talk about what are the things they disagree with and then we have everybody vote for what are the biggest disagreements here.
[00:22:50] Um, and that at least kind of structures this conversation to go after things that are the highest priority, rather than allowing for one lone person to just kind of like monopolize the conversation. And so that leads us then to this decision, right? Which is this single punctuated point, ideally. It’s the smallest amount of time in the entire process, ideally.
[00:23:07] Um, and, and probably includes the least number of people. And so inside of Google, where I was last at, um, inside of, I would, I would say Facebook, not so much. Um, inside of GitHub, I’ve definitely seen some consensus driven decision making, but the sense of being like, I think in tech world type of, uh, jobs, there’s a lot of like, how do we just get everybody to agree?
[00:23:25] Because we can’t move forward until everybody that’s part of this cross functional team agrees. And that’s not the only way to make decisions, I would say. Um, and so, there’s a bunch of other ways. And, and, and this is, um, related to a really interesting, like, app and website, um, and actually a Slack plugin, uh, by this com this group called The Ready, um, I’m sorry, uh, by Noble, which is a kind of design, uh, organizational design consultancy.
[00:23:48] So they created this app by which you can go through kind of a flowchart, uh, or a tree to get to different decision making. But what I want you to think about here is not the tool so much, but I think they’ve done a really good job at the very bottom of this website. They have a bunch of different ways that you can actually.
[00:24:02] Do decision making, and so I want to talk through really briefly what those are. Um, so autocratic, I actually think that you should not be in this mode, in this like heavier weight decision making capability. If you are going to do something like autocratic where a single person decides, they should just decide.
[00:24:18] Like, maybe there’s a need for preparation, but we get into that with like consultation later. Avoidant, it’s okay to defer decisions, right? Like, if, if we don’t have enough information, that’s fine. Information does actually have like If you’ve ever read, uh, you know, How to Measure Anything, um, I think it’s a great book because it starts to talk about the value of information and timeliness of this, but if there is a moment that you don’t want to make a decision, that is completely appropriate for us to make a decision where we push it off.
[00:24:46] Consensus, this is something that is default a lot of places. Consent, which is one that I’ve started to experiment with more, where we focus more on vetoes than on, um, people that are, uh, actually all agreeing. And so. Um, this means that you will say, okay, I’m going to give everybody 48 hours. If anybody vetoes this, we will, we will go back to like decision making mode, but otherwise we’re just going to move forward.
[00:25:10] Uh, consultative. This is where, again, it’s kind of like autocratic, except they’re asking for all of the opinions. So that’s like kind of the default vote here. I think it’s like a less valuable one because you still need to then decide like, is it one person, a group of people, whatever that is. Delegation is that.
[00:25:25] You know, the, the group of people that are now here in this, this room, they’re the wrong people to make the decision and the right person is actually someone else that we’re now going to delegate it to, um, democratic again, kind of like, uh, not a consensus where it’s a hundred percent, but we just need to have like enough of these people to agree and then we’ll move forward.
[00:25:43] And then my favorite is actually stochastic decision making, um, where we kind of use randomness and now this is where people would like if they’re actually in a place where they have kind of. Analysis paralysis, stochastic decision making can help, and I think it gets us into the realm of like, not all decision making is pure data driven.
[00:26:03] A lot of it is actually intuitive. And so from those cases, even just using stochastic decision making, if people can’t make a decision, people will become very opinionated about what should happen if you just decide to flip a coin. So that’s something I just like found in general, but I think it’s also helpful to like probe what is actually valuable or not for a lot of people.
[00:26:23] Um, so. I think one last thing is I want to mention this talk, um, you know, by, by Janice Fraser, and I think when we talk about, like, who is actually supposed to be in this group, it should ideally be the people that have the most expertise, the authority, and the people that have to deal with the outcome.
[00:26:40] And so in the Rossi chart, it’s kind of the responsible, um, it’s the R, A, it’s the R and A people together. That should actually be making the decision. Usually, um, I realize that’s not always the case inside of organizations, but I wanted to just mention, I think this is a great talk and I’ll link to it afterwards.
[00:26:56] But, you know, we, we, we should have less people. And usually the people that are really being impacted, they’re the ones that should be making the decision. The times that it makes sense for leaders to then get involved in decision making is when there’s multiple people that fit this type of thing. And they are not able to agree in some way.
[00:27:14] And so I think escalation is actually another mode where inside of our group, we’re actually just not able to make a decision right now. So we need to escalate to someone else. And so I think that’s another type of decision making in general. Um, so now that we have, oops, sorry. Oh, no, I clicked on it.
[00:27:32] That’s the problem. Sorry. One second. Um, so now that we’ve talked about, uh, these like different ways of doing decision making, um, I want to move now into you’ve made a decision, right? The group of people that are appropriate to make the decision or the single person, right? Like in, um, Amazon, they would call it the single threaded leader or other people might call it the directly responsible individual.
[00:27:57] Um, those people have made the decision and now we need to communicate it out. And when we talk about communication, I think there’s like one key structure that I really like. And this comes from pedagogy where you are, what you’re trying to do is you’re actually trying to understand how this decision now impacts other people.
[00:28:13] And so, uh, I can link to another article where it really talks about this methodology of just like what, so what, now what. And so what you’re trying to build, I guess my key point here is that you’re trying to build a narrative. Really about like, how does this decision now fit into everybody else’s lives?
[00:28:26] Um, and I think there’s some really key things that decision, uh, communication should enable. And one of those is action, right? So, if you made a decision and there’s no action, was that a valuable decision or not? But that should lead to basically other people taking action. It should provide clarity. It should allow someone to read the communication and understand whether it impacts them or not.
[00:28:50] And that means that, um, you may have multiple different communications for different audiences, but they should know whether this is important to them or not. It should also allow for a certain amount of ambiguity, though. Uh, because I think like, people tend to sometimes over communicate the way that things will happen when it comes to decision making.
[00:29:08] But there’s a really interesting paper that I read about this thing called the, I think it was called the, um, the blind spot vision or something, something like that. But it was about this idea that like, Actually, for everybody inside of an organization, they don’t always need to have perfect clarity.
[00:29:22] It’s okay to have a certain amount of ambiguity because they need to be able to, like, link whatever’s happening at a very high level to their thing. And if there’s a certain amount of ambiguity, it becomes easier for them to link whatever their goals and priorities are. And this is not meant to be deceitful.
[00:29:35] This is just meant to be that. If you are overly specific about things, especially to people that are not necessarily part of the decision making process or part of the discourse, um, it may actually create more confusion if it’s more specific in some ways. So there’s, there’s a happy balance between these two things.
[00:29:50] It should allow and enable disagreement, even at that point, because maybe there was someone that you missed that should have been part of the discourse or should have been part of the decision making, and they should then be able to disagree in some way, or at least, you know, be able to talk to the team that we actually have to go back and consider something else.
[00:30:06] And then it should allow for anybody that is actually going to be opinionated to then dig into what is going on. And so, um, if you, if you ever read something like Smart Brevity, um, uh, which is all about this, like, way of, like, writing news stories, it’s really about the headline and then, like, three sentences or bullets.
[00:30:21] And then, yeah, of course, you should provide more information for the people that that have gotten past that and they said, yes, I’m actually the person that needs to care about this. And so that’s where you then link off to like the decision log or document. That’s where you then link off to more information.
[00:30:35] But like you should, you should allow people to like dig in. Um, otherwise you’ll get questions about like, where should I find out more about this? And so I think. When we talk about like delegation and action, right, um, this is a really amazing article by Roger Martin who’s, who’s awesome in, in the kind of the strategic, uh, world, um, where he talks about something called strategic choice chartering, which I think there’s like a real issue for a lot of people that are especially senior leaders that try to make decisions and then, um, you know, try to then hand it off to someone else.
[00:31:07] But the problem is, is that that handing it off to someone else, can feel hard sometimes because what they’re not doing is they’re not actually specifying the parameters of what is being handed off and what is expected back. And so I think this is a really amazing article. If you, if you do work as a leader and you, a decision now has to be taken, um, like action has to be taken on that, that decision and other people have to do that.
[00:31:30] Sometimes setting the parameters of like, okay, well, I’m delegating this part of it to this person. And here’s the, if it goes outside these guardrails, I want to know about it. And I want to be part of the decision making process again, or I want to know afterwards. And this is where delegation poker, again, I think is really like an interesting mechanism to do to kind of figure this out between managers and employees or like leaders and their team.
[00:31:50] Um, but I think we need to do a better job of like actually providing the parameters of an actual kind of, um, you know, delegation. And so this article, I think talks about that a bit. So the final step is really about like learning. And so how do we Learn about the way we make decisions and I want to be very clear that this is really about the process rather than the outcome.
[00:32:10] Um, the truth is, is that inside of a complex world where it’s very uncertain, we have a lot of ambiguity about what is right or what isn’t there are going to be times where we. Um, you know, did the best decision making process that we could have with the information with our systems and the way that we’ve set up our teams, and we still get a result that is not what we wanted.
[00:32:31] And I think we need to be aware that that is a possibility, and that it’s okay that that sometimes happens. What we should do is we should learn from that. We should learn. Was there a process component that in that moment we could have actually done something differently? Um, or is this process still the one that is going to give us overall better results than a different process?
[00:32:51] And this is where I think like a lot of people end up trying to do things like five wise, uh, to understand or root cause analysis, all that type of stuff, to get back to like what happened when we made that decision. I think the issue there that we, we don’t take into account when we do that type of root cause analysis is we don’t actually look at Was there any way that that person could have made a different decision?
[00:33:11] And if they could have, that sounds like a process thing that we should change. But if the system was set up by which they had the information they had, they were kind of, not forced, but they were kind of incentivized or, uh, focused to make a certain type of decision, that is an issue then where we should change the other types of processes that are surrounding them.
[00:33:31] Rather than saying you did a bad job at this thing because the system was the only way you could have made that decision. So anyways, I try to bring these types of like complex system thinking to decision making and I think we get get at like root cause analysis. We get too much like finger pointing this person should have made a different decision, but could they have if you were in that same situation?
[00:33:49] And that’s where like. Models like the veil of ignorance is interesting, I think in strategy, but I digress about that. Um, so techniques for learning, and here’s a bunch that I think are just like really valuable, especially decision making processes. Of course, retrospectives, postmortems, whatever you’re going to do.
[00:34:04] If there’s a really important one, you should follow up on it, and you should see how it actually turned out and how the process worked. Um, maybe even before you know the results, you should talk about the process. Setting tripwires. So when you do delegation, there’s certain kind of guardrails you may put on this, but also you may say that, like, if this decision results in a particular type of outcome, we want to come back and revisit this type of decision.
[00:34:27] And those are referred to as tripwires, which, again, I can share something about, like, the way to think about tripwires, but these are things that either could be automated from a metric standpoint, Um, or they’re based on news and the environment or behavioral stuff, but we want to set up like, when do we know that we made a mistake and we need to like revisit this, um, creating a decision log overall, like if you’re doing the work to actually create these documents, you should log them somewhere else.
[00:34:51] And that’s because it actually ends up leading to this idea of, like, auditing for themes. And so, um. I think as we start to think about all the ways that we have to make decisions, there are types of decisions that may require a process. So like, we may think of something as a very complex realm type of thing, but we make enough of these decisions and we know that this decision process usually results in a better outcome, that we should now kind of standardize this in some way.
[00:35:14] We should make it more of the regular way to make this type of decision. And then I’d also say that when we look at things like escalations inside of teams, if we talk about strategy as kind of Enabling constraints being pushed down on the team, like here’s where you should go in this direction, but not this way, um, escalations that come up the organization and the themes around those escalations end up saying that there’s like a part of the strategy that maybe is not formed enough that people understand how to make those hard decisions then.
[00:35:42] And so I think doing audit for themes, looking at themes of these types of things, um, end up being really valuable, I think, especially to senior leaders that are trying to set strategy. And then finally, like, we should always be asking the question, like, was this process worth the cost? And we involved a bunch of people, we, you know, talked for a while, we made a decision.
[00:36:01] Was it valuable to do that? And next time, should there just be one person that just makes a decision autocratically and moves on? And, and so that’s something we should always be asking about this type of thing. So with that, I would love to hear, you know, maybe just in chat, you know, how many people have actually Um, you know, analyzed your decision making process, maybe how have you done that?
[00:36:30] And you can go and chat or people want to come off mute. I’d love to hear
[00:36:38] realizing perhaps not as much as I should. That’s totally fair, Ben. Thumbs up from faith. Yeah, we definitely should do more of it. Oh, okay. Your, your thumbs up for Ben. Cool. All right. Well, so yeah, please. I’m working on a new team and. Historically, product management hasn’t been part of the decision making and so I’m really that’s 1 of the reasons I attended this process or this webinar to truly think through a decision process and get consensus on the decision process as we as we start to think about requirements for a new project.
[00:37:20] Well, and even thinking about, like, what is the way that people make decisions today and then highlighting maybe key moments that could be improved could be very valuable here, right? Like, um, and, and that’s why, like, again, I think I would just say every team should be doing, um, whether you’re agile or waterfall or whatever, like, you should just be doing retros all the time.
[00:37:38] And the reason why is because we’re constantly trying to improve. Just a little bit the way we do things and there’s no such thing as a perfect team or process and so we should be constantly moving on that. So yeah, thank you for sharing that. So it’s like putting it all together, right? Um, we go back to this.
[00:37:55] So you should be doing the identification. Is this the thing that will impact people? You should be having the discourse. Making space for that type of discussion. You should have a punctuated decision moment. You should have communication to the appropriate people. And you should do learning. And when I think about this thing, you know, kind of like how design thinking for a while had like the five steps or, uh, there’s a lot of like things here that seem to be very linear, but they could be loops as well.
[00:38:19] So like, you may get to the decision and the decision maker doesn’t like the options, right? Or you may get to communication and people are not being considered. Right? You may get to discourse and people are like, Why are we even talking about this? You should just make the decision. You may just, like, short circuit these things.
[00:38:33] But here’s the one thing I do think you should start to think about. Is that, for identification, you’re trying to minimize the number of times you actually enter this loop. You want to make as many decisions as possible as ones that are easy to make. You want to maximize the viewpoints inside the discourse.
[00:38:49] You don’t want to spam people. Um, I almost included that to minimize spam, but you want to maximize the number of viewpoints. And so that may mean that you curate who’s actually part of this. If you have like. You know, um, a bunch of different engineers. You may not need every engineer in there, right? You may want to specialize in particular expertise or knowledge or something like that, but you don’t want to maximize the number of people you want to maximize the viewpoints want to minimize the number of people involved in any decision making process.
[00:39:16] That is part that is the decision maker. Um, and then for communication, you want to maximize the ability for people to contextualize that may mean multiple different ways that you communicate to different groups. Like you may send one thing to an engineer’s may send one thing to customer support, but you want to allow them to contextualize it as much as possible.
[00:39:35] But you want to minimize the number of people that are actually getting this information. And so that means that the shorter your kind of like TLDR is, and the more impactful it is for people, the better it will be. And then finally, we want to maximize learning about the process and minimize concern about the outcome.
[00:39:50] We, of course, always care about businesses being able to survive, serving our customers, doing the right thing when it comes to all of that. But we should really be thinking that we also just don’t know what’s going to happen in the future. So, you know, how do we know that this type of process is working?
[00:40:08] Right? Not only are we seeing all of these things starting to happen, but I would say one, that less people deciding means that we have less wasted time. So, I think when we start to take on these types of decision making processes, this meta type of process, and we, we kind of minimize and maximize the right things, there’s less wasted time for everybody involved.
[00:40:26] And then I think that the fact that there’s going to be less ex, escalations probably, means that there’s also less learned helplessness. When people have to escalate every single decision to a particular person, that is a bottleneck. That is not a leader. That is a bottleneck. And so, if people have to do that, they will not make decisions of their own.
[00:40:44] I think that’s a bad state for a team to be in. So one thing I wanted to also talk about is like a lot of the work I do ends up being around like AI machine learning and there’s a lot of really interesting things going on right now. Um, I won’t mention particular terminology or technologies, but I do want to talk about, like, what is maybe the future of decision making?
[00:41:01] And so, one of those things, uh, is this paper, um, that came out, I think, about, I want to say, like, three or four months ago at this point, but it was about how, in particular, uh, When we have human and machine teaming, we may actually be doing the worst of both cases, which is that, you know, algorithms are really good at maybe parsing data, giving like, you know, forecasts or predictions on certain things.
[00:41:29] Um, but. You know, humans are not, things that humans are not always good at, and vice versa, humans are good at kind of like intuition, at trying to make trade offs, being lazy, actually being lazy and deciding what to put your effort into is I think a really important human trait. Um, but this paper starts to talk about how like a lot of the cases where it’s human and machine paired together, we may actually be using the worst cases of both of them.
[00:41:53] And so this is related to Paper that I read a couple of years ago at this point, maybe seven years ago, eight years ago, um, by, uh, Phil Dunn Allen, which is about this idea of like animistic devices and agents within an environment. And what it really inspired me about was that, you know, we have lots of different people taking different kind of stances within.
[00:42:14] A decision making process and in this case there’s like a bunch of devices that would sit on your desk and they would have like different reasons for being there and I won’t go in too much depth. I’ve done other other talks about like animistic design for machine learning and agents and things like that.
[00:42:28] But I think there’s something interesting about, like, how do we start to think about, like, Machines as specific use within the realm of decision making. And so, you know, my question is always like, how might we reduce our, you know, the biases of machine learning, or not just reduce the biases, but use them when appropriate, when there’s like, not the bias that we, we don’t want to have, and that’s a double negative, but, and then how might they reduce, reduce our biases when we.
[00:42:52] You know, the group idea still is something that if you have other agents that are inside that group, could they also help reduce bias like other humans do? That’s an open question. Um, and then I think the last one is like really are agents ever decision makers? And when I say agents here, I mean machine learning or other types of like virtual agents.
[00:43:12] And if we’re using this framework again, I think there’s some interesting ways to start to think about like what could be automated inside this process. So one, like Automation of really like entering this process could start to be something where like it starts to predict that this process needs to take place based on something happening in the environment that could be via documents or discussions or meeting transcripts and it automatically kicks something like this off.
[00:43:36] I think automating, collecting of viewpoints, not only from people, but also simulating different viewpoints. And so I wrote a recent article about like, creativity being, and creative thinking is really just about like, adding enough confusion, or just enough confusion to a situation, so like, what would be some agents that would cause like, provocation here?
[00:43:54] Um, and then finally, you know, within the discourse, like, how do we actually do a better job of even like, simulating outcomes from the different options that we’re considering? I will get to the decision making point in a second, um, but I, I’m not sure. I’m not sure where actually the decision itself, should or could be made by a non human entity.
[00:44:13] Um, automation in the communication sense, like, not only who should get this, like, um, this, all these people should get this particular piece of information or this decision, but also, like, contextualization. So because you can go in and you can, like, take a very long piece of text and summarize it, how could you also contextualize it for that particular person and the job that they do?
[00:44:35] Could you use the language that they use? Could you even expand or link to the impacts that may be specific to them rather than more generic ones that would be generalized for like a group of people? And then finally learning, I think this idea of like automation of like tripwires. So not just the idea of like this metric happened or this metric change in a way that we wanted to worry about, but also even just like news stories.
[00:44:56] So like. Where we’re at, where we used to be at, where everybody, every PM would have like Google alerts for their like competitors. What if there were ways that we could start to like pluck this type of information out of the environment on a map? Um, and then theme collection. Like, so as we start to look at all these different like decisions that are being made, what are the themes or topics of these different decisions?
[00:45:14] So I think there’s like a lot of opportunity for automation here, but not for everything. And I think it was related when we talk about the decision thing, um, to this slide, which I think is from like 1969 or 79 or something. It’s, it’s from like over decades ago. Uh, which was part of an IBM manual. Um, and so a computer can never be held accountable.
[00:45:31] Therefore, a computer must never make a management decision. And I tend to agree with this. I, I, I want to be proven wrong. But I think like in decision making processes, we still need humans to do that decision. And so, Not sure. I’m not sure how we actually do that in the future. So that’s maybe an open question.
[00:45:46] So in summary, you know, I want you to be more deliberate in the way that you make decisions. And if you want to learn more about decision making and strategic thinking, that type of thing, I’m part, I’m a co founder of this Uncertainty Project, which is a newsletter plus like tools for being able to do decision making.
[00:46:03] Um, we have a newsletter. I think it’s almost like almost 2000, it’s something like 1700, 1800 people, I think, um, that are subscribed to this. And so we send out things weekly. Um, if you want to learn more about this, I think this is a great place to do that and would recommend that. Um, and finally, just thank you.
[00:46:19] You know, I really appreciate your help here. Um, you know, I’d love to be in touch and, you know, one of the things I’m also offering to people is if they would like to, I would love to be able to help you with like strategic thinking and decision making process. So if that’s something you’re interested in for your organization, I’m happy to do that.
[00:46:37] Um, and so now let’s like kind of move into like a little bit of a. Uh, mastermind, and I would love to hear for the people here that are, um, you know, doing this type of work, like, is there a decision we could talk through right now and see whether, um, we could discuss, like, how might we change this? And, and of course, I’m also open to questions if people have that, but if, if someone feels brave enough, I’d love to talk through a decision making process.
[00:47:00] So we’ll stop there. 10 minutes. Actually, that’s less time than I wanted, but yeah, Pete, go for it. Some. A lot of times we, we go to meetings just to create the next set of OKRs goals and how we want to measure it. But that is almost like a pre mortem, so to speak. But last company I worked, it was never about continuing the last quarters.
[00:47:32] Our quarter before like, and it’s always new. It’s always new. Yeah. Yeah. So how do you influence decision maker, even in a startup company to at least include the decision maker, decision, a strategic decision more on a continuum basis versus just toss out because it didn’t work. Well, this is the problem is that I think we, this may be like a slightly different discussion than like decision making it’s more about like follow up and like, and honestly, like we don’t build.
[00:48:01] Um, I’m doing a talk next week at Product World, which is, which I’m calling, like, Product Management Fiction. Um, which is that we create a lot of documents, but we kind of never reference them again. And so, what we should have, and, and, um, you know, John Culler talks about, like, a latticework of documents. Um, and so I appreciate him if you, he has a great newsletter as well called This Beautiful Mess, I think.
[00:48:20] Um, but he talks a lot about, like, At a very high level, like, if you need to have, like, strategy documents that are constantly referenced, but they shouldn’t be 20 pages long, right? Like, the last place I was at, these documents were humongous, and so no one would ever read through them again. So, like, what is the succinct summary of this?
[00:48:35] Um, there were lots of teams I’ve been part of where everybody like re litigates principles within every document or who’s the key customer set or whatever. That should be one document that we all reference. And so, I think there’s something about like the way we end up doing almost like world building for what is this future that we imagine as product people.
[00:48:53] And how do all these things link together as like a Wikipedia, right? Like, I think that’s the problem. The fact that we’ve gotten into like Google Docs and not necessarily Wikipedia, and SharePoint tries to do this as well. Technically GitHub is owned by Microsoft. I just wanted to like give you that acknowledgement.
[00:49:07] Um, so I think there’s a lot of things we can do about more regular. Creating a better latticework because unless you start to include those things in like weekly meetings, for example, right. And I’ve talked about this as like a proto strategy where you’re not presenting this page that’s like, here’s our strategy document every meeting.
[00:49:25] But if you start every meeting with like, we’re here to solve this problem for these people in this way. And you just say that as one sentence. And that’s the summary for right now, that probably orients a lot more around strategy documentation than not same thing with like OKRs, which. You know, for a company that I used to work with, um, that, uh, is known for OKRs, that does a terrible job at them, even though they’re known for them, um, like, they would try to include them in more of the regular check ins, but this is the problem, is that we’re not doing, like, continuous planning, right, we’re doing fits and starts and huge lift type of planning, and that’s why everything gets thrown away, is that you’re not adjusting things over time, and so that anybody that’s, like, in the OKR industry and trying to, like, be an advocate of that type of thing will say that you should be checking them every week.
[00:50:10] Um, and so that’s what I would say is like, I think making these things part of every document, you need to have the latticework of key documents, which is probably a very small number, but they’re referenced by everything. Everybody knows what they are. They’re even talked about in every meeting, right?
[00:50:24] Like, that’s what you need to get to. And that’s why I think like, Some of Amazon’s process around like weekly business reviews, for example, I think through that operational excellence, right? Like they start to get back to like, are we still doing the right thing right now or not? Um, so those are some things I would, I would say it’s not quite decision making in the way that I’m kind of pulling together this meta framework, but I think that like virtual loop of learning does start to point to like, Hey, we didn’t check in.
[00:50:46] Like we threw away our old OKRs and just started new ones. And it’s like, why, why didn’t we start as an editing this document that was the OKR document rather than doing that? That would be the question. Um, I have one more question then. Oh, sure. So you, you brought a very good point saying you need a different thought, thought process or different people Different opinion.
[00:51:07] Yeah. Versus different people from all different aspect. Yeah. So how do you, within, within an organization, how do you kind of align that? That has, that’s probably the biggest, like an, uh, yeah, yeah. Yeah. So I think it’s that, you know, again, if, if you’re going to take action on the thing and you’re impacted and you have like the expertise, that’s usually like, just figure out who those people are usually and include them.
[00:51:33] But I think there’s like. There’s wild cards that you should be bringing in as well, right? Like, a lot of the time we talk about the cross functional triad as the way to, like, talk about this. So design, uh, engineering, and PM. But there are lots of other people that have opinions too. Like, like, I’m not a legal expert.
[00:51:47] But, and I’ve, I have worked for three companies that have been under antitrust regulation. So, I’ve had to involve lawyers a lot of the time into different things. And so, like, Involving them early is really good because you get their thoughts on like risk trade offs around things. Same thing for, again, privacy, legal, customer support, like all these different groups.
[00:52:06] And so that’s what I’d say is like Don’t just, like, if you have an engineer in there, there should be a really good reason why there’s another engineer in that discourse. Right? And they, again, they should bring a different expertise. But if you just have, like, a bunch of engineers, like, piling on and saying the same thing, that’s actually not helpful.
[00:52:19] Like, if someone says plus one to a comment that is in the same role, that is, that probably means that they don’t need to be there. Maybe that’s not very fair, but, like, if you’re just, like, plus one ing everything and there’s already someone there that’s writing the comments, you probably just, you can just bow out.
[00:52:32] You don’t even have, you can just say, like, Hey, this person’s got it covered. I don’t need to pile on like that. That’s fine. Right. Like, that’s what I would argue. So, yeah. What else? Questions or thoughts people have about this. Yeah. And Chris,
[00:52:45] we’ve got a, we’ve got a good one in the chat.
[00:52:47] Uh, does it need to be regulated right
[00:52:49] Ben Senior Product Manager at McDonald’s: away or do we need to leave it for a later time?
[00:52:52] Chris Butler: Encouraging innovation? Let’s talk through this. Okay. Well, I’m not speaking on behalf of Microsoft for GitHub when I say this, but I would say, like, you’re regulating what? You’re regulating like HTML or, uh, you know, like data science. Is that what we’re regulating now? I think that the, the fact that the, uh, the, like the document that came out of the Biden executive order for AI talked about like number of flops that it took to train or something like that.
[00:53:18] It was just ridiculous to me because all you’re doing is you’re creating like a ceiling. This is where we were at when it came to like. Um, like size of an encryption key in the 90s, essentially not being allowed for export. People are going to build systems that will be underneath that thing to like avoid, one, that type of regulation.
[00:53:37] And then two, so in some ways, I guess what I’m asking is like, What we should we already have systems in place that regulate things. We should just make sure they regulate it well And so that’s like food and drug, right? That’s like the SEC. That’s the FTC That’s all these different groups that are there to regulate the outcomes of these things So like robocalling like the fact that they had to create a special law about like synthetic voice for robocalling is bad, right?
[00:53:59] They should have actually just said robocalling is bad And if you do this type of thing, whether it’s a synthetic voice or not, like, that’s bad. So that’s what I would argue, is that we already have things, it’s just that those regulations don’t always work well. Regulating technology is more about the means rather than the outcome, which is, I think is, that’s the way I would think about it at least.
[00:54:16] So yeah, I will digress and get off my soapbox for now. What else?
[00:54:28] Yeah, I do have to leave in just a few minutes. Um, but yeah, I mean, thank you everybody. I really appreciate the time here. I will make sure that everybody gets slides. I realize that I talk about a lot of different things here. So we’ll make sure we get that out in the next like couple days. And then, yeah, but thank you so much everybody for being here and listening to me.
[00:54:49] And after this, I would love to hear if you try this out and if it works or if it doesn’t. If it doesn’t, I definitely want to hear about it. And then too, like I said, like I’m trying to help more organizations. Um, from the standpoint of like consulting and advising. So if there is something that you are interested in getting help on, either from a strategic point of view or decision making point of view, please let me know, and I’m happy to talk to you and your team.
[00:55:14] So Chris, uh, just
[00:55:16] Ben Senior Product Manager at McDonald’s: thanks again, really appreciate having you come back, return to us in a virtual format this time and, and sharing, sharing all your
[00:55:24] Chris Butler: insights. Really appreciate it. Absolutely no, you know, I love I love the food in Chicago. So that’s that’s the reason why I’m here in the virtual.
[00:55:33] Ben Senior Product Manager at McDonald’s: If we can get you get you to come back physically, we need to do that very soon.
[00:55:36] Yeah, but thank you. Everybody really appreciate you joining us today for our inaugural. webinar for Product Camp Chicago. Glad you all could participate. Great questions. Great discussion. Chris, always a pleasure. And we hope you all have a wonderful rest of your day. Stay tuned for recap with Lake to Recording, the plethora of resources that Chris, uh, shared with us.
[00:56:01] And like Chris said, let us know if you try it. We’d love to hear your feedback.
[00:56:05] Chris Butler: Thank you, Ben. Can I Do a plugin in here. Uh, Dave, go ahead. Um, first Friday of every month. Almost first Friday. We do what we call a PDMA, a coffee hour breakfast club. I think you’re part of that. I know Dave is part of that.
[00:56:24] And we’d love to have you all join in there. We discussed a different topic. Pre decide the topic. You can find us on, um, meetup. Thank you.
[00:56:35] Ben Senior Product Manager at McDonald’s: It’s awesome. And Pete, I’ll ping you offline so we can include a link to that in our recap. Thank you. Awesome. Thank you. All right. Well, thanks so much, everybody. Hope you have a great rest of your day.
[00:56:46] Chris Butler: Take care.
