THE YES, a shopping app for fashion brands, uses a sophisticated algorithm to create and deliver a personalized store for every shopper, based on her style preferences, size, and budget. After launching the app in 2020, the founders must decide whether to continue developing the algorithm to deliver on the company’s customer value proposition or to focus their resources on new customer acquisition, with the idea that more users on the app would improve the algorithm’s performance.
Harvard Business School senior lecturer Jill Avery and THE YES co-founder and CEO Julie Bornstein to discuss this make-or-break dilemma in the case, THE YES: Reimagining the Future of e-Commerce with Artificial Intelligence (AI).
This episode was recorded live at Harvard Business School on March 30, 2022.
BRIAN KENNY: On June 18, 1956, ten mathematicians gathered at Dartmouth College on the top floor of Kemeny Hall for a six-week workshop on how to make machines improve themselves and solve the kinds of problems now reserved for humans. It was called the “Design Workshop on Artificial Intelligence,” giving a name to a field that, depending on who you ask, could either save the world or destroy it. The truth is probably somewhere in the middle of that spectrum. Although, much about AI is still the stuff of science fiction, many practical applications have made their way into our daily lives. Brands like Netflix, Tesla, and Pandora are pushing the boundaries on AI and finding new ways to delight and engage customers. But for many sectors, the field is still wide open and waiting on a game changer. Today on Cold Call, we’ve invited professor Jill Avery and case protagonist, Julie Bornstein, to discuss the case entitled THE YES: Reimagining the Future of E-Commerce with Artificial Intelligence. I’m your host, Brian Kenny, and you’re listening to Cold Call on the HBR Presents Network. Jill Avery is an expert on brand management and customer relationship management. Before coming to HBS, she managed several world class consumer brands, including Gillette, Braun, Sam Adams, and AT&T. Julie Bornstein has spent her career at the intersection of retail and technology. She is the co-founder and CEO of THE YES, and the protagonist in the case we’re going to discuss today. She’s beaming in from San Francisco. It’s great to have you both here today. This is a special episode of Cold Call. We are actually taping this in front of a live audience at Harvard Business School, make some noise so they can hear you. That’s the audience that’s in the classroom. This is one of the classrooms at Harvard Business School, where we teach cases every day to our MBA students and Executive Education participants. So, we’re really happy to be able to do this live taping as part of a celebration of the 100th anniversary of the Case Method at Harvard Business School. So, this is a big day at the school, and we’re happy to have several hundred people streaming this as well on YouTube. So, let’s just dig in. Why don’t we start with you, Jill. Because it’s the 100th anniversary of the case method, I thought I would just pull the lens back a little bit and ask you the first question about what makes an enduring case? Why is this case one that will endure over time?
JILL AVERY: Sure. I think to me, the biggest thing an enduring case needs to be is a compelling story. What we ask of our students in the case method is to allow narrative transportation to take them away, to put them into the shoes of a case protagonist and to experience the managerial decisions that that protagonist is trying to make. So, if I think about what makes a compelling story, it has four components. It’s a empathetic and relatable character that students can see themselves in. Julie’s character came from a recommendation from one of my MBA students who had the privilege of working with Julie as an intern, who came back from her internship so excited about what she was learning at THE YES, that I knew that this might be a great setting for a new case. Biggest contribution to a compelling story is a central conflict that introduces some tension upon which students can debate and disagree. That’s what makes a good classroom discussion. And what I’m looking for there is a conflict upon which smart people will disagree, a gray area of managerial decision making where there’s no right or wrong answers. And then I want to situate that conflict into a really compelling setting that has all the complexity of the real world so that students can grapple with that decision in a very real way. And then finally and most importantly, is a moral or a message, something that the students will take away, a learning objective, the point of the case, what are we trying to teach and what are we trying to learn with it?
BRIAN KENNY: Wow. So, every case starts with a cold call. The name of the podcast is Cold Call. So, let me ask you what the central theme is of this case and what’s your cold call is to start the discussion?
JILL AVERY: Sure, sure. Maybe we’ll try it out on our studio audience. What do we think? So, the central tension in THE YES case revolves around whether the company has achieved and demonstrated product market fit. They are four months post launch, and they’ve amalgamated an incredible input of data; customer data on economics of customers, what they’ve bought, how often they’ve purchased, demographic and psychographic data on which types of customers are coming into the business, satisfaction data, usage data, all of this data coming in, and JULIE BORNSTEIN and her team are trying to analyze that data to figure out if the company is ready to continue investment, and in fact, increase investment in customer acquisition. And this is a make or break moment for Julie, because if she invests in customer acquisition too early when product market fit is not demonstrated, then she risks burning through cash inefficiently. If she waits too long, then maybe competitors get a jump on her and maybe she doesn’t feed the algorithm that drives this platform with enough data to be able to optimize it. So, my opening question for students is, how has THE YES achieved product market fit? This is a big global wide open question, which really requires them to look at all the data presented in the case and amalgamate across to try to figure out where is this company in its trajectory of scaling, and is it ready for future investment?
BRIAN KENNY: Julie, let me turn to you for a minute. Maybe you could start just by describing for our listeners what THE YES is, what business you’re in and how it came to be, how was it born?
JULIE BORNSTEIN: THE YES is a shopping platform that is AI powered, and the concept is really… There have been many, many websites and apps built since the late nineties, Amazon of course, being the first big one that you can buy lots of product in. And over the last twenty years, the amount of places to go and buy product has grown dramatically. And the problem today is really overwhelmed there. Any consumer has to go to a million sites, brands and retailers for whatever category they’re shopping in to figure out what they want if it’s not something that’s carried on Amazon, and even Amazon has made it harder because they have ads kind of boosted as opposed to the right products boosted. So, really what we’re trying to do is take kind of the capability of AI, which is to understand a user and behavior and be able to adapt the experience to the consumer similar to, as you said, the way a Pandora or Spotify might do with music and apply that to commerce, and we started in the fashion category. It’s a really complex category to shop, and it’s also a really complex category to understand and be able to make good recommendations on. So, ultimately what we’ve built is sort of a next generation shopping platform where we have a very large brand assortment. Our goal is to have the largest of branded fashion; we’re in women’s today and we’ll be in men’s and other verticals in the future. And for the consumer to be able to come in and answer some quick and fairly straightforward, but high signal questions that help us understand her and then the entire platform and shopping experience adapts to each user. So, every customer is sort of shopping a store built around them, and so that is the basic premise of the business. The reason that I started it was I’ve been in this space for a long time. For about 20 years I helped Nordstrom build their e-commerce site in the late nineties, early 2000s. There were a lot of things I wished we could have done that technology wasn’t yet able to do, but I had in my mind, all of these things that would be so much more both efficient from a business model standpoint, as well as better for the customer. And so, as technology has evolved over the last twenty years, we decided the time is right, let’s take advantage and rebuild the whole infrastructure so that it can be built on this much more intelligent sort of system.
BRIAN KENNY: So, Jill, you’ve managed a lot of great brands, I teased a little bit of that in the introduction. I think as consumers, we’re all used to filling out surveys and giving information about ourselves to retailers that we’re making purchases from and we assume all that information is being fed in somewhere to make our experience better the next time we go back to that place to make another purchase. Can you talk a little bit about some of the trends that you’ve seen in retail as it relates to AI?
JILL AVERY: So, we’re seeing AI applications all over the field of marketing, from using predictive analytics and data driven decision making for new product development input, for product innovation, to media planning and media buying for using algorithms to drive, placing an ad at a particular moment in real time. We’re seeing it on pricing where algorithms are driving changes in pricing within seconds for retailers to be able to maximize price customization opportunities. We’re seeing it certainly in customer service where chat bots are developing and nurturing and managing relationships with customers. So, this technology has put its tentacles into lots of areas within marketing, and certainly the retail application that we see in this case to me is one of the more powerful, because that’s where personalization and customization really delivers something of true value to customers that we haven’t been able to deliver before without the technological innovation that’s happened.
BRIAN KENNY: Generally speaking though, do you feel like it’s lived up to its promise? Are we serving people the information they need right when they need it, or maybe in advance of when they know they need it?
JILL AVERY: I would say it depends. The technology has come an amazing distance even within the last five years. So, when someone asks me that question, I always hesitate to answer because five years ago… Just taking chatbots as an example, five years ago, most chatbot interactions with customers failed. Today, much higher success rates with chatbots and customer satisfaction. So, every day the technology is getting better at taking in information, analyzing that information, and putting it together in a way that creates realistic interactions that are personalized for customers, so it’s getting better and better. Do machines make mistakes in interactions with customers? Yes, unfortunately they still do, but I would say so do humans. So, to me, the best solutions here are very much like THE YES, companies that combine machines and humans in marketing decision making to bring the best of both sides to that to deliver the most value for customers.
BRIAN KENNY: And the more information the better able the service or the machine is to provide the right experience. We’re going to get into that a little bit, because there’s a tension that presents particularly in the case of THE YES. But let me turn back to you for a second, Julie. Can you tell us a little bit about who your competitors are and how you feel THE YES is different from what the others in your landscape are doing?
JULIE BORNSTEIN: Well, this is a case where a startup is coming into a very crowded space and there are a lot of competitors, so we are not inventing a category but we are trying to do it very differently. You know, I would say our competitors today are anywhere that you can go and buy branded fashion, so from Nordstrom and Net-a-Porter and ShopUp. I also think a little bit about even like the companies like Google and Instagram, where people are discovering new product. I think the basic way that we’re different is that we, for the first time, have built a shopping experience that actually understands you as a consumer and you’re at the center. So, if you go to any of these other website experiences, it’s one-size-fits-all and the experiences, whatever there is, fashionable is fashionable, whatever product is featured on the homepage, that’s what everyone gets. The whole concept for us is we’ve built this store that’s customized to you. And it’s funny, when I look at my daughter who’s 19 and her home feed for THE YES and I look at mine, I’m really convinced it’s working because I would be offended if any of those products showed up on a homepage for me and vice versa, frankly. So, our difference is we’re really sort of helping sift through this very overwhelming assortment, finding the things that are relevant for you, and then doing a lot of things like notifying you when new product from a brand you like or might like comes in, when a product that you’ve already said yes to goes on sale. So, it’s a very sort of one-to-one experience and feels more in some ways like some of the media and social media businesses that are geared towards you rather than kind of a generic store that you go and you have to find what it is that you’re looking for.
BRIAN KENNY: How does THE YES’s AI work? Can you describe a little bit about what the experience is like?
JULIE BORNSTEIN: Yes, so one of my sort of thesis in the industry having been in e-commerce my whole career is that it’s very hard to have a good recommendation system that you’ve built using AI across multiple categories all at once. I really believe you need to understand the specific category. You need to understand what the buying decisions that matter in that category, what the dimensions of that product base is that help someone to find the product they’re looking for. So, if you take the category of fashion and then you compare it to beauty as just another category, the things that go into buying a mascara or a foundation have nothing to do with what goes into buying a dress for an event. And so, you need to really understand the category well, we have built a taxonomy that is far deeper than anything else that exists in the market to understand every dimension of a product. So, as you could imagine, it’s everything like the silhouette and the shape and the “Is it short sleeve, long sleeve? V neck, deep V neck?” You know, there’s differences; crocked, regular? Then understanding, is it good? So, those are objective factors. We also have subjective factors, like it’s good for work, or it’s good for going out. So, we need to understand everything about the product so that as we know what the customer is looking for, we understand what to serve up. Then what we did was we tested sort of a whole series of questions for the consumer to understand where we got the quickest, highest understanding of the customer, and then we asked those questions so we can basically set up a system that does matching. The one thing that is different about also the way we’re working is it’s not just an upfront series of questions where we learn about you with sort of really important information like what styles do you like and colors do you like? What do you never want to see and wear? Which is important for those of us who are no longer wearing crop tops. And then also we ask things around size, so we understand what’s available in your size and if a brand runs big or runs small, we’re able to recommend the right size for you. So, we get that information, but then we also continue to collect information on you as you shop. You can yes and no, kind of like thumbs up/thumbs down product as you’re shopping. The nos go out of your way and we sometimes will send you a confirmation to say, “Do you never want to see string bikinis?” Which is one that I got the other day. So, it’s confirming. So, we are gathering data, but also confirming the user experience. And then we save THE YESes, we organize them and then we notify you when things happen with them. So, we’re learning as you shop as well as kind of in this upfront series of questions. And all of that information: understanding the category and the product set really well and understanding the user, allows us to basically match score every product in the system. One thing I would say around kind of the use of machine learning is one of the things we had to do was train models to understand fashion. So, what you would do is you put in… We use both computer vision and natural language processing. We take all the text associated with a product and the image, and we basically put it through a system that we’ve already trained to understand, oh, that’s a black ruffled v-neck blouse made of silk. So, it’s a combination of visual and text that helps us really understand the product. So, artificial intelligence is basically allowing us to match characteristics of a product to all the other characteristics of other products and even sort of understand how close or far they are together. So, you may take an off-the-shoulder shirt and a cold shoulder shirt… Sorry for the men who don’t understand what I’m talking about. You know, you understand that they’re more similar than a turtleneck, for example. So, it’s sort of building these models that understand the product and then building these algorithms that understand the match score to the consumer.
BRIAN KENNY: So, it’s incredibly complex. There are so many variables, and basically nothing of what you’ve described to me fits into my classic definition of a marketer and I’m curious on about both of your opinions on this. How do you build a team that’s able to compete on this kind of playing field, because it’s a very different set of skills than I would think about hiring from my marketing team, let’s say. Jill, why don’t you start?
JILL AVERY: So, I think what’s so exciting about this is everything that Julie is describing is what marketers through history have strived to be, right? Close to the customer, complete understanding of the customer and the product and the interaction between the customer and the product to create value. So, what the technology innovation here enables us to do is to be able to get that knowledge deeper and dynamically in real time. As customers change, as culture changes, we’re getting that ability to understand the customer deeply at a particular moment in time, at an individual customer level, and that just takes that customer understanding and how we’ve thought about market research for hundreds of years and brought it into a really exciting domain. So, I think fundamentally the skill sets of a marketer are exactly the same, but now every marketer needs to understand how these digital tools and technologies enhance what we want to do around our customers and that’s understand them, understand what their needs are and be able to deliver against that.
BRIAN KENNY: Julie, what does your management team look like? How did you construct that?
JULIE BORNSTEIN: It’s interesting because my former roles were in more traditional multichannel retailers, so I was at Sephora and Nordstrom and Urban Outfitters and even at Stitch Fix, we had merchants, people who were picking product to sell through the store. We had planners who were deciding how much to buy of the product. We had photographers who were reshooting. They were taking photographs of all of the product and people who were managing that system to get the samples in. None of those jobs exist in my new company. My co-founder is an engineer who’s done a lot around deep learning and machine learning engineering. And what we needed was we needed a technical team who could build systems, so about half our company are engineers and we built this recommendation system. We built sort of a front end user interface to allow us to continue to evolve what the user experience is like and gather information most effectively, and then we have a team that basically has built this system that integrates with all the brands automatically. So, if you think about it, we actually work with hundreds of brands, we take their entire catalogs. We are actually not deciding what to buy and we’re not owning the inventory, so we have no planners who need to actually buy the product, instead we take the entire catalog and then the algorithm dynamically merchandises the site for every customer. So, it’s an entirely different model. The person that is running sort of what’s closest to merchandising is an amazing woman named Lisa Green, who came from Google and she worked at Google for twelve years, working with brands to think about how to move their marketing budgets from Vogue and W Magazine and New York Times to Google and YouTube. So, she’s been helping in this sort of marketing transformation of brands and how to understand and think about customer acquisition in this new digital world. And she really comes more from the business development sales side than from the merchant side, she knows the category really well and has a lot of relationships. So, our team is quite different, we’re sort of a combination of… I would say everybody is technically savvy, really interested in sort of where the future is going, using technology both for customer acquisition, which we haven’t really talked about yet, as well as sort of customer experience. Then we have a very robust product management team. This is where we’ve had some HBS students come in actually. And they’re really thinking about what is that experience that we need to create? What does the consumer care about? And the product manager role is almost like the new GM in a tech environment and then they’re working with the engineers to build out the systems.
BRIAN KENNY: But everybody knows what a crop top is, that’s like a theme.
JULIE BORNSTEIN: That’s true. The male engineers did not originally, but they do now.
BRIAN KENNY: So, I’m wondering, do you have to convince consumers… Do you have to sort of reeducate them or train them to behave differently on the platform? What does that look like? And is it a big hurdle to overcome or is it pretty straightforward for people when they first experience THE YES?
JULIE BORNSTEIN: It’s been super fascinating. I would say some people get it right away and others just don’t. I would say the younger generation definitely gets it faster and loves the product because it feels so fun and interactive. I would say the older generation, let’s call it sort of 40 and older, even though-
BRIAN KENNY: Thank you. That was kind.
JULIE BORNSTEIN: … many of those millennials. It’s hit or miss. Some of them understand what exists today and that makes sense to them and some of them come on and immediately get it and love it. Part of what helps with the app actually is that we onboard you. So, we explain what’s happening as we go, we ask you questions so you’re starting to get a feel of how this is different from just a traditional e-commerce website. I would say once someone actually buys sort of for the first time and they see how the whole system goes, you check out right on the platform, we communicate with you where your product is and when it’s arriving to you, then I think that we find that customers are really sticky and so it’s more about people who come and aren’t quite sure what’s happening. We continue to think about how to educate them and explain so that it is… or show by doing. But it’s definitely a change of behavior.
BRIAN KENNY: Yeah. Jill, let me ask you this. There is a creepiness factor associated with artificial intelligence that I feel like surfaces anytime we talk about firms that are collecting information in this way. I’m wondering just generally speaking for retailers, is it getting easier or harder in this day and age where we’re worried about cyber terrorism and other things, identity theft and things that can come along? How do you think about that as a marketer in this space?
JILL AVERY: Yeah, so it’s definitely a concern. Consumers say in survey after survey that privacy is important to them. However, we’ve seen consumers give up lots of private information to companies of all stripes. I think the better companies in this area are doing two things, one is opt in/opt out so that consumers have visibility about what they’re sharing and whether they want to be sharing that information. Then secondly, if you’re collecting data from your customers, how are you using it in a way that’s valuable to them? So, if the data collection feels entirely selfish that it’s only benefiting the company, then I think we’re going to see customer backlash against that data collection. If the data collection on the other hand is creating a customized and personalized shopping experience like it is for THE YES, then we see consumers very willingly swiping right, swiping left, doing yes, no. The data collection actually becomes an enjoyable experience and it’s almost the gamification of sharing who you are as a customer with a company to have kind of a shared value creation process. Customers by sharing their data are almost co-creating the product and the shopping experience that’s being presented to them. So customers see great value in that, then they’re willing to share. If they’re not seeing that value, you’re going to see customers holding back or avoiding revealing their data.
BRIAN KENNY: Yeah, that makes perfect sense. There’s an interesting tension that arises in the case and this was a sort of a discussion that I think you and your partner had, Julie, about when’s the right time to launch and there’s a discussion about he wanted to launch sooner than perhaps you did. There was this sort of a conundrum about, we need really good information about our customers so that we can get the AI to work and we need customers to provide that information. And your point of view was, “No, we need to provide a really good experience or we’re not going to get those customers back.” So, I’m wondering how that played out.
JULIE BORNSTEIN: Yeah, it’s interesting. I was really struggling with launching because I wanted the product to be so good the first time you used it. One of my investors is True Ventures and they do this annual offsite with all their founders and someone said, “If you wait until you think it’s perfect, you definitely waited too long.” That was really helpful to hear because as the creator of the product, you feel like there’s so much more I want to do. And I still feel that way today actually and so it was helpful to sort of get that reminder. We actually were planning to launch in March of 2020, and then COVID hit. So, what that ended up doing was giving us… We ended up launching in May of 2020. It gave us two more months to get just to the point where I felt comfortable. I felt like it was still not quite ready in March, but in May it was, because we took those two months to do sort of some final cleanup. And by the time we launched, I felt like the recommendations were good, the experience was good, I was proud of it and I felt people would come back, even if it wasn’t perfect. But yes, that is a big tension and then that tension sort of continues in the case because the question is when do we start investing in paid customer acquisition versus getting sort of continuing to make sure the product was working well enough that it would pay back if we spent money to acquire a customer?
BRIAN KENNY: Yeah. Let me just ask you, you’re working with a lot of brands, you sort of explained how that works before. I’m wondering what the value proposition is to the brands to work through THE YES rather than just going direct to consumer.
JULIE BORNSTEIN: Well, first of all, all the brands we work with do have direct to consumer. They need to have a website in order to work with us, so that’s a requirement. We use their photography. Basically, the way our order system works is it goes back into their system and places the order through their website. So it actually looks like volume for them on their website, but really the reality is that the cost of customer acquisition just continues to rise and for a single brand it’s quite expensive. And so really they’re looking for partners who can help acquire additional customers and drive additional volume in a way that feels brand appropriate and comfortable for them. And when you take a traditional wholesale brand that sells through other retailers, it takes an entire team to manage that account, they sort of pay… They lose 50% of their margin, they ship the product to the warehouse of the retailer. It’s a big system that involves a lot. For us we’ve made it really easy, the brand doesn’t have to do anything. We actually are able to just automatically ingest all of their information. We auto-crop and normalize all the data through technology. We assign all the attributes to the product and sell the product. We share all the data with the brands, we give them insights at the end of the month because we have all this fascinating aggregated data. And so our goal actually is to help the brands thrive as well. We are completely dependent on the brands, they are our business, and we see ourselves as really an additive partner to them. In addition to all the data that we get, we also allow customers to follow a brand directly right through our app. The reality is I think unlike traditional wholesale retail relationships and other distribution channels, we really see ourselves more as a matchmaker than we do sort of owning the customer data and hoarding it. So we really see it as a win-win for us and the brands when we work together.
BRIAN KENNY: Yeah. We’ve done a lot of Cold Call episodes on platform companies and this sounds very familiar to me with other sectors that we’ve talked about the same sort of setup. So one more question for each of you before we wrap this up and I’ll start with you, Julie. It’s relatively early days in this journey for THE YES, things are still taking shape and I’m wondering what would you say is one of the most important lessons you’ve learned as you look back over the last couple of years?
JULIE BORNSTEIN: So, many lessons. I would say probably just to be practical for the purpose of this case, the thing that I learned that was most interesting is if you are building an e-commerce business, you need to have a website. So the world was moving towards mobile and we built an app first. Apps are closed systems, and they’re very hard to acquire new customers through. The reason we built an app is we felt like it was the sort of strictest confines to basically create a great experience. We wanted it to be mobile first because that’s where the world is going. And it’s true that the majority of e-commerce sales are now happening on mobile, but if you don’t have a website, you are missing out on all of the sort of out-linking opportunities that really allows you to drive consumers and drive awareness across the board. So we ended up building a website about six months after we launched our app, and that really gave us the chance to start to scale the business and grow so that’s probably, in a very practical sense, the lesson that I learned the most.
BRIAN KENNY: Great. And Jill, let me give you the last word by asking you if there’s one thing you want our listeners to remember about this case, what would it be?
JILL AVERY: You know, I think when most people think about artificial intelligence, machine learning and some of this technological advancement, they tend to think of the efficiency gains that companies can get from cost reduction, “Let’s take the humans out of the process and that will make us more efficient.” I think the real promise of these types of technologies is actually ineffectiveness. I think that we can form better and deeper consumer brand relationships, that we can enhance customer satisfaction, that we can create more value for customers through the use of these technologies, because it comes down to seeing you, understanding you, seeing you as an individual and building products and services that absolutely deliver against your needs at that moment. So efficiency gains are great, but effectiveness gains, I think, are the thing that’s really going to revolutionize the future of retail through these types of technological improvements.
BRIAN KENNY: Jill Avery, Julie Bornstein, thanks so much for joining me on Cold Call today to discuss THE YES, and thank you to our studio audience. We are excited to be celebrating the 100-year anniversary of the case method at Harvard Business School. If you want more on the history of the case method, visit our website: www.hbs.edu/casemethod100. Cold Call is a great way to get a taste of the case method, after all each episode features a business case and its faculty author. You might also like our other podcasts: After Hours, Climate Rising, Skydeck, and Managing the Future of Work. Find them on Apple Podcasts or wherever you listen. If you enjoy Cold Call or if you have any suggestions, we want to hear from you. Write a review on Apple Podcasts or wherever you listen or email us at [email protected]. Thanks again for joining us. I’m your host, Brian Kenny, and you’ve been listening to Cold Call, an official podcast of Harvard Business School, brought to you by the HBR Presents network.