The Business of AI: Insights from Industry Leaders

As we navigate the rapidly evolving landscape of Artificial Intelligence (AI), it's essential to understand the challenges and opportunities that come with integrating AI into products and businesses. Recently, OpenAI brought together three esteemed industry leaders – Kathy Baxter, principal architect of ethical AI practice at Salesforce; Oji Udezue, chief product officer at Typeform; and Miqdad Jaffer, director of product at Shopify – to share their perspectives on the business of AI.

According to Kathy Baxter, one of the most challenging aspects of building a product with AI is developing trust and ethics. As technology advances rapidly, it's easy to learn that some safety or human alignment elements may not be as efficient or effective as initially thought. To address this challenge, she emphasizes the importance of constantly evaluating research and ensuring that trust remains at the core of what you're developing.

Kathy also highlights the need for companies to stay true to their mission and vision when it comes to AI integration. "We want to make sure that AI isn't just some random layer over what we want to do, but that it's really prescient about the workflows of our customers," she explains. This approach is exemplified by Typeform's goal to make the web more conversational and human using AI.

Oji Udezue, chief product officer at Typeform, agrees that staying true to one's mission is crucial when building an AI-powered product. He notes that Typeform's vision is to enable humans to do better work or give them superpowers in very human ways. The company's experience with Formless, a standalone product launched earlier this year, demonstrates the importance of disrupting traditional approaches to form design.

For Oji, the hardest part of building a product with AI has been figuring out what that final mile looks like. Unlike traditional software development processes, where there is a linear flow and expectations around when things will happen, AI products are non-deterministic. To address this challenge, he emphasizes the need for companies to align their efforts across the organization on the right use cases and approach.

Miqdad Jaffer, director of product at Shopify, also notes that building an AI-powered product requires careful consideration of trade-offs. When launching shop.ai marketplace, which allows buyers to search for products using a purple button, Miqdad and his team weighed the pros and cons of disrupting their own product. The result was a brand-new product optimized for speed and learning.

The conversation also touches on the importance of collaboration and communication within teams when it comes to AI integration. Kathy emphasizes the need for constant evaluation and ensuring that trust remains at the core of what you're developing. Oji notes that his team is always experimenting with new approaches, and Miqdad highlights the value of having a clear vision and approach when building an AI-powered product.

As the business landscape continues to evolve, it's essential for companies to stay informed about the latest developments in AI and its potential applications. By listening to industry leaders like Kathy, Oji, and Miqdad, we can gain valuable insights into the challenges and opportunities that come with integrating AI into products and businesses.

"WEBVTTKind: captionsLanguage: enAll right. Hello everyone.My name is Aliisa Rosenthal.I'm the head of sales here at OpenAI.Which meansI have the privilege of workingwith our customers and partnersevery day all day to help themfigure out how to integrateAI into their productsfor their end usersand within their own organizations.I have a few esteemed customersjoining me up on stage today.First up, we have Kathy Baxter,principal architect of ethical AIpractice at Salesforce,where she develops research-informedbest practices to educateSalesforce employees, customers,and the industryon the development of responsible AI.Salesforce is, of course,a close partner to OpenAI,and we've been collaboratingtogether on a handful of initiatives.Welcome, Kathy.Next up we have Oji Udezue,the chief product officer at Typeform.Oji has a stellar backgroundin product managementwith chapters leading product at Twitter,Calendly, and Atlassian among others.Typeform launchedFormless earlier this year,totally rethinking the genericform experience with AI at its core.Welcome Oji.Finally, we have Miqdad Jaffer,director of product at Shopify.Miqdad is responsiblefor the group's integratingAI across the Shopify productand platform, including Sidekick.Welcome, Miqdad.We're so gratefulthe three of you could join us today.Today's conversationis going to be about the business of AI.That is all things about integratingAI outside of the actual coding.On the sales side,we get to work with our customers all day,and we hear several different themesthat tend to come up over and over againaround customer management,experience pricing, and go to market,that you really needto consider when buildinga durable revenue-producingproduct or business.I'm excited to dig in.Let's start off with a one-minutepitch from each of you.We will start maybe with Miqdadand work our way over to Kathy.First question is,what is the hardest partof building a product with AI?I think the hardest partfor us has been tryingto figure out what that finalmile looks like.I think it's very easy to getto 70% product.I also think that the way that you developAI products is very differentthan the traditional softwaredevelopment process.In the traditional software developmentprocess, there is a linear flow,there is expectations around when thingswill happen and how they will getto a good state.With AI products, it's non-deterministic.I think you start outwith this is the goalthat you have for the product.In our case, it was how do we useAI to accelerate entrepreneurship,and how do we then integrate itinto the various parts of the productand make sure that there is alignmentacross the org on these are the right usecases to worry about,this is the right way to approach it,and this is what good will look like.As you work towards that,there's going to be a partin which the productmight not meet that final mile.How do you account for that as you build,and how do you make surethat your users still managesand maintains control as you build?The hardest thing.Look, there's a lot of workit takes to build an actual product,a lot of the technical stuff,the stitching,the prompting, and so on and so forth.What may be the hardest thing for usis the idea that we want to stay trueto our mission,our vision for the customer,that we want to make surethat AI isn't justsome random layer over what we want to do,but that it's really prescient aboutthe workflows of the customer.For us at Typeform, our goal is to makethe web more conversational, more human.We feel like we needto really use AI to enable humansto do better work or to give themsuperpowers in very human ways.I've used human many times,so I feel bad about that ,but we want it to wraparound our workload, want it to matter.We want people to come back to it 80%of the time because they love it.We have all these testimonialsas we've done fFrmless,and also we brought AI into Typeformabout how people feellike it feels natural to them.That's the biggest accoladebecause we sweated that a lot.Kathy.I think in my opinion,one of the most challenging thingsis developing at the speed of trust.As this technologyis advancing so rapidly,a lot of the thingsthat we have identified to bakein trust and ethics and responsibleAI from the beginning,we learn over time that maybethey're not as efficient,they're not as effective.There was a recent study that foundthat with fine-tuning models,sometimes it can unintentionallyundo some of those safetyor human alignmentelements that you are adding in.Constantly doing evaluations,staying abreastof all of the research that's being done,and ensuringthat as you are developing these tools,you are always keeping trustat the core of what you are developing.Moving at the speed of trust. I like that.Oji question for you.When you launched Formless,that was a pretty big move.You launched it as a standalone productversus integratingit into your core function.Can you talk a little bit abouthow you weighedthe trade-offs of that decision?Just going back, so our goal - Hasanyone here used Typeform before?Typeform makes beautiful web formsthat you can usefor zero-party data collection,lead generation, and all of that stuff.Part of the inspiration,by the way the co-founderis right there who created Typeform.David, how are you doing?The core of that is conversational.There are all kinds of forms on the web.Even the Google thing is a form.The chat thing on ChatGPTis really part of a form,but we wanted to make it feel human.Now, when GPT came out,David has been working with GPT since 2.0,but when GPT 3 came out,something clicked for us.We just realized that we could transformthis entire vision ultimately.Now, we had two choices.We could build it into Typeform,150,000 customers, a lot of revenue,but it was going to be slow.We're going to be doing retrofits.What we decidedto do was instead disrupt ourselves,create a whole brandnew product that was optimizedfor two things, speed and learning.That's what animated us.Look, we're going to have to makedecisions later down the lineabout how those things come together.Actually, we are buildingAI into the original product,but we are trying to build a racecar that allows us to go fast,allows us to learn as fast as possible,allows us to experiment unfettered,allows us to imagine--I can't tell you how muchcode we've destroyedbecause it didn't work on Formless,but that's why we did it that way.Wow. It takes a lot of courageto disrupt your own product.I'm curious Miqdad, didyou all weigh any of the same trade-offsas you thought about launchingsome of your AI initiatives?Yes. For those that aren't aware actually,maybe you are,has anyone bought anything usinga purple button on Shopifystore somewhere, show of hands.That little purple button is shopand we actually started there.We started with the shop.ai marketplace.This is where the buyercan go on and searchfor any given productand do it by an event.We said, \"Okay,let's put semantic search in here,and let's try and go from the perspective,as Oji said,the speed to learn as fastas we possibly could.\"We put it in front of buyersand tried to figure outwhat they would askand how they would ask it.In some cases,it was help me plan a dinner party.Or it's I'm trying to build an outfit,what's the right things to do?Using that to build an initial search.When we started gettinga little bit of progress there,we said this is something that we wantto be able to put everywhere.We have a tenant within our annualplanning or even longer than that,we have this notionthat we will bring technologyto our merchants as earlyas possible in the cycle.It already fitwithin the ethos of Shopify.Now the next stepof it was how do we get this everywhere?What are the right places to put this?Where will this utility be realized?For us to deal withsome of the safety concerns,we just had a four-word solution,human in the loop.We just made surethat no matter what we generated,no matter how we produce something,we always put it in frontof the user to be able to interactwith and respond with.Our principle is that the merchantcontrols their message to their buyerand generative AI is a way to augmenttheir ability to get there faster.We started with the areas that we thoughtwould be the biggest needand saw how users would interact.They asked for more of it,so we tried to put it everywhere.Then we've moved into the phase of,how do we rethink the entire company?How do we think about how a userinteracts from a previous imperativewhere you're clicking and choosingand filling out forms to a declarative onein which you just simplystate what you want?We knewthat it wasn't going to be possiblegiven where the technology was there,but we trusted that the technologywas going to movefast enough that by the timethat we got there,we'd either be behind the technologyor we'd be right there with it.It was always a matterof keeping up and making surethat our merchant's use caseswere being solved as we went.You mentioned using AI everywhere.What is an exampleof a way that you're usingAI that might not be obviousto the outside observer?Sure.We have one feature for usinggenerative AI that's just auto-write.Anywhere there's a field or a form,we allow users to adda couple of keywords and declarethe voice and any kind of specialinstructions they might want to do,and they can generate any sort of content.That one's probablya little bit more obvious.Another place we've doneis send-time optimizationon an email where you might not knowwhen to send a thingfor the highest open rateor highest click-through rate,and so we take care of that for you.Craft a subject lineso that it'll get more open rates.We have thingsfrom a Sidekick perspective.For those that don't know,Sidekick is an AI assistantthat's across the admin.What we've usedit for is both curating helpas well as directing the back officeso you can do things like, say,\"Change my themeto make it look more like summer,\"and having the semantic understandingof what that design would be,as well as the understandingof the tool itselfto be able to generate the codenecessary to make that happen.I think those onesare a little bit more like,you can use the AI as a design aid,but we want to moveinto a stage of it's a strategic coach.I think that that's where thingsstart to differentiate.Great. Kathy, question for you.Salesforcehas really been at the forefrontof building AI productsfor business workers,for the information worker.I'm curious how you balance the needfor innovation withensuring that AI is safe.Absolutely.We've been working on AI for several yearsbut really started focusingon creating AI ethicallyand responsibly since 2016.It was just a natural flowfrom our core values of trust,customer success, innovation,equality, and sustainability.It wasn't a large leap to gofrom that to more specifictrusted AI principles,which we published in 2019.At the beginning of this year,as we started puttinggenerative AI into our products,we recognized that we neededmore specific guidelinesto help all of our teams think abouthow do they putthis technology into B2B products?Then we came up witha set of five guidelinesfor responsible, generative AI.Accuracy was the first onethat we really had to prioritize.Everybody as a consumerwants to ensure that their searchresults or maps or anythingelse that they're getting,that those answers are correct.In a business setting,if you get the content wrong,it can have materialimpacts on the business,it can have brand impact,legal impact, safety impact.Really focusingon that and then ensuring that it's safe,honest, empowering our users,and then also, of course, sustainable.All of this has driven every decisionthat we are makingfrom within the product,how we build our own models,how we leverage OpenAI in our products,as well as the UI, the design,giving our customers the toolsto empower them to know,is this content that you're getting,is it accurate?Is it trustworthy?-Great.-Can I just jump on the ?Please.You sparked something in my brain,which is,I remember I used to workat Twitter before Typeform.When we were building new things,if you make $100 millionor $1 billion from something,you're just so careful about it,you craft all these rules around it.This is not advocacy,by the way, but I thinkthat we're in an inflection point withAI where the two things matter, right?Your ability to be creativeand your abilityto understand your customer.The combination of those two thingsis what unlocks value.Because we always underestimateabsorption rates of our customers.One is creativity,one helps you with the absorption rate.Ways to learn quickly, to applyfewer rules so that you can learn,and then integrate that into the main bodyof development is so critical.It's so critical.Optimize for learning, optimizefor fewer rules, then don't be unethical.I'm not saying that.I hope not.It's really important to do thatand that animatesour approach at Typeform.Yes, understandingyour user is so core to that.We talk a lot about mindful friction.We want to slow the users down enoughthat there isn't just this mindlesstrust like, \"Oh, this must be right,\"and just submit, but to actually check it.We don't want to put in so muchfriction that it's viewed as a speed bump.It's an annoyance.It becomes like banner blindness.It's not useful at that point,but creating the mindful friction,creating the signals for a customersupport rep in a call centerwhere some actuallyhave a countdown timer to getthose customers offthe call as quickly as possible.Very different experiencefrom a sales rep generatingan email to a cold call,or a marketing campaignmanager generating a campaign,or developers writing code.Understanding your customersdeeply to understand how they needand how you can best empower themfrom the beginningis so critical to getting it right.Not doing that just launch,move fast, and break things.That doesn't work,especially in a B2B scenario.Yes, I like the mindful friction.Miqdad, anything to addon the enterprise side of the equation?Yes. I'm going to go on the opposite side.I tend to believe that puttingthis product out in front of peopleand seeing how they interact with itand how they break itis actually how we can learn to make itmore ethical and how we can learnto apply that appropriate friction.The customer service example is great.We actually produceda chatbot for our help center.It was previouslyjust search for articles,find the article, see what makes sense.We introduced a chatbot on top of it,took all of our data,made it into a set of QA pairs,and created embeddings from that.Then we said,\"Let's see how people interact with this.\"The first thing that happensis someone comes in and whatevertheir normal pathof search was going to be,they'll just start typing,and then we'll form it into a conversationand return the referenceresults as they go.Then they will ask for clarifications,they'll hit a wall somewhere,and then we will add a human into the loopand let a customersupport person take over.The goal is to solve the problem.It's just that sometimes the waythat the individual is searchingor the way that they are identifyingwhat they're looking for isn't great.That's where an LLMis great at interpretingthe semantic meaning of what they wantand then getting to something else.Is it the best experience?Is it always right?Probably not.I think the fun part about this is thathow many people's problemsdoes this now solve?I think that that numbercontinues to improve.When we get into metrics later,I'm sure we will.We'll talk about how we showthat this is effective.For us,it's pairing an LLM with really good UXand really good engineering.It's got to be both.The engineering sideof it is very critical,and I think probablythe majority of the work.The beauty about the waythat OpenAI keeps doing thingsand my whole world map just changed,is to be able to--They react and they giveyou the framework around the LLM,and that box keeps getting bigger.The sort that doesn't getbigger is your interactionsfrom the LLM to your applicationand to your user.The UX is critical of formingthe right basis.I do not want an entrepreneurto have to worry about AI.-That's not the goal.-That's right.The goal is for the entrepreneurto get betterat what they are doing, and that's it.They don't need to knowwhat's going on behind the scenes,they don't need to know AI.That's my problem,not the merchant's problem.I'm going to say amen to that.That's the way that we focusand we continue to pushthe boundaries of what's right on the UXside of it and the engineering side of it.On the LLM side of it,we just plan for every error possible.Where we don't have an error,we put a human there to deal with it.That's what's really important,is that human backstop.It's like if it fails,then the user is left struggling.No. That's where the humansteps in and that's really critical.You're going to do experimentation,always having that safety netto catch your users is so important.100% It keeps us honest as we go throughand a merchant can alwaysturn it off if they want to.For us,it's important that because we don't knowsome of the use casesthat will get solved,I'll give you a very trivial anecdote.We had a Polish entrepreneurreach out to us and they said,\"We're really excited aboutyour product descriptions product.\"All it does is it generatesa product description,50 to 100 words basedon a couple of keywords.Initially, it launched only in English,and then we expandedout to other languages.They said a thing I wasn't expecting.They said we were scaredto sell to English marketsbecause we worried that the waythat we wrote English wasn't good,but you solved that for us.I was like, \"If we didn't try,then we don't learn that one lesson.We don't learn that that anecdoteis out there and there are hundredsand thousands of others like that.If it makes 1 more entrepreneur,10 more entrepreneurs,it's a worthwhile effort.\"I love that example.I want to switch to a hot topicthat we get a lot on the sales side,which is pricing.As we know, GPT-4 is not cheapthough it just got a lot cheaper.Hooray.We get a lot of our builders wondering,\"How do I think about the pricing?Should I bill it back to my users?Is it an add-on? Should I eat the cost?Do I build a standalone product?\"I'm really curious to hear Oji and Miqdadhow you weighed the trade-offshere and what decisionyou ultimately madeand how to price this in your products.-Should I start?-Go for it?GPT-4, we were excited aboutthe rest of the world when it launchedand it was better on all the dimensionsthat are public, but it was expensive.I think fundamentally we thought that AI,the way we thinkabout it is time to value.Because it's time to value,shrinking our customer's workflowthat we already knowby 50% 100% whatever that is,we needed it to be for our customers.How we thought about pricing,first of all, was to say, \"Okay,this is better and betteris good for our customers,but it's really expensive.\"We x-rayed the functionalitywe were trying to deliverand tried to find outwhat was really important to do with GPT-4and what was not important to do withGPT-3.5.Then we spent a lot of timemanhandling GPT-3.5to be really, really good at the thingsthat we didn't need GPT-4 for.That allowed usto do more experimentation with pricing.First of all,AI is baked into the product.Even at a base level,we are B2B SaaS, PLG.People just walk in the door and sign up,but we still want to make surethat their first taste of that power,that acceleration is GPT-powered,but then if you really want to do thingsthat require GPT-4 exclusively,then our pricing getsa little more dear for the customer.The other thing I'm going to just leaveyou with is sometimeswe think of pricing as sacrosanct.As something that peopleif you change it a little bit,people will be upset.I think that's a dirty lie.We do a lot of price experimentationin Formless, but also in Typeformbecause the value propositionto customer is always changing.Some use case is more valued,some are not.The question is, \"How do we adjust that?How do we impedance maxthat to customer's perception?\"Experimentation in this age of AI,especially when models--When 5 comes outit'll be really expensive.This is an evergreen topic.How do you price, how do you experimentwith different ways to price?Even now with doingsome experimentation on pricing.That's the way we think about it,is just reach deeper than justthe cost of the LLMor whatever version you're using.That's us.I think you guys are a little bit furtheralong on the pricing side of things.What I'll do is I'll walk you through justhow we've been thinking about it.Shopify has three principlesby which it operates.Principle one is \"do whatever it takesto make the merchant successful.\"Principle two is \"make moneydoing it so you can do more of it.\"Principle three is \"never swapthe order of one and two.\"We'll always start with the notionof what is the problem that we're solvingand what is the best wayto be able to solve it.With this, what you do is you really lineup the incentives.The products that we release,the featuresthat we actually green-light are the onesthat are going to move the merchantin the right direction towards success.When they are successful,we are successful.However, it needs to make surethat these aren't unsustainablecosts for us.In some cases,GPT-4 can get expensive and GPT-5,I'm told is going to be cheaper.Is that right?I'm just kidding.We can hope and dream.I think the idea is that we didthe same thing that Oji's talking about.We looked at use cases for GPT-4versus use cases for GPT-3.5when GPT-3.5 Turbo came outand it was significantly cheaper,we're like, \"Okay, how much can we leaninto this because this will bea really good place for us?\"Ultimately, what we're trying to do rightnow is give everything to everybody.We're starting with everyone gets it,and we'll start to seewhat some of the patterns are of usage.Where there are potentialand problematic instances,we'll look at addingUX for it to add frictionwhere necessary or where appropriate.The other partof it is we will likely introducesomething but what that is rightnow is not clear.I think we're still sortingthrough what the specifics are.As the usage patternsbecome a little bit more obvious,we'll work with that.Great. All right, another question.It seems like everyoneis building chatbots.That has definitelybeen the product du jour.I'm curious, what are some novelways your teams are integrating AI?Maybe we start with Kathyand work our way across.We have such a broad rangeof industries that we support,not only our cloud, sales,service, marketing,of course, our developer tools as well,and then we also have our individualindustries.Developing bespoke solutionsfor each one of those industries.We announced at our Dreamforce event,Copilot.Creating assistants that are bespoketo each one of those kindsof applications.We've also talked about,on our AI research science team,developing large action models.Being able to createa whole series of modelswith an orchestrator that justgoes off and does individual,smaller tasks for you,but they're coordinated together,and being able to identifythe specific actionswhere we want that humanin the loop to make sure thatthat particular pieceis reviewed and approved,but all of it can worktogether seamlessly.That's one of the areasthat we're working on right nowis creating those bespokekind of solutions.I can't wait to get my salesteam on your Salesforce Copilot.That sounds great.Yes, I have so many thoughts on this.I would just say that AIuser experience is a thing.Maybe I'll say somethingeven spicier here.I don't know that I believein a text-chat interface for all of AI,basically.Nothing, obviously,I use ChatGPT every day.If this is truly to be pervasive,we have to get really innovativeabout how people interact with AI.It feels like everyoneis saying it's chat.I don't believe it is.We actually know a lot about chat.If you're old enough,you understand command-line interfaces.There's a reason as Steve Jobsand Bill Gates createdmice and interactionsand so on because it's just less calories,less cognitive load.Customers don't have to be as creative.I think AI/UXis important and is the thingthat's going to differentiatefrom chatbots.I'm very impressed withthe Assistants API, the demo app.It feels new and fresh.People will have to keepinnovating in that space.That's what I really believe.Because you have to lowerthe bar of how people interact.People don't want to think.For them, technology is now utility level,and so it just has to work.The other thing I would sayabout is that I've said it again,but I'll repeatit because I think it's important.Study your customer's workflows.If you're a product person or a developer,then you've heard about specsand use cases, no, forget those.Workflows. What do they do?What do they walking?How do they actually accomplishtheir tasks?Rap around the AI userexperience around that, okay?If you can do that,you will unlock more value.There's a lot of great ideas in AI.We've done them, we've released them,we've talked about them here.Our industry forgetsabout absorption rate.It depends on how you wrap it around.I'll stop there without being specific,but there's still more to be doneon how AI interacts with humans.Typeform is about makingtechnology for humans,not humans for technology.We believe in innovating in that space.I like that.I might steal that from you one day.I'll give the continuum of why I thinkchatbots are the thing right now.I would say that most peoplehave been buildingapplications from the traditional senseof, \"I want to solve a problem,here's how it works,here's what makes it work better,here's how I add featuresto make it more operational.\"I think the switch now is,\"Oh, AI is easier to get to.\"It's much easier to throwa large language model in thereand tell it to do a classification problemor tell it to do a predictionproblem and see how it does.I think we're on a continuum right now.We're starting in the understandthe user's intent,then it's predict the user's intent,then it's predict the user's action.I think that's the continuumwe're on right now.Chatbots come in to explicitlyget the user's intent.\"What would you like to do?Tell me what the weatherin San Francisco isand whether I should bring a jacket.\"Cool. I'm starting to understandbut now I have the contextof when that intent was made.It's like, \"Oh, they were here.This is what the situationwas and this is what they asked.\"Now I have the potential to predictwhat the intent was and why it was there.Then you go into the next phase of itof potentially predictingthe intent and going rightinto the applicationlayer of bringing forward.Here's an action you could take based onwhere you are and whatyou are currently doing.Then it's a matterof automating that same intent.It's like, \"Okay,well we know what you're doing.We know what the right way to do it is.Here's a potential paththat will get you to the final stage.\"I think that's the continuumwe're on right now.A lot of peopleare in the how do I make chat do a thing.I think it's useful and I thinkit's really helpful for peopleto feel they have much more controlabout what the set of actionsare and potentially have a better wayto describe what they wantwhen the UX falls apart.The thing I want to cautionpeople on as you build,do not let AI be a substitute for bad UX.Make the UX great.Make the AI orchestrate the good UX.If you have a bad UX problem,solve the bad UX problem.Don't worry about AIbeing put right on top,bandaid it on topof it to solve for that problem.I think that's where peopleget a little bit tripped upand it's not one or the other.It's both at all timesas you move forward.Can I compliment that demo app?It says create and then it says,what was that toggle?It was create and then configure.Then you start with a little bit of chatand then the configuration is fine-tuningit and then it had all the UI on the side.I thought that was really good.I thought that was a good sequence.It walked the walk of customers.I'm sure if I thought about it,maybe there are things I would do.I thought that was very smart. Thanks.You guys have some good UXpeople out here.Great.Just to hitone of the potential novel cases,I thinkthere's a lot of here's the customer,here's the solve,here's the things that we can do.It's very user-facing.There's a lot of internal use casesthat are interesting and very compelling.One example of it is we havea massive migration to do.In order to do that, we have to takeone form of code and switch itinto kind of an object-orientedextensible platform.We have hundredsof thousands of variationsof how people have done things with codeand we have to figure outwhat the right extensions to create are.We just wrote a classifierthat takes all of the code and identifiesall the opportunities for extensibilitythat exist today versus onesthat we need to build.That gives us a hit listof here's what we have to build.It's like we put it as partof our internal process too,just to accelerate but you havea classifier at your fingertips.You have a generator at your fingertipsand a large language modelcan be used in many, many places.For us, it's about acceleratingeven our own business flows.It's not just about the external user,it's also about the internal usermaking it faster.Yes. I'm curious, are eitherof you using AI internally right nowto help with your own jobsthat you want to share?Yes. I'm a product person.I'm an engineer even further back,but usually,I'm in the product and design realm.One of the things that I usea lot is the copilot tools,but also the native codeinterpreter inside ChatGPTbecause it just allows meto very quickly make myselfmore productive by writing a small tool.I wrote a Chrome extension the other daythat I've been plotting for like a year,but it took me 30 minutes to do it.ChatGPT feels like a Swiss Armyknife right now in many ways.I use it in creative ways.I do a lot of writingand I give a lot of talks.Sometimes I hit a writer's block.I remember there was a talkthat I had agreed to do like three monthsin advance and I keptkicking it down the road,down the road,down the road because it was so far away,and then suddenly it was the next week.Just that writer's blockthat panic hit me in that moment.Being able to pull up ChatGPTand just start playing with itand getting inspiration and talkingabout some of these topicsthat I've talkedabout for a while but in a different way,it was such a relief of just havingthat inspiration to get me started.It was quite a right lifesaverin that moment.ChatGPT might help writesome of these questions today.Want to do a little bit of rapid-firequestions, a couple of questions,just answer in a sentence or two.We'll start with Kathyand work our way across.What is one AI myth you'd like to debunk?I think there are still some individualsthat believe that AI is neutral, that it.doesn't come with bias or opinions,but it very much does.We have to actively work to ensurethat it is safeand representative for everyone.I think that people thinkthat all these announcements,all these advancements in AI will maybeautomatically guarantee productsthat make a lot of money.That's amazing and the world will change,but there's a lot of workto be done at the productand design layer to unlockthis actually gettinginto the hands of the peoplewhose world you want to change.It's not just prompting,there's a lot of design in itI want to change the narrative on.I'll go the opposite wayof myself usually.I don't think AIis a silver bullet and I don't thinkthat the level of confidenceit projects is warranted.I think that that's somethingwe still have to manage.Next question,what is your go-to metric to measurethe success of your AIproducts or initiatives?I absolutely appreciate user feedback.For every piece of AI-generated content,we have that very familiar thumbs-up,thumbs-down.We also have a drop list where we askif the user selects thumbs down,what is the reason it was inaccurate,wrong, voice and tone,toxic or unsafe, other.Being able to get that feedbackfrom users and knowwhere the miss was to be able to helpus just continually improve our products,I think is incredibly powerful.There are two that I think about,but I already know that Miqdad--You'll steal my thunder.No, I won't,because one leads to the other,so you can talk about a second one.I won't take your thunder.The first one is time to value.For things that are conventional,using technology of today,I'll say pre-AI, or pre-generated AI,how can you compress it?How can you make the timeto value even shorter?It took 30 minutes to activate.How can you make it in five minutes?Typeform, for example, you can take20 minutes to make a really good form.How can I take that down to a minuteand it's still as good.Superpowers, like cold dead hands,territory of holy shit.Time to value is a huge one,and then you'll talk aboutwhat it leads up to, I guess.I think whenever you do products,it always breaks down to two core needs.We're saving money or we're saving time.You hit saving time.I think that for us, one of the big thingsis sustained adoption.It's people don't necessarily have to payfor a feature to let you knowwhat's valuable.I think people pay with their time.If people are using it consistentlyand the people are getting value from it,they will showyou that through sustained usage.Regardless of how good you thinkyour product is,if your users don't use it,then it does not matter.I think that that's whythat's a pretty critical one for us,is just sustained usage.I think it's very easy to saythat these thingsare going to be hot and these thingsare going to be popularand people will try thembut if your curve lookslike this and goes to zero,then what do you really have?That's right.Great. I want to end withfinal thoughts or takeawaysfrom all of you on the futureof AI product development.Why don't we start with you,Kathy, and work our way across?This technology is an amazing opportunityfor businesses to becomeeven more efficient and productivein what they are able to do and then passthat value down on to customers.I think ensuring that all of us understandwhat it meansto do that responsibly is critical.You can't outsource that,your ethical responsibility,to a single teamor a single individual on your team.Every single individualthat is building the product,that's responsiblefor the marketing of it,that's responsible for the sales of it,everyone needs to understandwhat it means to create, implement,and use this technology responsibly.I think on one hand,you can be confident, on the one hand,you can be completely,we don't even know what's going to happen.I do think that AIwill infuse every single thing,every single tool we useto build products today.Maybe in non-obvious ways.That's number one.Two, it feels like a resetfor the toolset of creativity, I think.Every dev tool, every coordination tool,communication tool,the things that let people work in teams,I believe will completelychange in the next five years,or even sooner than that.I can't predict exactlyhow they will change.I am confident thatyour collaboration innovationworkflows will change completely.I guess before I hand it over to Miqdad,I'd say expect change, anticipate it,maybe lead it if you can,think about ways that these thingscan change your workflow.Maybe do a startup to allow teamsto collaborate better in this environment.We have Typeform Labs,and we're constantlyimagining that actually,because we have the luxuryof doing some of that,and that's how we created Formless,that AI.Those are my thoughts.I swear we didn't knoweach other before this.I think that past researchis an indicator of future acceleration.If you look at past researchon machine learning,you can see that there was a startof a spike around 2020,and it started to climbin an exponential curvetowards 2023 ish, even more so now.Then what do you know,ChatGPT shows up on the scene end of 2022and we get accelerationafter acceleration after acceleration.I think that if you lookat the amount of researchthat's being done rightnow in the space of machine learning,you have subscription models that are,here the daily number of papersthat have been introduced.It's no longer a matter of a paper a week,it's seven a day,and it's difficult to keep up.I think that what we needto understand what that means for usis that all of the problemsof today will be gone very quickly.Whatever you thinkis a problem right now of,\"Oh, this is very expensive,\"people like OpenAI come outand they reduce the cost by 10X overnight.Then you have, oh, the latencyon this is not tenable for my users.Then all of a sudden,GPT-3.5 Turbo comes out,and all of a sudden everything is quick.The context window is too small.Oh, 128,000 token context comes out.Every problem that you see todayis going to be solved whether next week,I don't know what your next conference is,or thereafter.a little break.What we need to understandis that if you miss the boat right nowand do not start buildingand do not start experimenting,there is an exponential curve.It will take you so muchmore effort to catch up later on.Build now, experiment now,get your users in front of it,and you will learn much fasterthan if you waitfor everything to be perfect.It'll get there,I assure you probably before you will.Great. Well, on that note,I hope you all enjoyed the sessionand feel inspired to go build AIin your productsand your organizations to build enduringand high revenue-producingwildly successful businesses.Thank you.\n"