Michael Fauscette

Welcome to the disambiguation Podcast where each week we try to remove some of the confusion around AI and business automation by talking to experts across a broad spectrum of business use cases and the supporting technology. I'm your host, Michael Fauscette. If you're new to the show, we release a new episode every Friday as a podcast on all the major podcast channels on YouTube as a video. And we also post a transcript on the Arion Research blog in case you want to stop by and read it. Welcome, today's show is a special edition from Deltek ProjectCon in Orlando, Florida at the fine Gaylord hotel. And I'm joined today by Warren Linscott, who is the CPO, Chief Product Officer. And we're going to talk a little bit about my favorite subject, of course about AI. Welcome, Warren.

Warren

Thanks for having me, Mike. Really appreciate the opportunity.

Michael Fauscette

Cool. So just tell us a little bit about yourself first.

Warren

Sure. So I'm the Chief Product Officer at Deltek, as you mentioned. I've been with Deltek for over 14 and a half years now. So it's been quite a journey, I got my start there as a product manager in the accounting and finance side of one of our ERP solutions for government contractors. You know, prior to that I worked at JPMorgan Chase in Treasury services for a number of years through an acquired company, been at that company for you know, eight or nine years doing import - export trade logistics and controls, software and services. So, you know, been in and around regulatory, regulated kind of enterprise software environments for a long time now.

Michael Fauscette

Great. And we've known each other for quite a long time, I should say. And in fact, this is not the first time we've done an interview together.

Warren

That's right. Yeah, we've done several before.

Michael Fauscette

Yes, we have, although not for this fun podcast, something else. Right. All right, well, let's just get into it. So, you know, and like I said, I wanted to talk a little bit about AI and what you guys are doing, and I did sit through a session this morning. So, I know a little bit more about it. But you know, for the audience, I think probably a good place to start is the most common use cases for AI in project-based business. And then which ones offer the biggest business benefit to your customers?

Warren

Yeah, I think I think when we're talking about Gen AI, you know, for talking about the new stuff. I think right now, where we see the biggest benefit is in a digital assistant that can really help them understand context to the data they have in their project based systems and also to help out their users. Because in most of our solutions, we differentiate on the project dimension. So, we're adding a whole nother dimension to the way that businesses normally operate. So usually, businesses are operating by locations by types of accounts. And then you throw the project on there, and things get a little bit more complex. And so being able to dispel that from an adoption perspective, from just to synthesizing data, I had a conversation with an engineering company earlier today. And they were really excited about the prospect of an assistant to help the people that they don't give access to the data, so they can get access to what they need to do their job. So I think, fundamentally, as we kind of get started in this gen AI world, that's probably the thing that's going to add the most value right away. Yeah,

Michael Fauscette

I mean, that that makes sense. Most project-based businesses are very dispersed, and you have a lot of remote workers. It's right. And that's, you know, they were remote way before remote was cool, I guess. So that that makes a lot of sense having the access to the assistant. Now, I know, you've done a few other things with it in the in the product. So maybe we could just talk a little bit more beyond the assistant, which you know, I do see a lot of business value. But what are some of the other things and I know, also in the presentation this morning, you were breaking them out by AI, traditional AI and Gen AI, which makes sense to me. So maybe we could talk a little bit about both sides of that. Sure. See what else you're doing.

Warren

Yeah. So we started talking about traditional AI in the sense of heuristic algorithms, where we're kind of hard coding all the parameters and kind of the way that like you even think of Monte Carlo simulation as a heuristic kind of form of algorithm, which we do for like scheduled realism and things like that. And then machine learning, we kind of put it into that traditional bucket now too, because it's usually narrowcast over a specific type of prediction that you're trying to do with this specific set of data and algorithms versus Gen AI, which is now with the large language models, it just has a much broader scope. Now, I think long term, I do believe that they'll start to converge a little bit as you get training datasets that cover what a distinct machine learning model would have covered into the general AI model, then that will disappear and so you get more narrow and error functions for machine learning, you know, versus just a generic model. But today we do see that they can be used together and in terms of some of the things that we've done. So, we talked about the digital assistant, I would say that's more like an uncurated model. In other words, you can ask it anything you want, we use it to actually figure out what you're asking for, and then try to direct you to the right place in the application to get the data versus a curated experience, which is some of the summaries that we've done. So we have something called Smart summaries that we have not only in our GovWin IQ, which is our information service product, but also in some of the other products where we can take a different data entity like a federal agency profile, or a client or project and generate a one to two page summary that gives you somebody that doesn't, you know, isn't steeped in that particular agency, or that project, just a high level overview of what it's all about. And so this is hugely helpful. As you know, there's limited people that would have access to know how to query that information out of an ERP, or out of an information solution, we can curate that journey for them to give them a generative response that they can then go and use, you know, wherever they seem to use it. So, it's really time saver, but also a way for these companies to more easily distribute information. So that's, that's kind of a curated general use case in terms of machine learning. So, we've got several different machine learning applications we've already put out there like one is, we take all the government procurement data, we crunch it into a model, that we have characteristics about federal contractors, and we kind of mash the two together and say this, you're a good fit for this opportunity or a bad fit for this opportunity. And we refresh that I think, once every day and to rescore everything as new data comes in, you know, that's an example of where we're just straight down the middle of doing machine learning, I think where these come together is where you can kind of string a workflow or job to be done together. So, we talked about, you know, being able to help a resource plan or identify resource needs with a machine learning model. But then using generative AI to help create the job requisition, the job description, the job posting everything that you need to help speed up the recruiting process.

Michael Fauscette

So like, even with the summarization like that could make for a project manager, you could do your status reports. That's right. So that would speed up that process as an example.

Warren

Yeah, that's an exact example, actually, that we're doing. So, in the government contracting space, there's a concept of variance reporting. So earned value management is required for DOD contracts over $20 million audited really over contracts over $50M. But they have to every month, any variants of schedule or cost, they have to have an explanation and impact and a corrective action plan. And so when you're managing a huge program, like a destroyer, or a fleet of fighter aircraft, you could imagine there's hundreds, if not 1000s of those, and to chase them all down, look at them for accuracy to even generate them from the information in the systems is a challenge. And so we see big opportunity there. 

Michael Fauscette 

Yeah, I mean, that could make a lot of difference in the, in the flow of the project. And the project managers time.

Warren

That's right, the other and they can focus on the real work, which is making, you know, the corrective action.

Michael Fauscette

Right. Right. Yeah. I mean, that's, uh, you know, I, I'm an ex naval officer, and I've been through the ship overhaul and the ship construction process. And I've seen those Gantt charts that like span the entire room. So I can imagine in a project that size are probably a lot of variations.

Warren 

Yeah, it's likely that it was one of our products and formulated that Gantt chart

Michael Fauscette

Might have been, might have been, maybe that's why that overhaul took so long.

Warren

I doubt it, we would have saved the overhaul.

Michael Fauscette

Oh, yeah. Okay. Got it. So let's focus a little bit on Gen AI, from a project perspective. You know, how the assistant can see, and we talked about that a little bit. But how does Gen AI in general, help project-based businesses do more efficient work? Better Work?

Warren

Yeah. So I think that, you know, right now, we're just kind of getting started in terms of taking our experience from supporting our products and putting that as using that to educate a digital assistant. So, it starts with the adoption, user adoption and kind of user kind of journey, meeting them in their moment of need, but taking your knowledge base articles, as well as our product documentation, and putting that on top of a generic model.

So that's like a tutor.

Yeah, exactly, exactly. And so when you think about many of our products, especially when the more sophisticated ones around or in value, or around manufacturing, bringing new people into the workforce, and educating them on how to do these things, and how to be a good project manager is a challenge. And we have a lot of that knowledge and accumulated metrics that people have allowed us to preserve where we can help to give people insight into what is a good amount of unbilled to have in a project at the beginning phase, the middle phase and the end phase, right, what is the good amount of open AR that you should have? So taking this stuff that we've accumulated 40 years of knowledge of getting projects done, and putting that on top of them? model is really going to help give people advice and context, along with, you know, the ability to create simple transactional data. Because that's really, I think, where the challenge is, is, as the workforce evolves, as new people come in, how do you take an ageing workforce and teach everyone the way it was done or the way it needs to be done now, right. And we believe that as a model that has kind of our intrinsic knowledge and IP can really help our customers with that problem. 

Michael Fauscette

So I mean, just in time learning, I, when I years ago, when I was at PeopleSoft, we did a study about that, about that exact problem, right before software, what's the most effective way to teach people how to use it. And, you know, at that time, of course, we didn't have Gen AI, we didn't have like, online tutors, we were trying to decide if online help was better than the manual hanging in the, you know, on the shelf. And, and but the biggest thing was that the most, the best way people responded was when it was in the context of the thing they were trying to do, right? Because if you learn in a classroom, you learn all the system, and you don't do it for six months, there's a thing that you've got to do. Now, I need to know how to do that, because I didn't remember right. I mean, there's no way I retained all that. So that actually, that could make a huge difference in the usability of the product.

Warren

Yeah, absolutely. And I think that so today, right, the step one is make Gemini aware of things that are easy, or knowledge base articles are help documentation. The next phase would be, how do we safely make it aware of the context meaning the configuration of the customer, and their data history, and if we can do that in a safe and effective manner, which I believe that we can, I think that security is and data privacy is really big here, and, and the big providers are not going to ignore that they're going to have to deal with it. And there's ways in which we can help with that. But you know, that context of how they've deployed the software of how they run their business, along with our context together, that's where the magic I think, can really happen.

Michael Fauscette

You know, that, I mean, that could be a huge benefit, I could definitely see that. So, you know, would be useful, I think, for again, talk about the project lifecycle would probably be useful to kind of put that product life's project lifecycle in context of the things that you can do around that to deliver a better experience to the customer.

Warren

Yeah, so just real quick on the project lifecycle for those who haven't heard or heard about it. So we really look at, you know, Deltek is industry specific. So we look at project based industries, it's like government contractors, manufacturers of, of engineer to order type of manufacturing, architects, engineers, construction, consulting, creative agencies. So that's we don't stray from that. That's one lens. The other is the project lifecycle, which all these businesses, their entire business is usually a book of discrete project work. So it really starts from the ability to be able to find and win new opportunities. You know, once you've won an opportunity than that, now you gotta run a project. So you got to have had to have a good plan, you have to be able to critique that plan, make sure that it's going to set you up for having a great project, you got to find resources within the business that have the right skill sets and match them up with those project needs. You have to find and develop talent, almost all these businesses, whether they make a product or not, you know, the their talent, their people is really what sets them apart from one another and makes them unique. So being able to find new talent, bringing into the business and cultivate it is important. And then delivering on the projects, being able to do all the back-office work to deal with a corporate, you know, goals that somebody might have, but also the project work, the deliverables, the tracking of it, the billing, the capture of time, all of those things, you know, go into kind of deliver side of the wheel for us. And then there's a measure component, which says, Okay, let's look at all those different phases. Let's connect them, let's understand, how can we get better? What do we do? What do we do? Well, where's our best profit margin on what types of projects and then using that to think about? How do I go after my next pursuit? So that's why it's kind of a lifecycle for us. And what we try to instill is it when you learn what you're really good at, you cut out what you're bad at, and you kind of accelerate so we had, you know, story we like to tell we had an architecture company that did hospitals and golf courses. That last once they implemented our software, they found out they lost money on every golf course they ever did. So they stopped doing that and the profit and growing up.

Michael Fauscette

Right, right. I mean, that's, yeah, that's good. And I mean, that's not data that's easy to come by either. Because you look at every project in a discreet way. But what you've got to do is now look at all of those projects and figure out which type and what yeah, yes.  

Warren

so around the project lifecycle, it's really there are distinct applications, which within each of those phases, but a lot of these phases kind of blur the life of a project, right? So you don't not just creating a plan, you're creating a plan and then managing it over time right. So being able to be more predictive about you know, what are the different characteristics around the project that are telling you that this is going well or not like I mentioned, you know, on Build time or open AR, right? So if you have open AR at the beginning of a project, not that big of a deal, if it's the end of a project, big deal, right? Something's happening, why is the customer not paying us? And being able to, to kind of pull out the characteristics of engagement of your employees, as well as engagement of your customers to tell you? What is the true temperature of this project? How does that roll to a temperature of a division or the company overall. So I think that the application of gen AI across some of these problems is going to help us to analyze all that vast amount of data that characteristic and tell us and I'll give you an example that we use actually ourselves. So we have this old algorithm that we developed to help us understand what's a good fit for government contractor, because there's 2 million businesses registered to do business with the federal government, but only, you know, 40, or 50,000 of them are good candidates for us, because they do cost plus work, project work, TNM, and so forth. And so we started to use gen AI, to say, hey, characteristically, look at this company's website, look at their social media, you know, look at publicly available information and tell us if they're a good fit for us. Right, so we can, you know, our customers can start to do the same thing and say, Hey, what's a good project for us? What are our key strengths? What's the key strengths of this project manager, that project manager? What makes a good even employee in our business based on the work that we do? So I think there's almost the sky's the limit. And now it's just about prioritization?

Michael Fauscette

Well, I know we, we focus a lot on productivity and in on, like, the tactics of using Gen AI. But what you just said is interesting, because I don't know that I've explored this that much with people, but it's the strategic use of gen AI, the way you can tie this to your business strategy to actually go after the right kinds of projects, find the right people, the people that will be the most successful in that context of that project, etc. I mean, that's really interesting, because it does move it up a level and talks more about how I use this to for broader business benefit. Versus I, you know, I make my content marketers put out three times the amount of content that they use to what's also good, don't get me wrong, but yeah, interesting. Let's go to the front of that, though, cuz I know, a few years ago, I know you has acquired a product now that you call GovWin.

Warren

I think about 10 years ago, but

Michael Fauscette

yeah. You know, how long I've been associated with you guys at all seems to blur after a while. So yeah, I guess that was time flies. So taking GovWin in the context of when, how does that help government contractors in that, I saw the some reason you talked about the summaries, and that's really interesting. But what about in finding the right work  

Warren

so a couple different things, right. So one of the advantages that we'll always have within the GovWin space, or the government contracting Information Service space is we have, you know, dozens and dozens of analysts that are finding opportunities before they hit the RFP stage with the government. So 70% of the $2.5 trillion, that were of opportunities we're tracking is pre RFP, and that is human intelligence, going to meetings, talking to people in the industry, things of that nature, you know, I think, you know, one of the things that we're doing is, we're going to be able to put more towards that, and less towards bid scraping activities. So we can use where before, we would, we would have a lot of different individual programs, 1000s of them actually to go out to over 60,000, state, local and federal websites to get information, right, we had a whole army of people that would do that, write code for that, you know, curate the content for that generic can, you know, really help us accelerate that. And we can take that, that human capital and focus it on that pre-Award, which is like one of the huge values. And so we're already starting to do that. So like just behind the scenes, we're going to get much more effective of the secret sauce for GovWin IQ to begin with, because of gen AI is going to make us a little bit more productive, and also more accurate in terms of the information that we're pulling down, then, folks, but I think that if we go to like that fit score that we talked about. So the Fit score, is really interesting, because that's all machine learning. You know, we're looking at procurement history, we're looking at the past history of the federal contractor, and we're saying how do these things marry up? Now we're starting to create a Gen AI model to train it on taking in that context and give us a level deeper of context as to why and it's pretty amazing the things that it's pulling out in terms of where there's kind of like work that's adjacency. So for example, you would score negatively if you've never done business with this federal agency. But if we look at the contract details, or the bid details and the detail of history of work with this company, if we found that, hey, this work is really in this particular sector, maybe it's in electrical engineering, and you have a competency there, even if you hadn't ever worked with the Department of Ed Agriculture, which it would, didn't you in a machine learning model, right? And I can pick that out and say, actually, there's a match here that you wouldn't have otherwise seen. So it's getting to a bunch of different layers of characteristics. One, by the way, where, you know, sometimes it's so smart, where we're not even asking it for certain context, and it's pulling it back. Right? We're like, why is why is this a good fit or not a good fit, you know, based on these criteria, give me some weighted values. So you're giving a kind of a frame, but you're letting it go out and decide what to pull back. And it's pulling back characteristics like that, which we didn't tell it to directly. So it's pretty incredible. So, I think if we can augment our Fit score with that kind of context, that's going to give a much richer understanding of what opportunities are really good for a company. And also, if they want to diversify into an agency, how do they need to go about it? Where are they their real shortcomings?  

Michael Fauscette

And they can, they can find places where they can leverage the skills they already have. And so that's an interesting comparison in that conversation, and what you were just talking about the comparison between, and I think this helps people understand this between an algorithmically derived score, and a gen AI derived score, which is an algorithm of sort, but it's doing something different. Yeah. It has that capability to go beyond the framework that you put in the algorithm. 

Warren

That's right. It's you're looking at the entire context of information out there. Whereas machine learning, you're picking context story, and you're training on specific contexts. And you also have to be careful about, you know, how close that context is to the question you're asking, asking, because most machine learning algorithms, like for example, you know, predicting project success, if you use an estimate at complete as an indicator, it's gonna just peg to that, because it's already designed to be a forecast of where you're going to complete at right. So right, so it plays to its own string, that is exactly, exactly and that finite parameters, whereas, whereas the Gen AI, algorithms, and the large language models are such a broad context, you know, it's looking all throughout that training, set a bunch of different vectors, you know, that you, you would never have the time to pull out and train a machine learning model.

Michael Fauscette

Yeah. I mean, I think that's really interesting, because it does broaden the capability quite a bit out of this old paradigm of building an algorithm to do a specific thing. So project based business, you know, obviously, the biggest thing are the resources, the people. So what are you doing to help them with people, both from hiring people, which we all know, is a problem right now, in the US, and probably other countries, too. And then also in matching those people to projects and you know, finding out good fit best fit?

Warren

Yeah, so I mentioned kind of just so one of the things that I mentioned before was just shortening that what we call a record requisition to revenue, right? How quickly can I identify a resource need, and then go through the hiring process, and then bring that person on board, train them up so that I can build them out? Right. So that's a critical kind of across the project lifecycle component to successful project based businesses. But as we just talked about, on the government contracting, contract fit side, there is a such a thing as cultural fit, right? So I know that when whenever we interview somebody, you know, we're very proud of our culture, we work on that a lot of Deltek. You know, we just got a Glassdoor award for that third, I think technology worldwide and culture. But that's all based on a human filter. So that's based on in, you know, in in today's world, you know, with the labor market the way it is, it's, you can't run people through the same kind of gondola you might have before, right, so you're dependent on a few people, making sure not only do they have the credential fit, but they have the cultural fit. So it'd be great to be able to use Gen AI in a way to understanding how fit for purposes is individual to our company, and also like, what is their chance of success at our company? So I think that that is kind of the next frontier and I think we're going to start to see this show up in a lot of, you know, the HR capabilities and talent capabilities is, is addressing people's fit based on what's publicly available about them. And that's where, you know, I think we'll because if you don't fit in culturally say, you know, a Deltek or a lot of businesses, then you're not going to work out long term and you're wasting everybody's time. And so to be able to answer some of that question unbiasedly upfront, to get at least an indicator so that you're walking in more informed into the interview process, you can press on the right things and really find out if that person is going to work out for you.

Michael Fauscette

There's a in the in the startup world, the person who ran HR for Reed Hoffman wrote a book Patty McCord about what she calls the algorithm and that's this idea that you have to as a company, as a business, you have a specific need, you're at a point in your growth lifecycle, whatever. And so you need somebody that has that skill set. And also, it needs to be, from their perspective, what they need to be doing what they want to do, right. So there's, there's a match in three different directions. And one of those goes away, then, and I mean, we've all had the experience of making bad hires, not because the person is bad, just because they weren't a good fit for whatever culture or skill set or even size of business or where you weren't, right,

Warren

it's right mismatch and interpretation of what the job description was. Alright, so I think both ways, right? I mean, I think it's, it can help the company understand, hey, what is this is this potential resource, a good fit for the job, a good fit for our culture. And conversely, it'd be great to reach like, candidate relationship management is a huge deal and how you keep candidates that are, that are good for you warm, you know, to give the job seeker the opportunity to understand, hey, you know, how do I fit to this company, right, so really guide how they interview and where they go. So overall, it could be a pretty dramatic revolution on both ends of the labor market.

Michael Fauscette

I mean, that would be a win for both sides. Right, that that definitely is interesting. Now, I do know, there's been a little bit of pushback in AI, in, in HR in some areas, like, I think New York City now has some regulations about using it, and I've seen a couple others that have popped up. So if you guys looked at that, I mean, how is that gonna work?

Warren

Yeah, so we've looked at it a little bit, I mean, obviously, we're, we're looking at a lot. And we've had internal governance, you know, from our corporate parent to Roper, all the way to down to each rep or company has their own kind of guidance, you know, with legal involved with our privacy Council involved. And we're looking at that really hard, we have a lot of European companies that are on the vanguard of privacy, you know, individual privacy, and, and those trickled down through us through Massachusetts, or California and New York, that usually starts a trend and then goes nationwide. So we're obviously very sensitive to that, you know, this would, you know, the Gen AI, you know, in terms of like Azure and OpenAI and so forth, they fall into a subprocessor category, the data is transient, in terms of it coming in, you can turn off the ability to train these models don't, you know, contrary to popular belief, generally don't train on the fly all the time, right, there's a training period, and there's a model produce, and so forth. And it's about whether the company is retaining the data or not, that's the biggest thing. And so most of them have a setting to turn that off. And so it becomes transient data. And then it's about now you have to treat them as a sub processor, you have to list them as a sub processor, which has defined characteristics in terms of how that data needs to be protected in transit and in process. And so that's what we're looking at really hard. So we've, we've released a few features that are really based on for more or less publicly available information. And we're putting through our kind of quality assurance and are illegal and data privacy paces digital assistants and things of that nature is to make sure that all the obligations that we would have for a set process are being met in terms of how the data is transmitted, stored us not stored, what have you. So I think, I think what you're seeing right now, and the GSA also, you know, basically told everybody in the government, they can't use chat GPT. You know, and a lot of government contractors, I think rightly so. I mean, some of them have very sensitive programs, they're reluctant to allow the, these engines to be used, they want to see how the security posture shakes out. I think that's just some of the newness and I think some of the, the misunderstanding or potential misinformation about how these models can be used and configured and what have you, but you have big players, like, you know, Microsoft, and its investment in open AI. You know, Amazon and its investment in anthropic securities not lost on these companies.

Michael Fauscette

no, definitely not. And they all are also in the government contracting business as well. So they understand the sensitivity there.

Warren

Yeah. So, I think I think it's a more of a moment in time than a pattern. But we have to protect employee data, we have to protect, you know, company's data, which is their IP, you know, more and more. And so it's just going to have to be part of the overall algorithm for how this industry takes shape.

Michael Fauscette

Now, that makes sense. I think, you know, starting to hear from several enterprise vendors, different ways that they're approaching that, and I think that's a, it's obviously going to be something that a lot of companies care a lot about. And from a regulation perspective, too, I think so that's, that's interesting. So to what, what are the challenges for, for business for project based business that are thinking about getting into this? What are some of the challenges that they face and how are you getting over those things?

Warren

Yeah, I think that sometimes, the biggest challenge for anything that's new is to begin so I used to have this conversation with, you know, Namita Dhallan, who was the head of the executive vice president of engineering technology at Deltek before me, and, you know, we always used to talk about that, right, let's just begin, let's just get started. And I think that's the most, because once you get started, then you start to move around and get more comfortable over time to get more experienced, so forth. But the fear of starting is what holds people back a lot. And so that would be my biggest advice is to get started, because in the last 10 or 15 years, we have data science, we have data engineering, you know, now we have prompt engineering, we have, you know, a bunch of different roles that have cropped up, and they're very difficult to find, but there's a lot of younger folks coming into the workforce that were trained on these technologies, in college, in high school that are more had the mindset to go after this type of stuff. And my advice would be really to, you know, formulate a small group, you know, don't necessarily give them an expectation, and have them go learn a little bit about what this technology can do. Obviously, as a partner, you know, we're going to help companies by embedding some of this technology and making it transparent to the end user. But there's a lot of usages within every company that they need to understand. And so there's, we've talked about this, right, there's a ton of players out there, right, cropping up like crazy, you know, and really understanding the good, the bad, and the ugly in terms of the vendors that are out there. And then really starting to look at okay, so, you know, in, in marketing, in project management in, you know, the different HR in finance, you know, what are some of the things where this can help us from an automation perspective, and so forth. But, you know, just beginning on the knowledge journey, here is the most important thing and like, I tell people with, with fit score, you know, Kevin Flexco, and his team started out without any end in mind. They said, We think there's something here. Yeah, but and so for about six months, they just played around with the technologies played around with the data solid, it could return. And then that helped them formulate this idea for what would go into the product. You know, but if they hadn't had that, you know, that that chance to fumble around in the dark, they wouldn't have come up with anything.

Michael Fauscette

Yeah, that I think that it's interesting, because I definitely, almost every episode that when I'm talking to an expert about this, is I always ask that what you know, what should they do? When should they get in that sort of thing? And that's almost universally the answer is you need to get in and do things now doesn't mean that you have to put everything in production right away. Right? It some of it is just simple experimentation. And you know, the vendor, the providers of some of the technologies, as you start to embed things, there are things that you're doing that will make that easier from an adoption standpoint, without mentioned prompt engineering. And sure, if you're using all the standalone tools to do things, you have to learn to be some sort of a prompt engineer, you'll never get anything back. But if you're using embedded technology, I would assume that part of what you're doing from a product perspective is taking some of that away. So you don't actually have to do that. Is that Is that accurate?

Warren

Absolutely. Yeah. So we've got very curated responses, like the you know, the, the federal agencies, smart summaries, right? That's a curated response behind the scenes, we're generating, like all the different prompts engineering, and really, the user doesn't have to do anything, versus the digital assistant, we're actually using Gen AI to interpret the natural language question. Yeah, right. So this is where natural language search and Gen AI come together. So we the first thing we would do is send the questions agenda and say, what do we think they're asking? And we also give it an awareness of what's possible in our application and say, match these two things together? What did they ask versus what are the entities and functions within our application? And it does a remarkably good job of pairing the two together, and then asking us for information that we then recursively kind of go until we get an answer. But But I think that it's it's interesting, you wouldn't necessarily think that using Gen AI to decode prompt engineering is the way to go. But that actually helps us abstract kind of, you know, that end user from having to be an expert and prompt engineer. Yeah.

Michael Fauscette

I mean, that makes sense. And I know, I've even tried some tools that are that are, you know, prompt improvement sort of focus. And I would assume over time that can slip behind the covers and end up being in a function that is there so that a person is able to be more conversational with. So one thing to touch on, and I we're getting close to the end of time, but I wanted to make sure I hit on this Sure, was around UI UX, and kind of where you guys are going there because I think that's really important. And I've started to see this across, you know, several different product lines. So what are you doing around that?

Warren

yeah. So just in general, right. So we've made a really big investment and not only, you know, interaction design In visual designers, user experience, folks, research teams, you know, because we really kind of want to understand jobs to be done, how are people using our stuff? What are they really trying to accomplish in the business? And then how do we, you know, have a contemporary look, feel and kind of the combination of user interface and user experience to get the job done. You know, I think that, you know, having a digital assistant be ever present. So you think about it as a colleague that's coming to work with you every day, that's hooked up to the machine, it's hooked up to all our knowledge, all our training, all our experiences that can help either be suggestive about how somebody might fill out a transaction or, you know, deliver things to someone. So one of the things we're working on is anomaly detection with Gen AI, so that we can look at transaction so you think about, you know, a statement coming in from your bank versus the transactions that you have in your system and reconciling around them. So that is a mundane task. It has to be done. I really hate doing that. Right? And so using Gen AI to help Hey, these are the anomalies you really need to look at, versus these are the things that are you know, probably okay, and then giving them the option to then accept reject. So things like that make the experience much better for the person doing the reconciliation rather than, you know, three way match or other ways that we had to do it that were very narrowcast. Raise it

Michael Fauscette

Yeah, no, that's that’d be a huge improvement. I can see that.

Warren

so I think, you know, having that enterprise ever present assist in making the whole concept of Clippy actually work. For those of us who remember, what did I

Michael Fauscette

honestly, I was so surprised, because I thought Microsoft, when they brought out copilot, I thought, why didn't they just call it Clippy,

Warren

bring it back, that'd be but yeah, so infusing it in that way in a couple different ways. So that it's, it's sometimes it's in the flow have a natural way that somebody would do a job with the project-based ERP. And sometimes it's just the ever present, hey, I want to pull something down and ask questions and give me some help on this particular area. So I think that's going to radically change it. The other thing, which I think is really interesting, right? So ChatGPT, for, you know, the capability for voice and the capability for graphics. So we talked about this a little bit, we're starting to figure out, you know, how can this be more democratized? In terms of BI, right? So we use IBM Watson today to help somebody can put in an agile questions, give me the profitability of Project X, you know, over this period of time, and it'll suggest a bar chart or, you know, a pie chart or whatever, then it'll generate that and give you insight into the data and say, Well, this is associated with this product manager, and so forth, you know, so forth and so on. Well, you know, now, you know, with ChatGPT-4, you can have that conversation over your phone to the ERP and have returned back the information you want. And not only that, you can then it'll understand the context of that continued conversation. So now drilling down to into this week, drill down into this location drill into this time sheet, or whatever you want to do to refine. So it makes that, you know, having an actual conversation, it just changes the dynamic of what user experience means when interacting with enterprise software.

Michael Fauscette

Yeah, I mean, it sounds like we're moving towards a natural way to use systems versus, you know, our evolution through forms and spread. Exactly. And all those things that we used to do that weren't that much fun. But were necessary, because we didn't have any other way to make programs

Warren

yeah, and it's like, I'll tell you this, I mean, so all our dashboards that we ever created for all time are always configurable, because the one that we created the box, while we do a lot of listening, and a lot of work on them, you know, most of the time people need to there's that one thing, right to know, yeah, so imagine more of a blank screen where it's like, well, what do you want a dashboard on today. And you just have a quick conversation with the ERP, and then boom, you've got, you know, interactive graphics that you can then dive into. Right? I mean, this is kind of where it's going, where it's almost like a UI less user experience that's generated on the fly, you know, I don't know is that one year is that five years out, it's in the future. And we also talk about conversational UI a lot in terms of meeting people in Outlook and teams and slack and places where they're working. And we have an adaptive card concept with Microsoft, we're using to do that. So we can send a timesheet via email, you know, or we have notifications that can pop up in team. So part of this would be making those that that UI less kind of UX available in that conversational UI. So the ERP might be in the background. But you know, they're interacting it in their normal course, they might eventually think that that's just part of teams. So that's the plugin for Deltek in teams. So I think that there's, you know, we got to think about a future like that. Yeah. For 90% of the people in the business. You know, so it's, it's, it's really interesting.

Michael Fauscette

Well, I wrote about that not that long ago. Salesforce and slack, right? Because Slack could easily be a UI for most of the people in your business or teams, same concept. Because I don't I live in salespeople they don't they don't live in, in the CRM they live in, you know, Slack or chat or Teams or whatever it might be right, because it's all conversational. So if you can do things inside of that, that's really powerful. That's right. Yeah. Submit

Warren

your timesheet, submit your expense report, find out the past history of this client. Like, I mean, you know, what's what, how did they pay within their terms? Right? Do we have to go do an extra financial review on that? Like, there's all kinds of things like that, that if you surface that assistant with context and knowledge of your business, our tools in those conversational UIs? That's kind of the whole?

Michael Fauscette

I mean, to me, that's really exciting. Yeah, I think that is that that will change a lot of people's lives in the way they interact with. And it certainly makes it easier to learn how to interact with the system when you're just doing it naturally. Yeah, yeah. Well, I hate to end the conversation, because frankly, I could keep going. But I know that my audience probably won't keep going. So, you know, that's all the time we have today. And I really appreciate you joining me. It's always fun to talk. And I think this is an exciting topic and a lot of things to a lot of directions to go. Before I let you go, though, there was the one thing that I always ask at the end of the show, do you have somebody you could recommend that mentor and author a podcast or whatever, that you have had an impact and influence on you that you could suggest for the audience?

Warren

That's interesting, because I don't really subscribe to very many podcasts. Like I usually when I'm when I'm listening to a podcast, it's either one of yours. You know, it's more recreational for me so 

Michael Fauscette

or a book that you think would be useful?

Warren

Yeah, I think one of the books that we've we kind of read and talked about recently, within kind of our leadership team is a book called Smart Brevity. And I think that it helps us kind of understand how to get to the point. Yeah, we live in a world where, you know, we've got a lot of information, we got a lot of things to talk about, there's a lot of changes, you can go down. And those are, there's a place for that. Oftentimes, when you're kind of in a fast paced business environment, you know, you need to get jobs done. And so having, you know, smart brevity, having the ability to really focus the conversation, or what's the most important thing to get done now helps in prioritization of thought, as well as execution. So I think we've really tried to embrace that.

Michael Fauscette

Yeah, smart brevity. That's great. I like the idea. And I mean, it sort of reminds me of the stand ups that I used to do back in the day when I ran projects, right, make everybody get up for 15 minutes and have a standing meeting literally. And, and things went really quick, because nobody wanted to stand for more than 15 minutes, right? Yeah, that's good. Well, thanks for it again, I really appreciate it. And you know, I've had a really interesting good time here. As always, when I get to come to project con now, it used to be called something else, but we don't remember that name. And yeah, I assumed. Yeah, that's good. Now, but I really appreciate it. And I'm sure that the audience got a lot out of the conversation. Thank you.

Warren

Well, thank you, Mike. I really appreciate it. Yep, it's a pleasure.

Michael Fauscette

And that's the show for this week. Thank you all for joining. Remember to hit that subscribe button. And for more on AI and other software, research reports and posts, check out the area on research.com/blog and slash research reports. And don't forget to join us next week. I'm Michael Fauscette. And this is the disambiguation podcast