From the found him department
Do you know the difference between closed-weight models, open-weight models, open-data, and open-source? More importantly, do you think it matters? In a recent webinar, the CEO of CIQ, Gregory Kurtzer, was joined by Addison Snell, CEO of Intersect360 Research, to discuss an emerging issue in the AI market: Unknown risk of model failure due to closed-data training. (e.g., not knowing where the model talking to your customers learned its manners, if at all.) The lack of open-data provenance in popular Foundation models may introduce risk for many organizations that are integrating AI into their business processes.
The OpenWALDO project from CIQ is designed to bring "open-data" to the AI training arena as a way to increase trust and reduce risk in the AI market and community. As the webinar transcript describes the project is designed to bring open-data, open-source software, community, and public governance to the mysterious practice of training AI models. Further background can be found in the HPCwire article entitled What is OpenWALDO and Why Did Greg Kurtzer Need to Create It?
The transcript is based on the video. A shorter version can be found on HPCwire.
Kevin: Welcome, everybody. And thanks for joining us today. We're going to be talking about OpenWaldo. And you know what it means to have greater visibility and control over the AI models that your organization uses. And, of course, today joining us is Gregory Kurtzer, founder and CEO of CIQ and founder of OpenWALDO. And we also have Addison Snell, CEO of Intersect360 Research. My name is Kevin Jackson, and I am an analyst here at Intersect360. Greg and Addison, it's great to have you here.
|
| Kevin Jackson is an analyst with Intersect360 Research and formerly the editor of AIwire |
Greg: It's good to be here. Thank you, Kevin.
Addison: I always have great conversations with Greg, so this is fun for me. Let's see how it goes today.
Greg: Absolutely. Great meeting. Interesting or weird.
Addison: You know, we'll see where we go.
Kevin: Exactly. Well, you know, we're going to explore what OpenWaldo does. You know why it matters for enterprises and what it could mean for how organizations evaluate and adopt AI going forward. So, you know, just to get us started, I'm going to put this question to both of you. But Greg, I'd like to hear your response first. If I'm running a model on my own infrastructure, I'm keeping my data private. Yeah, I might feel that I've handled it or I've checked the sovereignty box, but actually what am I still not controlling? And why does that matter?
|
| Greg Kurtzer, CEO of CIQ is the creator of OpenWALDO |
Greg: There are a lot of aspects about running your own model, trusting your own model, trusting the infrastructure that that model is running on. And how do you check that box on sovereignty? And it's even more than sovereignty. It's ownership, it's control, and it's visibility, knowing what it is doing. So a lot of organizations are pulling open-weight models and then running them locally. We don't know what a lot of these open-weight models have been trained on. We don't know what they're going to do in a variety of different situations. We don't know what the code they're going to be writing is, or what the interfaces and tooling they're going to be using are. We don't know what they're going to, what it's going to be doing with it. I mean, that's kind of, it's kind of one of the, I'll say, benefits of a non-deterministic system such as an LLM. But at the same time, even though it's non-deterministic, they definitely have kind of directions that they will pull toward based on the statistics of what they were trained on. So you need to know kind of what's inside your model. You need to have proper boundaries, guardrails, and sandboxing of your tooling, your agents, etc., and you need to know what everything is doing. So you need to have proper infrastructure visibility and monitoring.
|
| Addison Snell, CEO of Intersect360 Research |
Addison: Yes. I mean, that's the way we've seen it in our research: of course, people are organizations. Enterprises are already moving to adopt AI, as, you know, they want to adopt it. They want the productivity gains. They want the efficiencies. But what I like about what you said, Greg, about control, a lot of that really comes down to risk. And we've seen this with any new technology that gets adopted, whether it's, you know, high-performance computing, modeling and simulation, or whether it's cloud computing or whether it's the Internet, right? What are the risks that are involved? And it's not that I have to have no risk. I just want to know what the boundaries are on the risks. Am I protected on the downside, on cost? Am I protected on the downside on what catastrophe might happen, right? It's basically the way you put it. And the topic that's in the popular press right now is all around regulation, the regulatory environment. What we find in our surveys is that people feel like they're kind of okay with understanding what the different regulatory environments are, but they need to know that they're stable, right? And, you know, how do I control my intellectual property? How do I control any of my downsides? And that's what having my own model kind of feels like. It gives me more of a sense of control. I think, you know, regardless of where you stand on the regulatory debate right now, we need more regulation. We need less regulation. I think what a lot of organizations want is predictability. And I think what a lot of organizations want right now is, you know, well, I think the result is: can regulation even stay ahead of AI right now? If I'm going to sit back and wait for regulation to explain everything to me, I'm going to get too far behind. That's just part of the risk environment right now. So how do I go forward with a model, feel like I have some kind of semblance of control, and manage my risk while I chase the productivity gains?
Greg: You bring up such a good point. A few of them, but one of them that I want to touch on is the data. So for an AI model, whether you're running it on-prem or you're running a frontier model, you need to give it data in order to get a good response, right? So inside of your prompt or inside of your tooling, it has to have information about what you want to be asking it. And if you're asking it things about your core business, core products, trade secrets, things under NDA, like you have to include bits and pieces of that- if not the entire IT in its entirety in your prompt in the context, which means you very well might be leaking data. And this gets really bad when we start talking about agents and, even more than agents, all of the tools that the agents write that are scattered all over the infrastructure or even on the Internet that have access to your data. And then what's securing those tools, what's securing those applications, and then what's securing the agents to make sure that they are being maintained within their guardrails and that we're not doing something, you know, like hacking another site or hacking a foreign government.
Addison: Well, right. I mean, there are so many analogies with that, that I expect my AI to act within certain boundaries, just like anything else that I hire or employ. If I tell my personal assistant to go pick up my dry cleaning and my personal assistant gets there and finds the dry cleaner to be unexpectedly closed, I don't expect my personal assistant to smash the windows and break in in order to follow the directive to go pick up my dry cleaning, right? Without at least checking with me first. "Hey! It's closed. Dude, how bad do you need it? Do you want me to write?" That would be outside the normal realm of what I expect. Now, that's where we talk about guardrails and how the models behave. And I guess you can train a model to do whatever you want. Maybe you're someone who wants it to go break in, but that should be auditable in some way. Right? Did you give it that directive or not? And you know, when we design a car, we expect it to behave in certain ways. Which doesn't mean you can't intentionally cause damage with a car, but you want it to be built so that you don't accidentally cause damage with a car. I think that's the point.
Greg: That brings up a whole other can of worms, by the way, which is accountability and liability. And where does that exist? Kevin, I'm going to defer to you on that particular can of worms to keep us on track.
Kevin: It's a conversation we're all having right now. I think something that Addison says all the time, and I'll fill in here, is that if you use a new technology to commit fraud, it's not that we need to write a new law. Fraud is already illegal, right? It's something that Addison brings up quite often -- we already have laws in place that perhaps we could be using and strengthening, but it's a whole debate that could suck up this whole hour if we wanted to.
Addison: Well, I think that's not the point. Right. But when it comes to control and auditability, there's been a lot of discussion of things like open weights. But then what's the difference between open weight and open-source, and what do you get out of it?
Kevin: That's where I was driving toward. I'm seeing a lot of people use open weight and open-source interchangeably, and they shouldn't. Greg, can you tell me what the difference is? And why does that difference matter to a company, or a business?
Greg: Well, the first thing that I want to mention is it's actually really confusing. And it's confusing because, if you think, I mean, several different aspects, if you think of open-source software, the open-source premise was kind of written around the notion of open-source software. So we kind of have a really good, firm understanding of what that means when we're applying it to software. You basically think about it as you have the source of something, whether that's source code or, you know, it's been debated if data is considered the source in programs, but you have your source, and then you have your output. And if your source is open-source, the output is open-source. Kind of, that's the definition of open-source, right? With weights, AI weights, it's different, and it's different because we have a bunch of open-source tooling that we're using to build kind of the foundation and architecture of our models. And then we take the input data, the source, which is not open, and use the open-source tools to create binary weights. And these binary weights then are distributed in some cases under an open-source license. So that means you can do whatever you want with the binary weights, but you don't know what went into those binary weights. The source for it is not available. So I'm a purist when it comes to open-source, in the most fundamental way.
Greg: And as a result of it, I think of it as any source that you're using to create an artifact needs to be open for that artifact to be called open-source. And so for an AI, that means that your training data- that's the corpus of data you're using. It is the composing and the build tools that you're using to create your model. And if you just have the open weights, well, open weights are great. Please don't get me wrong on this. I love the fact that we have open weights. A lot of the complaints that we have around open weights, a lot of the concerns, the transparency that you were mentioning. Well, the binary weights are kind of like a black box. You can't ever go from those binary weights. And then understand how they are produced. And so, as a result, we need to kind of be thinking about this from the perspective of what's inside those, and if we want to mitigate the concerns of what it means to be an open model. Well, we've got to go back to the source, and we've got to have trust in everything that was used to create this model. And if we have that, then we can fundamentally call ourselves an open-source model, but without that, open-weight models are not bad, but they're definitely not open-source.
Addison: Yeah. I think what you're getting at, Greg, is it hits right at that question that we had about mitigating risk because the large organizations or even small organizations we talk to generally have processes in place for when something goes wrong. They want to trace back to how it happened, whether it's a product defect or a failure in policy somehow. And, you know, there was some customer harm. You want to know not just how to close this gap, but what caused that to happen. What allowed that to happen? And with AI and some of these open-weight models, you don't know how they were trained. There's that worry that I'm going to trace it back and I'm going to get back 1 or 2 steps, and the answer is going to be, well, the AI did it, and it stops there. Well, why did the AI do that? And I'd kind of like to know why the AI did what it did, so that I know that I'm not going to repeat the same mistake, which isn't necessarily really fear-driven. It's just kind of best practice in any organization to be able to audit why we're doing the things that we're doing, particularly in areas that are considered to be strategically important. When we polled organizations, enterprises, medium and up to large enterprises on where they've adopted AI, we found it really prevalent in areas like coding and office productivity and, you know, that kind of thing. But it's least prevalent in areas having to do with product development and engineering and R&D, which is supposed to be the promise of AI, right? It's going to help us predict the weather and find new products and cure diseases and all these areas that we're used to associating with high-performance computing and scientific discovery. That's where AI is least deployed right now. It's where the biggest gap is. And I think this notion of control and auditability is one of those pieces that's been missing.
Greg: You know, there are multiple science fiction books and movies and whatnot written about, like, we have, we have put some of our content, some of our, what we're, what we're excited about and, and humanity. And we've sent it up into the stars, and aliens have received that. And then they have assumed that this is what our civilization is like. I don't know why, but the movie Pixels, I think that's the name of it, with Adam Sandler, is popping up in my head right now, where they did that with some video games, and then the aliens kind of figured, well, this is how humans operate. This is how they deal. And so they had to go beat a bunch of real-life kind of video game stuff and whatnot. Metaphorically, it kind of reminds me of what we're doing with AI a little bit. We're taking all this training data, which is a gigantic corpus of knowledge within humanity. And in most cases, we're just trying to get the quantity, not the quality. And in that quantity, well, you know, there's quite a bit of science fiction about how AI has completely destroyed humanity and how we are at war with AI. Like just as a, like if you, if you know how, how training a model works, you'll recognize immediately, like there's tons of weights and how everything statistically kind of maps parameters to tokens and, and comes up with kind of the next thing that it's going to come up with and what it's thinking about, etc. thinking about,, etc., but if you are training it on things that, kind of declare war between AI and humanity, I mean, don't be surprised if you ask the AI then like should I be concerned that you're going to kill us all? And the AI is like going back to the Terminator series, going, yeah.
Addison: I mean, look, I'm not a doomer about this. I think there will be things that happen that are bad. Just like with any new technology, if you invent a car, you're going to have car crashes. Overall, the car accelerates a lot of things and does a lot of good for humanity. Right? And you could say that about any new technology. Overall, I think organizations do want to chase the productivity gains here. People are adopting it. The question I keep coming back to is then: how do I mitigate the risk? How do I control or understand the downside? And in order to activate the upside, because AI should be accelerating discovery in all of these ways, it just feels like there's been a piece missing to help get to that level of control. That gives me confidence to move it into those areas of forward-leaning R&D and scientific discovery.
Kevin: Absolutely. And, let's talk about that missing piece. Let's make it a little bit more concrete with this discussion about OpenWALDO. What can an organization do with OpenWaldo today? And what decision would it help that organization make? How is this going to further that organization's mission?
