Addison: So start by saying what OpenWaldo is. We just dropped this term in here.
Kevin: Please, Greg, why don't you give us a good, deep discussion of the entirety of this?
Greg: Yeah, absolutely. So kind of going back to the open-source versus the open weights discussion, there isn't really a way for community members to kind of join together and band together to start creating an AI from scratch. This would involve, as I mentioned before, the AI corpus or training material. This involves data that you're going to be potentially post-training on and teaching it, for example, how to interact with an API, how to interact with a data structure, how to actually respond properly. So all of these things have to go into the training data, into the corpus. And then you have your tooling, and then all the tooling kind of brings this whole thing together. And then you, of course, need a community around this to be able to say, well, you know, I'm going to bring in this piece of data, or I'm going to work on this tooling, or I'm going to be reviewing this corpus, or I'm going to be looking at the different types of compose files that we have for creating different types of AI models. And I want to ensure, and again, I use this metaphor, tongue-in-cheek humorously, but just to make sure there are no Terminator movie scripts in it or books and whatnot on the AI being not aligned with human humanity and laws, right? So let's make sure that we are putting laws into this. Let's make sure that we are putting appropriate ethics into this and having that transparency and visibility. Addison, what you're mentioning is super important because if I have the ability to choose between two types of models, and I know what went into both of them, I can make an educated decision on which model I want to run, and I can make that decision as a company now, managing some of my risks and compliance.
Addison: I think that's exactly the point that we were building up to. You've created an AI bill of materials that I can investigate, and that the community, as it will, if you think about the open-source community and this becomes an AI community that's contributing to this corpus, if you've got a question about it, take a look. Here it is under the covers. And boy, when I think back to how open-source software really took off, you know, going back to the advent of Linux in the mid to late 1990s, what happened was you had this terrific open-source community of programmers that were then all working on something together. And it didn't take very long before that raced to the forefront; this is what we're going to build our organization on.
Greg: Yeah, actually, going back to origins is a really great way of looking at this. So, the entire GNU movement started with Stallman, who wanted to change the firmware in a printer because he was having issues with how the printer was behaving. And, I believe- and I was not there, so I can't say for sure- but I believe, as I understand it, that it wasn't a difficult fix. And he even tried working with the companies to try to fix how the printer was behaving and what the printer was doing. But because the printer was this black box and you didn't, you couldn't figure out what was inside. He had a really hard time doing that until he was able to get the source. And that's what really started open-source. The entire free software and open-source movement is that we all have the ability to look at the source for whatever artifacts we're talking about, right? Whether that source is software. And again, I'm a purist. So whether that source is software and code or whether that's whatever went into the artifact itself, if you have that ability to see into it, you now have a level of trust that doesn't require faith. It requires it because you could see it.
Kevin: Absolutely. And this kind of brings me to something I wanted to discuss here. Most companies, most enterprises, most businesses, they're not training foundation models themselves, right? And if I'm using or maybe fine-tuning someone else's model, how can OpenWaldo help me? And what would it take to use it in order to benefit my organization?
Greg: Training a model, I want to say, is a very difficult thing to do. There are a lot of things that can go wrong, and it is exceedingly expensive to have the appropriate hardware to train a model. And an interesting aspect about it is, and I'm just going to kind of talk about this very superficially, but training a model and running your model are two different things, right? So you have your training side, which is where you're taking all that input data, and you're kind of managing the orchestration of how you're building that model, and you're running it on a very large, very expensive resource, usually with many, many GPUs. On the light side, this is probably tens of millions of dollars to build one of these systems. The large frontier AI labs have ridiculously large investments in their training infrastructure. And I want to be very clear: you could be using that training, that infrastructure, and after months of training, you realize that the model went askew, or your training plan went askew, and you're either starting over or starting over from the last known good checkpoint. And it's an expensive, very frustrating, very high-risk kind of process to train a model. And a lot of times you end up with a model that comes out the other side.
Greg: That is not what you, not what you expected and not meeting the benchmark of what you needed to do. We've all, I think, seen this. If you've been using AI for any amount of time, it's like a newer release of a model within a lineage or a series feels worse than the previous version, or it's doing something kind of different than you might have been expecting. And that's all because the training process is very hard. Once you've trained it, you now have these binary files, these weights, and whether you're talking open weight, open-source, or even frontier labs, right? You've got your model, and that model is fixed. That model is not going to change anymore as you start doing what's called inferencing. So you've got your training on one side, you've got your inferencing on the other, and you're taking that fixed model, that fixed brain, in a matter of speaking, and you're going to now go and allow people to use it. Now, the interesting thing is that a lot of discussions have come up on this, which are very tangential to this, but just to quickly touch on it. The reason why AI models appear to learn. They appear to grow, and they appear to understand what you're doing, is because of the context window.
Greg: So as you're doing AI, as you're, as you're querying the AI, you have this data structure that the agents and whatnot can pass to it. The AI model does not change for you, right? So it is a fixed model. So when we're talking about AI, there's the training side, which, as of right now, even with open-weight models, is a black box. We don't know what's happening there. The inferencing side is where you're running that model. And we're not going to unpack that a whole lot for this webinar, but that'd be a fun next webinar, maybe. But on the training side, this is a process which we've actually been doing. And Addison alluded to this already a couple of times: to HPC, the training side of AI is an HPC problem. And we use HPC tooling in order to build those models, even down to the type of scheduler and architecture that we're using to build those systems. This is 30-plus years of knowledge of how to build the infrastructure to scale up and train. AI is all kind of built within that. Now that I've said all that and laid some groundwork and talked way too much. Kevin, what the heck was your question again?
Kevin: It doesn't matter. I was saying, just like most enterprises, they're not training their own. And if I'm fine-tuning someone else's model, how can OpenWaldo help me?
Addison: Yes, and the question was around how OpenWaldo helps? And I think you laid the groundwork for that, Greg, because, you know, I don't really want to train a whole foundation model myself. Let's take something simple, like a chatbot. I want to use that for my business, whether it's internal or for my customers. I don't need to retrain basic human vocabulary from scratch. I do have to give it some knowledge about my products and my business, including my own internal jargon that's important to me, that might not be in the general corpus, right? So there is some training, but I'd like to start from an auditable, understood foundation because I don't want to find, you know, later that my chatbot has been, you know, picking up a habit of sexually harassing the people who interact with the chatbot because it learned from the Internet. Right. That would be bad. I don't want that to happen. I don't want to train that part. I'd like to have something that comes in starting out good, and then I'm going to specialize it.
Greg: Actually, that brings up. And now I'm doing Kevin's job. Sorry, Kevin. There's a question that I just saw come through. Would you mind hitting on that question?
Kevin: Yeah, absolutely. So, you know, the alignment issue that he's discussing here- he wants to know who determines the alignment. You know, he says in a somewhat joking fashion, "Your alignment is fine as long as I agree with it, because I'm almost never wrong."
Addison: I mean, this comes down to training again, right? We use the term training. Like you talk about training a dog. And the metaphor I use sometimes is if I train my dog to bite people and it does, as far as the dog is concerned, that was the right behavior. Is it the right behavior? I don't know; that kind of depends. Is it a police dog? Was it stopping a crime? Or did it bite my son's friend at a birthday party? Right. But the dog doesn't know. Right? The dog does what it's trained to do, which, you know, is a long way around of saying it's a new technology and people will use the new technology. Like they're going to use it. And if there's bad actors, there's going to be bad actors. If I invent a car, someone will commit a crime using a car. If I invent a computer, someone will commit a crime using a computer. I don't think you can regulate or wish away bad actors. I do think we should talk about what the remedies are. And, you know, in terms of where's the kill switch? How do I prevent a bad actor from harming me too much? That's kind of beyond the scope of this conversation. I think what this conversation is more focused on is: how do I keep my tool from becoming a bad actor by accident, right? And that's where OpenWaldo has a big role.
Greg: Yes.
Addison: I was trying to set you up there, Greg. All you did was agree with me. Sure. Fine. I win.
Greg: That was perfect, though. I totally agree.
Kevin: I know. It's a great discussion. It's something that I'm sure we'll touch back on. But, you know, just to keep this rolling, we've kind of been circling around this, but I'd like to dive into it specifically. Say I'm a decision-maker at my company, and I'm deciding on how I want to move forward with the project. How should I decide whether to buy a supported AI product or build on open-source? What should drive that choice?
