Greg: So, I'll jump in on that one. I would make the relation back to what we've been doing in open-source for decades now. There are companies that stand behind products that will give you guarantees, assurances, and take the liability on, in many cases, for those products to at least do what those products say that they're going to do to a reasonable extent. And, when organizations are buying part of their software infrastructure, that's part of what they're buying; even if it is 100% based on open-source software, you're buying that, that commercial relationship; in a more direct sense, you're buying the throat to choke, and you are mitigating your risks and compliance needs by doing so. Now, if a company wishes to go direct to open-source and download an open-source operating system or build their own kernel or, you know, run without a support contract, without a commercial agreement. What they're basically ending up with is, well, they're taking on the liability themselves. Now, with open-source, it's not an unreasonable thing in many cases to take on that liability yourself because you can see what's in it; you know what that software is going to do. And you can audit if you're so inclined and/or able; you can audit that code and make sure that it's at least as good as, as you know it, it is supposed to be, with what we're seeing right now with open weights.
Greg: And again, I want to be very clear: I very much support open weights, and I think it is a fantastic progression towards open-source for this. So I really support open weights. But if you download an open-weight model from the Internet from Hugging Face and you go and run that in production, who owns the liability for that? It's 100% on the person or the company who has downloaded and is running that. And it's a little scary to me when it's a black box, and you don't know what's actually in it. Now, is there going to be compliance and risk mitigation for that? Probably. But as of right now, it does scare me. So if I have the option and the best-case scenario, there are companies doing this. One such company is Arcee AI, and Arcee has an open-weight model; an American company with open-weight models. In fact, I think they're the leading open-weight model in the US. You can go to Hugging Face and download it for free, or you can work with them as a company. And you can now have the assurances and mitigations for controls and liability and whatnot that you'd have with any open-source provider of software. So what would be my recommendation? Well, as a company, I'd want another company to take on that liability and risk as much as possible.
Addison: I think, Kevin, just the fact that you're asking the question underlines the point that there's going to be segments for all of these things, just like there has been ever since Linux and the early debates in the late 1990s about whether to trust my enterprise on open-source software. I think we found over time that a lot of people did, or what I want is something that's open-source, but I want a supported version of the open-source, right? But either way, taking the boom in AI now and all of the dialogue we're already having around open weights and saying, what about something more like open-source with an open corpus and a community-driven effort? I mean, I think this is something that really is going to take off. OpenWaldo is kind of an early mover here; first to market has the opportunity. Is it going to be OpenWALDO in the future? I guess that's for CIQ and to try to drive their product. But we haven't seen this before. And I think for those who are worried about, you know, us or Western technology and what's going on with the Chinese, I mean, you know, imagine if this were a Chinese product right now, would there be concern about what the control around it is and who's auditing it? I think the idea that you have an open community, open corpus, open weight, open-source, like, here's something I can understand and build AI on it. Companies will evaluate how much control, how much support they want over time. But an open community effort around AI, I think we've seen enough in the past to know that this is an idea that ought to take off.
Greg: You know, we've been we introduced OpenWALDO; we talked about it. And you just kind of gave this string of openness that I think is really important. And it's funny because I gloss over this so often because, like, maybe I'm so close to it, but the name itself is actually really cute. Not because Waldo is cute. It's because, well, maybe, but it's because WALDO stands for open weights, open artifacts, open licenses, open data, and open origins. And when you put that all together, that's basically what we're ending up with in terms of what it means to be a truly open-source model. And when we're talking about the corpus, we're talking about the data and the origins of that model. So, anyway, just wanted to put that out there that all those things together is really where OpenWaldo got its name.
Kevin: And it's a really interesting technology. Addison and I obviously have been able to speak with you about it. I'm glad we're able to share. Just a reminder to anybody watching, please send us some questions if you,, if you're interested. I've, of course, got a litany of them, but we'd love to hear from you. You know, we're just talking about transparency, and I'm interested in something: making the training data inspectable is one thing, but having the expertise to evaluate it is another. How does an enterprise turn that transparency into confidence? And who should do the checking?
Greg: I'll take another stab at this. Not to dominate, I'll go back to how open-source works today. We, holistically, as a community of enterprises and individuals and people, we all rely on open-source software every single day, whether you know it or not. And there are, I mean, I can't even tell you how many millions and billions of lines of software that we are relying on that is open-source. We have several ways of trusting that because there are examples of malicious actors, bad actors trying to infiltrate open-source communities to try to get malicious code into the open-source ecosystem. And this has happened 100%. And we've learned lessons as a result of that. Are the lessons foolproof? No, they're not. But we're continuously learning. And if you look at the amount of contributions that we get, it's an incredibly small amount of contributions that have actually turned out to be malicious, and what we've been able to do is leverage things like the developer certificate of origin (DCO), leveraging accountability, understanding who the contributors are, and being able to monitor those contributions with reviews, code reviews, data reviews, etc., before we allow anything into the corpus, whether it's open-source software or the OpenWaldo corpus.
Greg: That creates trust: the fact that you can, even though many people might not be able to either technically, or it may just be too much data. The Linux kernel has tens of millions of, I think, tens of millions of lines. It's definitely in the millions. I don't know if it's tens of millions. I need to go back and check that now. But there's so much code in there, and it requires such specific expertise that even many software developers who are very senior in their roles cannot audit the Linux kernel. They can't audit the core toolchains of the C library and the compilers and whatnot within every single operating system that we're using that's based on 0open-source. Can't do it, but specifically knowing that others can is where you start to develop confidence and trust. And when we're talking about data, like what's in OpenWaldo, we're not talking about the goalposts being a little bit shifted because we're not talking about software and code that people can, anybody can review; in this case, we're actually just talking about such a mass quantity of data. It's hard to review. We're talking about hundreds of billions and trillions of tokens going into an AI corpus.
Greg: For a small model, you could be talking about hundreds of millions. For a large model, you're talking about trillions, and that's a lot of data to review. But if you know what, if you have metadata on all of this corpus and, you know, each shard, what that shard is responsible for bringing in, it gives you the ability to start pinpointing. And it starts saying, well, here's the whole corpus, but here's what I'm seeing in the AI. I want to actually fine-tune and get into a very specific section of that corpus. And even though it's still going to be very, very large, it's now a reasonable task. And inside each shard of data, which is contained within a parquet file, you have different rows. You can think of it almost like a database in a file. And you have these different rows inside of it. And each row is queryable and gives you the ability to say, here's even more metadata, more specific metadata about the content that it includes. And so you can start really narrowing down and kind of following through the hierarchy of the data to get to the point where, yeah, you can actually, you can do this.
Addison: Yeah, I think you really highlighted important things there, Greg, with regard to how these things get adopted. And I would start by looking at what processes we already have in place, again, not reinventing things, but then we'll find that processes have to evolve somehow. My, I like that you were talking about software development in general. Vibe coding is one of these areas that's really taken off. And there's this idea of, look how much faster I can produce software. Well, but any professional software engineer will tell you that writing the code is one part of it. You know, have I checked it for dependencies against all the other modules? Is it well documented? How about security checking all that software? Right. Who's doing all that as you pile it on? Is it another AI that's doing the security check? What's that process like there? A lot goes into software engineering beyond just, " Hey, I produced all this code that needs to be checked. It needs to be, you know, integrated in certain ways. And there should be a process for that. Maybe the process needs to evolve because you're producing software so much faster. And I can extend that to scientific discovery and, you know, hey, I'm producing papers so much faster now because AI is helping with the research. Okay. What about peer review? Right? I've just put out 2000 papers. Who's doing the peer review on 2000 papers? Is peer review important? It's been a cornerstone of our scientific process.
Addison: Maybe you're saying we don't need it anymore. I think probably what you're saying is it needs to evolve. We need to figure out how peer review evolves to keep up. And, you know, ultimately, I think we learn a lot from our heritage in how we do science and engineering and how we've adapted to other technologies over time. I visited a company once that was starting to use modeling and simulation in its new product designs to help get to market faster and be more responsive. The products once designed still had to go through the same testing they would go through before. Like, all right, I'm going to put it in this stress or in this bender, and I'm going to press, you know, to make sure it still meets spec. But the process had to evolve in a way, because the old way that the physical guys were doing it was that they would press until they knew it was in spec and then stop. And the guy who was designing the modeling and simulation piece was begging them, please, I want you to keep pressing all the way to break, regardless of whether that's past inspection, because I would like to see whether the break matches what I predicted in the simulation? Right. Because I'm trying to build faith in the simulation. Yeah, I know it's in spec. Keep going.
Addison: I want that data. Right. And it was like this change in policy of how they want to do their testing, which implied some extra cost, but it was a worthwhile cost in order to get the gain. Now AI is going to be a little different, but it's kind of the same. What I want is to start with my existing processes and figure out how to adapt the policies to get the AI gain, but still keep confidence and, in fact, build confidence that the AI is taking me in the right direction. And again, having an auditable or open AI that I can look at and say, this is how we got to where we are. That's what we already went through with modeling and simulation. Right? How do we build the confidence that the simulation is good? It rhymes with the idea that how do I build confidence that the AI is good means evolving my process. So I feel confident in the peer review. I feel confident in the product test. I feel confident in the customer interaction. You know, if I have a customer service rep, I train them, and I monitor them to make sure that they're interacting with the customer the right way. Chatbot, same thing, right? I want to train them. I want to monitor them. But the process has to evolve to incorporate the AI if I'm going to really take advantage of it.
Kevin: Yeah. I'm glad you mentioned confidence, Addison, because we have another audience question that kind of dives into this. You know, this audience member seems to run into issues where they ask the question, " Where did that particular answer come from. Right? And they say that OpenWALDO seems to be a great solution for this. You know, Greg, how far can OpenWALDO take us toward that today? And, you know, maybe what's the difference between tracing a model's training data and tracing an individual response?
