Unqork Chief Technology & AI Officer (and inventor of IBM Watson) Dr. David Ferrucci joined host Dave Vellante on theCUBE at the NYSE Wired Studio to explain why raw LLMs aren’t enough for mission-critical enterprise systems.
As Dave explains, generative AI has mastered natural language, but AI’s “best guess” can quietly inject unacceptable risk into core enterprise functions.
Key takeaways from their conversation:
🔹 The Limits of Probability: LLMs are incredible at extracting knowledge conversationally, but they are designed to make decisions probabilistically. For transactional or regulated systems, a small AI hallucination can have catastrophic effects.
🔹 The “LLM Sandwich”: An AI enterprise development platform requires a hybrid architecture, with an LLM as the fluent natural language interface and a formal deterministic engine to strictly govern outputs.
🔹 Knowing When Determinism is Non-Negotiable: Enterprises can’t just unleash probabilistic agents. They need to understand when AI must translate human intent into governed, auditable, and 100% predictable software architecture.
Full Transcript
John Furrier: Palo Alto Studio Connection, Silicon Valley and Wall Street. I’m John Furrier, co-hosting here with Dave Vellante.
David Ferrucci: Hi, everybody.
Dave Vellante: I’m Dave Vellante. Welcome back to theCUBE’s NYSE Wired studio, high above the Options Exchange. Dr. David Ferrucci is an important figure in modern AI and is widely recognized as the inventor of IBM Watson. He initiated and led the team that achieved that amazing landmark Jeopardy! victory in 2011. Later, he founded Elemental Cognition, where he focused on combining large language models with formal reasoning agents to solve complex problems with transparency and rigor.
Today, he is the Chief AI Officer at Unqork, responsible for the strategic application and integration of reliable AI across that platform. He also contributes to the AI research agenda at the Center for Global Enterprise’s Institute for Advanced Enterprise AI.
What makes this conversation timely for our audience is Dave’s view that LLMs alone are not sufficient for complex enterprise reasoning. Businesses increasingly expect AI systems to solve problems that require precision. To do that, you need formal logic and accountability—a bar that LLMs alone cannot meet strictly based on how they work. He has coined the term the “LLM sandwich,” which we’re going to discuss today. The idea is to use large language models as a fluent natural language interface to formal systems capable of reasoning. In other words, LLMs become the human interface, but deterministic reasoning engines provide the accuracy. David, welcome to theCUBE. Thanks so much for coming on.
David Ferrucci: Thank you.
Dave Vellante: I hear all the time that we’re in the early innings of AI. When you hear that, what do you think? You’ve been at this for many decades.
David Ferrucci: Exactly. When I was trained in AI, people talked about John McCarthy coining the term in 1954. AI has been in the lab for decades, and there have been an enormous number of advances over the years. But once AI started tackling language as fluently and capably as we’ve seen with large language models and GPT, it became a social phenomenon. Everything just took off and became a common household word. Everyone is using it—not just in their personal and social lives, but it is becoming a huge force in industry because of how much it can accelerate the work we do.
Dave Vellante: When you saw something like ELIZA back in the day, did you see it as a harbinger?
David Ferrucci: For the audience, do you know what ELIZA was?
Dave Vellante: ELIZA was the original AI chatbot. At the time, people felt we could help those suffering from depression by providing a companion. It was groundbreaking and ran on IBM mainframes. But did you see it as a harbinger for the ChatGPT moment, especially since we went through two “AI winters” after that?
David Ferrucci: There was always an expectation that computers would ultimately tackle this problem, that we could get them to treat language as fluently as we do. In many ways, that was a measure of intelligence. Alan Turing set up the Turing Test: put a human and a computer behind a curtain, speak to them, and see if you can determine which is which. For a very long time, nobody thought this would ever be achieved because of the struggle we had in dealing with language computationally. Now, we are in a situation where I can always tell if it’s a human or a computer: the computer is the one that uses language more fluidly.
Dave Vellante: Right, it writes really well.
David Ferrucci: We are completely past that benchmark.
Dave Vellante: How should the average person think about AI when you hear frontier model companies saying, “Slow down, we need regulation, there’s a chance it could wipe out humanity,” while on the other end of the spectrum, you have people like Jensen Huang saying there’s zero percent chance? Does anybody know for sure?
David Ferrucci: I don’t think anybody actually knows. There are a lot of different incentives to exaggerate some of these claims for a variety of reasons.
Dave Vellante: Good marketing. There’s no such thing as bad press.
David Ferrucci: It is good marketing, in a way. On one hand, we’re being told to use them aggressively. On the other hand, we’re being warned about how much damage they can do. I believe we will continue to learn how to control them. Underneath the hood, they’re not as much of a mystery as most people think.
The technology will continue to get easily reused and replicated, and the cost of AI will drop. It’s in everyone’s interest to “open the box” and make sure the world at large can leverage these tools. There is always risk, but we are going to learn how to manage it. The same technology that generates the risk can help us guard against it.
To me, the larger risk is that we create monopolies, or that AI becomes centrally controlled and managed. In my opinion, it really has to remain open. It has to be advanced in ways that are beneficial for humanity.
Dave Vellante: Don’t you think the open-source, open-weight movement is heading in that direction? I was talking to George Kurtz the other day on theCUBE, and he’s very confident that we can use AI as a defense mechanism.
David Ferrucci: Correct. In fact, it’s our only chance. These models are incredibly powerful in the types of inferences they can make and the enormous amount of space they can navigate quickly. Because of how much data is compressed within them, they can generate alternatives very fast. We would be defenseless if we couldn’t advance those models from a defensive perspective to combat malicious uses.
Dave Vellante: Opening them up is essential so you can use them offensively for red teaming, which exposes vulnerabilities you might not have seen before. That brings me to determinism. The discussion in the industry right now is that we have to bring the stochastic and the deterministic together. It’s the only way we will actually be able to deploy AI in the enterprise and trust autonomous agents.
David Ferrucci: It’s a journey, and it’s very ambiguous right now for a variety of reasons. Many people who are quickly becoming AI practitioners don’t necessarily have the background to understand the difference between a deterministic algorithm and a probabilistic algorithm.
It is becoming easier to create agents—programs that, just through prompting and probabilistic token generation, make decisions you might not have anticipated. Sometimes you’re happy with the result; other times, you aren’t. But those agents weren’t carefully planned and deterministically modeled, meaning we don’t know exactly how they will behave under specific inputs.
The classic investment in computing has always been about getting a precise, deterministic response. That is what we count on for regulatory and transactional systems. In any scenario with asymmetric risk—where a small mistake on one number can have catastrophic effects—we fully rely on determinism.
At the same time, it’s very hard to predict every possible input a human might type into a chatbot. Deterministically mapping out every one of those conversational pathways is overwhelming. This is where AI shines, provided you govern it. You must ensure that, ultimately, the system is not allowed to execute anything that isn’t deterministically validated.
Only hybrid architectures that combine probabilistic agents with deterministic algorithms will succeed in the enterprise space.
Dave Vellante: I want to get your opinion on tacit knowledge—what people often call “tribal knowledge.” I trust my Oracle or IBM databases to do the right thing because they are deterministic. But business involves so much human interaction and decision-making. Right now, it seems like AI is very good at mundane, data-oriented tasks, but perhaps not as great at capturing tacit knowledge, learning from human reasoning traces, and then acting deterministically. How do we get there? Am I off base?
David Ferrucci: I would argue that AI is better than BI today. Large language models are better than any past technology at extracting tribal knowledge. Through dialogue, or by munging through huge amounts of corporate data, LLMs are excellent at understanding intent and retrieving past information. The real question is: what happens next?
Turning that understanding into reliable, deterministic processing is the critical next step. I tell people that half the time you want to instruct an agent to do something, you should actually be using AI to help you write the code that ensures the action is executed deterministically. You need to know when a generative AI solution is beneficial versus when you need strict code.
Having platforms and architectures that can advise on when to use probabilistic agents versus when to generate testable, deterministic code is becoming incredibly valuable. We shouldn’t just look at AI as a way to fire off autonomous agents. We should treat them as architectural systems that help us apply the right technology to the right place.
Dave Vellante: I think you’re describing the “LLM sandwich.” Can you explain what that is and what you’re building?
David Ferrucci: The “LLM sandwich” is the concept that at the center of any hybrid architecture, you must have a deterministic process. You need a formal inference mechanism that can compute answers, be traced, understood, and mapped onto a known algorithmic problem-solving approach.
However, capturing requirements and tacit knowledge usually happens in natural language. The translation from natural language to formal logic is what matters. Can I take your requirements through a conversation and formalize them in a way that guarantees results? I don’t want to just take your natural language and feed it into another probabilistic language processing machine. I want to put it into a deterministic machine that follows a process I can count on, validate, and ensure delivers the exact business outcomes you want.
Dave Vellante: And Unqork is building that machine.
David Ferrucci: We are building that machine. Doing that translation well, with an architected, auditable outcome, is ultimately what matters. LLMs play a huge role in acquiring requirements, teasing them out, ensuring they are complete and consistent, getting human agreement, and formulating the output in a consumable way. But in the middle of that sandwich, you have to be absolutely certain you are following a traceable, auditable, and deterministic approach.
Dave Vellante: Last question. What do you want to be able to say 12 to 18 months from now that you’re not able to say today?
David Ferrucci: I want to be able to say that we can trust the software we are writing. We come from a world where you wrote code, tested it, and could be 99.9% sure it was going to give you the expected answer. We are rapidly moving into a world where that is no longer the case. A couple of years ago, I was working with a major airline to develop a chatbot based on a deterministic process, but with an LLM front end. Their corporate team insisted that every single phrase the chatbot outputted had to be pre-approved before deployment.
Dave Vellante: It sounds like we might be going in the wrong direction right now.
David Ferrucci: I don’t think we are going in the wrong direction. I just don’t think we fully understand the speed at which we’re moving and what the long-term implications are yet.
Dave Vellante: Dave, thanks so much for coming on. It was a pleasure to meet you.
David Ferrucci: Thank you.
Dave Vellante: Thank you for watching. This is Dave Vellante for theCUBE’s NYSE Wired coverage. We’ll be right back.