Val Bercovici, Dyrektor ds. sztucznej inteligencji w WEKA – Seria wywiadów

Val Bercovici, Chief AI Officer w WEKA, jest dyrektorem ds. sztucznej inteligencji i infrastruktury danych, skoncentrowanym na rozwijaniu technologii leżących u podstaw sztucznej inteligencji nowej generacji. Od momentu dołączenia do WEKA jako Chief AI Officer w styczniu 2025 r., skupia się na budowie infrastruktury agentów AI, przyspieszaniu obciążeń treningowych i inferencyjnych oraz poprawie ekonomiki obliczeń AI. Oprócz roli w WEKA, Bercovici pełni funkcję doradcy AI w Home Dock, strategicznego doradcy w FermiHDI i The Hive oraz przewodniczącego PencilDATA, gdzie jego praca obejmuje AI, cyberbezpieczeństwo, blockchain, przetwarzanie w chmurze i infrastrukturę danych. Jego kariera odzwierciedla długotrwałe skupienie na opracowywaniu i doradzaniu w zakresie nowych technologii zaprojektowanych do wspierania coraz bardziej danych intensywnych systemów AI.

WEKA jest firmą zajmującą się natywną dla AI infrastrukturą danych, budującą platformę definiowaną programowo, przeznaczoną do wymagających potrzeb danych sztucznej inteligencji, uczenia maszynowego, obliczeń wysokiej wydajności i innych przyspieszonych obciążeń. Platforma WEKA Data Platform zapewnia organizacjom jednolitą architekturę, zdolną do działania w środowiskach on‑premises, w chmurze, hybrydowych i brzegowych, pomagając eliminować wąskie gardła w przechowywaniu, poprawiać wykorzystanie GPU oraz przyspieszać trening i inferencję modeli AI. Firma coraz częściej pozycjonuje swoją technologię wokół rozwijającej się gospodarki inferencji i agentowej AI, a jej infrastruktura została zaprojektowana tak, aby zapewniać wysoką przepustowość i niskie opóźnienia dostępu do danych na masową skalę, upraszczając jednocześnie złożone potoki danych AI. WEKA obsługuje przedsiębiorstwa, dostawców chmur, organizacje badawcze i deweloperów AI pracujących w jednych z najbardziej wydajnościowo intensywnych środowisk obliczeniowych na świecie.

Your career has taken you from shaping NetApp’s early cloud strategy and serving on Kubernetes’ founding governing board to building AI infrastructure at WEKA. How has that evolution shaped the way you think about preparing infrastructure for the next phase of AI?

Every era of my career has been defined by the same pattern: the bottleneck moves, and the industry takes years to notice. In the early cloud and Kubernetes days, we watched compute become elastic and orchestration become the new choke point. Coming out of NetApp, where I was CTO after the SolidFire acquisition, I thought I knew what fast looked like: low milliseconds for a truly random first-byte read under production workloads.

The reason I joined WEKA is genuinely geeky. It was one stat: the first uncached random byte read is 70 microseconds, which is not a storage-class number. I had never entertained the notion of microsecond-level latency from this class of system. That was the light bulb: this technology could serve memory applications, DRAM-class applications like Redis and KV cache, not just storage. And right on cue, inference started superseding training because the industry had to monetize these models, and agents arrived to make memory the whole game.

That’s the lens I bring to AI infrastructure. We’ve seen this movie before. Cloud FinOps emerged because companies spun up infrastructure with no unit economics rigor, then got the bill. AI is on the same curve, moving faster.   As businesses inevitably pushed back on tokenmaxxing with the arrival of API consumption bills far in excess of planned token budgets, we’re witnessing the rise of AI FinOps. This is the point where organizations stop treating inference as a cheap utility, and start managing token efficiency as a financial discipline. AI FinOps starts with tokenomics: optimizing every hardware and software layer of the inference stack that impacts the unit cost per token. Right now the biggest waste in that stack is expensive GPUs and new ASICs sitting idle, waiting on memory and data (aka ‘decode’) rather than FLOPS (aka ‘prefill’). Whoever fixes that owns the next phase of AI.

The White House is keeping the details of its new AI safety framework confidential. How can enterprises prepare for regulatory requirements when they do not yet know exactly what will be tested or required?

Companies shouldn’t wait for the final checklist. The specific tests will change, but the obligation underneath them will not: you will need to demonstrate what your AI model did, what data it touched, and how it behaved at a specific point in time. And waiting is already not an option globally. The EU AI Act became enforceable this month, and it classifies most agent orchestration as high-risk.

That means the preparation work is infrastructure work. Data lineage, observability, reproducibility, and the ability to reconstruct model state on demand are all critical for enterprises to get right. Finally, organizations need to implement guardrail models before outputs, with associated latency and token budgets for the semantic defense layers. If you build those capabilities now, any framework becomes a formatting exercise. Wait for the final rules, and you will be retrofitting accountability into systems that were never designed to explain themselves. That retrofit is always more expensive than building it in from the start.

Ultimately, the solution for building safe AI is more AI, applied optimally and very intentionally.

What types of new infrastructure demands could AI safety testing create, and how different might those workloads look from conventional model training or inference?

Training is a firehose. You push enormous volumes of data through a model in a sustained, predictable pattern. Safety testing is the opposite: thousands of evaluation scenarios, repeated probing, version-over-version behavioral comparisons, and adversarial red-teaming that never really ends.

That profile matters. Safety training and testing is bursty, read-heavy, and comparative. It generates and consumes enormous amounts of intermediate state. Guardrail models need to be inherently heterogeneous and layered, implemented within tight latency budgets, amplifying evals with this new dimension of safety criteria. For such advanced or cyber-capable models, the workload profile is 24/7 persistent rather than episodic. You are not running a test once and filing the results. You are running continuous agent swarm workloads that compete in production with compute, memory, and data bandwidth for those critical apps they protect. After quality and speed, most infrastructure today is not designed for this third tenet.

There is a measurement problem underneath this, too. Most AI benchmarks today run 8,000 tokens or less, one prompt, one response. I joke that they are artificial benchmarks of artificial intelligence. In mid-2026, real agent workloads run 100,000 to 400,000 tokens of context across thousands of turns. If safety evaluations inherit those toy benchmarks, we will be certifying systems for a world that doesn’t exist. Regulators are already building better muscle here: NIST has open-sourced agent security evaluation tooling, and early published results show novel agent-hijack attacks succeeding at several times the rate of known baselines. That is exactly the kind of continuous, adversarial, expensive testing I expect safety frameworks to converge on.

Should organizations be building excess compute and data capacity specifically for future compliance and safety workloads, or is there a more efficient way to design for that uncertainty?

Buying excess GPUs and hoping utilization catches up just leaves capital stranded in depreciating hardware.

The efficient answer is infrastructure that flexes between production and evaluation workloads without a separate stack. That is fundamentally a data problem. If you can move and reuse data efficiently, preserve context between workloads, and keep your accelerators busy doing real work, compliance becomes an incremental cost instead of a parallel buildout. The economics of AI increasingly come down to how much value you extract from every token, byte, and watt. Get it right, and you can generate 3–4x more value from the same infrastructure, or cut your rack footprint by as much as 75%. Compliance should be held to the same standard.

The current framework reportedly focuses on advanced closed models while excluding open-weight models. What infrastructure or security challenges could emerge from treating those two categories differently?

If you treat closed models and open-weight models differently, you end up with two compliance frameworks for technologies that do the same things, and the gap between them is where the risk lives.

An open-weight model can be fine-tuned and deployed in environments where the original provider has zero visibility. Regulating the provider does nothing there. And the split is already visible: export controls landed on the newest closed frontier models this year, while open-weight models cross borders freely and now sit near the top of public capability leaderboards. However the administration ends up defining frontier models, governance cannot stop at the model itself. You need visibility into where models are running, what data they access, what prompts, responses and metadata are retained, how they have been modified, and whether the infrastructure underneath can actually support governed AI at scale. New updates to ISO27001 and SOC2 will be required.

My answer is trust but verify. If your infrastructure gives you the token capacity, you can run heterogeneous guardrails against any model before its output ships: domestic or foreign, open or closed. Objective verification beats blanket trust or blanket distrust based on where a model came from. As open models proliferate, that verification capability lives in the infrastructure layer, and that is where enterprises will differentiate. Policy may decide which models are permitted. Infrastructure decides whether those models can be deployed responsibly and economically.

As AI agents become more autonomous and operate across longer contexts, how does that change the amount of data, memory, and compute organizations need to allocate to monitoring and safety?

A chatbot is a prompt and a response. An autonomous agent is a running process. It touches dozens of systems, retrieves information, makes intermediate decisions, and accumulates state over hours or days before it finishes a task.

You cannot monitor that by sampling individual tokens or responses. You have to capture the full sequence: what the agent knew, when it knew it, and what it did next. Every hour an agent runs, its state grows, and so does the memory, data movement, and infrastructure required to capture and analyze it. Monitoring stops being a logging feature and becomes a first-class workload with its own resource budget.

Defense is where this gets urgent. AI’s memory problem is becoming a security problem. A coding agent can spin up, ship, and shut down. A cybersecurity agent can’t. It has to hold context across daily shift changes in a security operations center, frequent model updates, and sophisticated multi-stage attack campaigns that used to span weeks, but also operate at coordinated machine speeds today. When that AI working memory gets evicted and recomputed from scratch every few minutes, an agent that detected anomalous behavior in hour one of an incident has no memory of it in hour two. Attackers don’t have that problem. Their agents persistently identify and pursue weaknesses, and kill chains now complete at tokenomics-optimized machine speeds, so AI-powered cyber defense has to run autonomously around the clock. And this is not theoretical. Security vendors are gearing up for 24/7 persistent cyber agents right now, and the first thing they discover is that the economics look nothing like chat workloads. Some organizations need those agents running at the edge, in facilities where you cannot ship a GPU rack or even a chiller this year. The real test for enterprise AI is sustained context retention, not point-in-time inference. This becomes a battle of token attrition, and whoever solves persistent context memory at scale will power the first horizontal killer app in enterprise AI: always-on blue agent swarms.

You have spoken about the growing importance of “context memory” as AI workloads move beyond simple chat toward persistent agents. Could context memory also become important for auditing, reproducing, or investigating AI behavior?

Absolutely, and it’s an important use case. For years, memory was a performance story: how fast you could feed the GPU, how much context you could hold. Once agents start acting autonomously, that same memory becomes the evidence. If an agent makes a decision based on context it accumulated over days, the final prompt and output tell you almost nothing about why it acted. The explanation lives in the accumulated state.

In technical terms, most of that state lives in the KV cache, and the industry still treats it as disposable scratch space instead of durable data. If you preserve that state and can retrieve it efficiently, you can reconstruct what the system knew at the moment it acted. Teams will use that for debugging first, then for safety evaluations, and eventually, someone will need it in an investigation. Throwing context memory away means throwing away the only record that explains why your AI did what it did.

Could AI regulation ultimately force companies to retain significantly more information about model inputs, outputs, checkpoints, data lineage, and agent activity? What would that mean for AI infrastructure architecture?

Directionally, yes. As AI systems become more consequential, visibility requirements will expand to cover every step of the pipeline. You can already see the early signals: teams are planning to retain stale context memory in cheaper object storage tiers purely for audit purposes, before any regulation requires it.

Authentication of monitoring with immutability is essential. Tampering of logs and other forensic artifacts by malicious agents has become routine, necessitating elaborate cryptographic verification systems not vulnerable to concentrated, centralized integrity targets of attack. Simple transparency logs or hash chains are not sufficient to defend against the collusion abilities of coordinated agent swarms. Highly decentralized, public blockchain architectures are ideally suited for this, highlighting the often overlooked Integrity value, in the middle of cybersecurity’s C-I-A triad.

Retention is not merely a daunting, immutable storage problem at scale. The hard part is keeping that information trusted, organized, indexed, and retrievable fast enough to be useful under a deadline, whether that deadline comes from a regulator, an incident response, or a courtroom. A petabyte of agent activity you cannot query is a liability, not a record. The architectural shift is from “more storage” to infrastructure designed around objectively verifiable, persistent, queryable AI data as a core workload.

Many organizations are focused on buying more GPUs, but where do you see the less obvious infrastructure bottlenecks emerging as AI workloads scale and safety requirements become more demanding?

The GPU gets the headlines because it’s not an insignificant budget item. But GPUs and especially new decode-optimized accelerators (ASIC+SRAM) are rarely the actual constraint. Memory bandwidth, data gravity and movement, storage performance, and networking determine whether those accelerators are doing productive work or waiting underutilized.

As AI becomes more context-heavy, the memory wall becomes the defining constraint. You can keep adding GPUs, but if they spend their cycles recomputing tokens or shuffling context between systems, you’re paying for wasted work, repeatedly. Scaling the memory wall means making the data layer behave like shared storage, but perform at true memory speeds. That HBM-adjacent speed bar matters for the profit center wave of KV cache offloading solutions arriving now: any of them has to deliver true memory-class performance to make the tokenomics pencil out. Retention value for KV cache is a cost center discussion, secondary to the profit center. And if reading cached context is slower than just recomputing it, the cache is worthless to the business equation. What matters isn’t how many GPUs you own, it’s how productive each one is. The economics of AI come down to the value you extract from every token, byte, and watt, and safety requirements will only raise the stakes on that math.

Looking ahead, do you expect AI safety and compliance to become a distinct infrastructure workload of its own, similar to how cybersecurity evolved into a dedicated layer of enterprise technology?

We will see AI safety and compliance become a distinct workload, and the cybersecurity parallel cuts both ways. Security became its own layer once the industry accepted that it could not be an occasional exercise. The cyber insurance industry made that non-optional. AI safety is on the same mandatory trajectory as models grow more capable and more autonomous.

But we should learn from where security went wrong. It became a bolt-on: a separate stack, a separate budget, a separate team discovering problems after the fact. Safety infrastructure should not repeat that mistake. Monitoring, evaluation, auditability, and immutable retention belong inside the AI infrastructure itself, co-designed in from the start.

Here is the part most people miss: safe AI requires more AI. Guardrail models are not free. They have to be continuously trained, fine-tuned, and inferenced at every step of agent runtime. Token latency budgets make this concrete: every response has a fixed window, and the more tokens you can process inside that window, the more verification you can afford to run before the output goes out the door. And the real threat from frontier models is their agentic application. Agents run as high-volume inference loops, making repeated calls to models over long horizons. Every loop is observe, orient, decide, act, and every step burns tokens. That turns AI security into a war of token attrition. Attackers field red agent swarms, defenders field blue agent swarms, and the side that can afford to generate more tokens per dollar and per watt wins. Tokenomics sit on the critical path of both attack and defense. This stopped being a thought experiment this summer, when a red agent swarm attack against a major model repository scared the industry straight, and a dedicated secure AI alliance formed within days. Meanwhile, the volumes keep compounding: industry token processing has moved from trillions to quadrillions.

Once safety becomes an always-on requirement, its compute, memory, and data costs stop being overhead. They become part of the fundamental unit economics of running AI. The companies that internalize that early will treat safety as a design input. Everyone else will treat it as a tax.

Thank you for the great interview, readers who wish to learn more should visit WEKA.