French AI startup ZML launches software to cut AI running costs

ZML launched LLMD on July 8, releasing a free inference server built to run open-source language models across Nvidia CUDA, AMD ROCm, Google TPU, Intel oneAPI and Apple Metal hardware. The French startup, backed by Turing Award winner Yann LeCun, is pitching the software as a way to make AI less costly to operate at scale.

The business problem ZML is aiming at is not model training but inference, the running of AI systems after they are built. That is where costs stack up day after day, especially when companies are forced to tailor their software to a narrow set of chips. ZML says its stack is built to decouple AI workloads from proprietary hardware, letting teams keep one codebase while moving models across different accelerators.

ZML says LLMD can serve several open-source model families, including LLaMa, Gemma, Qwen and Mistral. The company’s own documentation and product pages describe the system as a self-contained inference server built close to the hardware, with support designed to make model deployment less dependent on Python-heavy runtime layers that dominate much of the market.

If that approach works in production, it could matter well beyond one startup. Software that runs efficiently on Nvidia, AMD, Google, Apple and Intel hardware gives buyers more room to switch chips, negotiate prices and reduce dependence on the dominant AI infrastructure stack. That would not remove the need for expensive compute, but it could change who captures the margin around it.