AI agent for configuring industrial HMIs through natural language
An LLM-based experiment integrated into a visual editor to simplify the creation of customized industrial interfaces.
Client
An e-mobility company.
The challenge
In industrial environments, configuring an HMI often means working with specialized tools, custom logic, and interaction patterns that require specific technical expertise. In this project, the goal was to make the creation of new interface pages simpler and more intuitive by introducing a conversational interaction model into software that would normally require a mouse, keyboard, and in-depth knowledge of the editor.
The complexity also stemmed from the application context itself: the system had to operate in a real industrial environment and interact with a non-standard language defined by the product. It was therefore not enough to generate a merely “plausible” interface. The solution had to respect the technical constraints, functional requirements, and operating logic already built into the configurator.
Solution
Methodology
- Kanban-inspired Agile methodology within the team
Services involved
Technology
- OpenAI API
- LangChain
Looking to integrate LLMs into your product line?
We integrate LLMS tailored to every industrial need.
Results
The project is part of the evolution of an HMI editor built on a web-based stack, designed to move beyond a previous Qt-based approach and provide a more modern and flexible way to manage customization. At the core of the solution is a drag-and-drop editor that makes it possible to build customized interfaces, going beyond simple changes to logos and colors and enabling intervention on specific workflows and functionalities as well.
Within this editor, an experimental solution based on the OpenAI API and LangChain was developed, featuring an AI agent capable of interpreting natural language instructions and translating them into an initial concrete interface configuration. This allows users to describe the desired outcome and start from an already generated structure, which can then be refined within the visual editor.
A key aspect of the work is that the agent does not produce generic outputs, but content consistent with the custom format required by the system and with the constraints of the application domain. For this reason, the model’s behavior had to be specialized for the specific context, making it useful in a real industrial scenario rather than merely as a demonstration.
Benefits
- The agent helps operators manage complex tasks within the configurator.
- It speeds up the creation of both the design and the initial page structure.
- It reduces the need for detailed knowledge of every technical component of the editor.
Key features
The distinctive element of the project is not the use of artificial intelligence in itself, but the way it has been applied: as operational support integrated into a real industrial software environment. The agent works as an abstraction layer on top of the editor, turning a request expressed in natural language into a concrete starting point. In this sense, it introduces a more accessible interaction model without artificially simplifying the complexity of the context.
There is also a second noteworthy aspect: the agent not only speeds up configuration, but also improves the initial quality of the outcome by proposing coherent layouts and providing operators with design support that complements their technical expertise.
Conclusion
The project points to a possible direction for the evolution of industrial interfaces: integrating natural language where it can reduce complexity and make interaction with advanced tools more efficient. The goal is not to replace existing paradigms, but to complement them with more intuitive interaction models that can support operators in configuration and design tasks.
In this sense, the experiment offers a concrete view of how conversational interfaces can find a place in the industrial world as well: not as an accessory feature, but as a useful tool when software becomes complex and configuration time truly matters.