Open Source Orchestration Framework

Practical example and current challenges

• Open source software is enjoying increasing popularity. It is often brought to market in competition with branded software in order to break monopolies. Many developers work voluntary on it. However, this results in unfamiliar areas of conflict for companies, since working with open source communities can be unfamiliar and non-binding. Additionally, the operation of this software, especially in the area of distributed systems, is often complex and at the same time critical for companies.
• Many companies currently use software components from the Apache Hadoop ecosystem to process and manage data in Big Data Use Cases. These are often open source and free software.
• Since individual components from the ecosystem are insufficient for a meaningful architecture, a combination of different components is necessary. The components are bundled and offered by so-called distributors. This includes proprietary management components to manage the overall system. Due to the almost complete monopolization on the distributor market, the prices of these solutions are too high for many companies to implement them profitably.
• Due to the high complexity resulting from the combination of components, the distributors available on the market are not up to date. Although the open source community continues to develop the components, the distributors cannot integrate the further developments of individual components into their combined products as quickly.
• At the same time, the latest versions are becoming more and more important for various companies in order to be able to use the offered functions for their purposes in a timely manner. A framework that orchestrates open source products could provide a remedy. A framework that makes it possible to orchestrate any software products and efficiently encapsulate the necessary know-how in code.
• Solutions of existing cloud providers are mostly proprietary, which results in a renewed vendor lock-in or high switching costs. A circumstance that many companies would like to avoid by using open source software.

What added value does the "GAIA-X project" offer?

• GAIA-X helps to reduce the high degree of dependence on a monopolistic vendor and is the crucial piece of the puzzle for a framework solution to orchestrate any open source software.
• GAIA-X promotes the use and acceptance of open source software.
• GAIA-X enables the implementation of modular approaches, which play an important role here: Firstly, software can thus be defined as a self-sufficient service and any necessary logic for the operation of the product can be implemented in a module. The domain knowledge required to operate the individual products is defined in code, and at the same time the operation of the software is greatly simplified.
• Secondly, dependent services can be defined. On the one hand, this makes it possible to depict complex architectures and, on the other hand, to check the necessary compatibilities.
• Thirdly, flexible distributions are enabled, which consist of various modules and can contain more or less architectures. The modules and components are arbitrarily expandable. This makes it easy for technology users to run self-developed or previously unsupported software with this system and to integrate it seamlessly into the defined architectures.
• This framework solution has to measure up to its own standards and is therefore developed as open source software in order to avoid a future strong vendor lock-in.
• In addition to the economic aspects, there is also the technical aspect of achieving significantly greater flexibility when designing data-driven architectures. Since services can be combined with each other in any combination within their dependencies, significantly more agile approaches are made possible.

Use Case Team

• Sönke Liebau – OpenCore GmbH & Co. KG
• LLars Francke – OpenCore GmbH & Co. KG