Overview

Here we go over the architecture of the UMH stack, how the various microservices and open source 3rd party apps interact.

The United Manufacturing Hub is a modular system, with several various microservices as building blocks interacting with each other and with open source 3rd party applications, such as Node-RED and Grafana. This is beneficial as it enables flexible use and development with more functionalities easily being added with further microservices. It allows us to create comprehensive solutions for various challenges of the industry.

In the following, we will go through all layers of the United Manufacturing Hub and provide examples.

Layers

Broadly speaking, the United Manufacturing Hub consists of three different layers: Data acquisition, data processing and data visualization. In the following chapter, we will discuss the purposes of each layer, before going into detail which microservices are involved and how they interact.

Data extraction

Data extraction entails managing the connections of the edge device to the data sources, such as external sensors (e.g. light barriers or vibration sensors), input devices (e.g. button bars), Auto-ID technologies (e.g. barcode scanners), industrial cameras and other data sources, such as machine PLCs. The data acquisition also reads out the data from the sources to provide the data to the data processing services.

Current examples of data acquisition:

  • sensorconnect, which automatically reads out IO-Link Master and their connected IO sensors, like the light bridge sensor in the basic installation guide

  • barcodereader, which connects to USB barcode reader devices and pushes the data to the message broker

  • Node-RED, which can handle proprietary or machine specific protocols

Further examples, which are either still in development or currently not supported:

  • cameraconnect, which automatically reads out GenICan compatible cameras and pushes the results to the message broker

  • PLC4X-connector, which connects to PLCs using the PLC4x library

Data processing

This layer is the central part of the United Manufacturing Hub. It provides an infrastructure including data models to fulfill all manufacturing needs for data processing and storage. All acquired data is made accessible in real-time for processing using either established solutions like Node-RED or microservices. Therefore, adding new data, processing it and integrating it with other systems on the edge is very easy. We recommend to start transforming data into the central data model at this step.

The raw or processed data can be sent to a central place (both cloud or on-premise) with our self-written microservices, such as kafka-bridge, which can connect the local kafka broker to a central one. Since internet connections or general networks are often unstable in manufacturing environments, buffering messages is required to safely get messages across even through internet and electricity outages. Existing services were unsatisfactory, so we developed our own solution for kafka bridging to other kafka brokers and to MQTT ones.

Once the data is at the desired server, it will be further processed with the same methods as on the edge device. For development, we integrated kowl into the stack to read out kafka in real time so debugging and observing is even easier. Therefore, one can achieve high-availability and enormous scalability through the load-balanced microservices.

Relational data, like data about orders and products as well as time series data in high resolution, like temperature and pressure can be stored in the TimescaleDB database. We do not recommend direct access to the database for security reasons. Instead, we have added an additional, self-written component called factoryinsight. It proves a REST API to access raw data from the database as well as processed data in form of KPIs like OEE losses or similar ones. All requests are load-balanced, cached and executed only on a replica of the database.

Current examples of data processing:

  • TimescaleDB, which is the database and as such stores the relational and time series data for further use. See also the database model

  • Node-RED, which is the tool to customize the data processing and allows to output processed data from the raw input.

  • factoryinsight, which reads out the TimescaleDB database for further visualization of the data.

  • kowl, which reads out kafka messages for debugging and observation in development

  • kafka-bridge, which connects a local kafka broker to a remote kafka broker for centralization of data processing

  • mqtt-kafka-bridge, which connects MQTT and kafka brokers with each other to allow for a maximum of flexibility

  • kafka-to-postgresql, which feeds the kafka message data into the TimescaleDB database

Further examples, which are either still in development or currently not supported anymore:

  • mqtt-to-postgresql, which feeds MQTT message data into the TimescaleDB database

  • kafka-to-blob, which would feed blob data and data models into the minio database

  • minio, which is the database for blob storage to store images

  • blob-insight, which provides the blob storage data via REST API

  • mqttbridge, which connects different MQTT brokers

  • factoryinput, which allows for data insertion via REST API

Data visualization

As a standard dashboarding tool the United Manufacturing Hub utilizes Grafana in combination with self-written plugins, which allows users to quickly and easily compose tailor-made dashboards with the help of modular building blocks.

Current examples of data visualization:

  • Grafana, which manages the dashboard for visualization

  • umh-datasource, which integrates the data from factoryinsight into Grafana

Further examples, which are either still in development or currently not supported:

  • dashboard-template, which provides a wide range of pre-defined dashboards to choose from

  • factoryinput-panel, which allows to enter data to the REST API endpoint through the Grafana dashboard

Deployment options

The entire stack can be deployed using only a single configuration file values.yaml and the corresponding helm chart. This allows to deploy the architecture in hybrid setups, from deploying it to edge device IIoT gateways to on-premise servers or even to the cloud (e.g. Azure AKS)

Detailed Example

Let’s take data from a light bridge sensor as an example. As you can see in the image above, the microservice sensorconnect is reading out the raw data from the IO link gateway and converts it into a Kafka message and sends it to the kafka broker. the Kafka broker then can send it to Node-RED if you have a Node-RED workflow, which subscribes to the correct topic. You can then put out different messages with processed data, such as how often items have passed the light bridge in the last couple of seconds as output.

The Kafka broker can then either send it directly into the TimescaleDB database or send it to a central Kafka broker, which handles the same thing.

When you want to visualize the data, factoryinput reads out the database and sends it to the microservice UMH datasource, which integrates the data into Grafana, where you can then make fancy dashboards with the data input.


Deployment options

The United Manufacturing Hub can be deployed in three forms: cloud, on-premise server and in an edge device. Please proceed with the present article in order to understand each option.

Last modified November 23, 2022: Feat/linkfixes (#176) (120bad7)