We connected a Event Data Logger, written in Java and connected to WinCC OA by JNI, to Apache Kafka, a A high-throughput distributed messaging system.

Apache Spark can read data from Apache Kafka streams. Apache Spark is a fast and general engine for large-scale data processing. Combine SQL, streaming, and complex analytics. Spark powers a stack of libraries including SQL and DataFrames, MLlib for machine learning, GraphX, and Spark Streaming. You can combine these libraries seamlessly in the same application.