Transforming Factories into Software Systems: A Visionary Chat with Dr. Josef Waltl

"We're doing to factories what cloud providers did with data centers in the early 20s," says Dr. Josef Waltl, CEO of Software Defined Automation, in his interview with Jeremy Theocharis, CTO of United Manufacturing Hub.

Imagine a time when technicians physically walked into data centers, DVDs and CDs in hand, to manually update systems. Josef paints a vivid picture: "That's where many factories are today—staff walking up to electrical cabinets with laptops, each running different versions of software, to manually code and update systems. It's inefficient and lacks oversight."

Josef's vision is groundbreaking yet straightforward: evolve factories into software-driven systems, replicating the seismic shift that cloud technologies induced in data centers. This is more than just an upgrade; it's a revolution aimed at making factories agile, secure, and efficient.

They delve into topics that make this transformation possible: the role of DevOps in manufacturing, simplified PLC management, and the critical merger of IT and OT.

Highlights of Our Conversation:

• Adopting IT Best Practices: Josef envisions factories that learn from the IT world, incorporating DevOps and continuous integration.
• Streamlining PLC Management: "PLC Ops" simplifies the complex landscape of PLCs by offering real-time insights and cross-vendor management, making the PLCs both agile and easier to handle.
• Security First: Weak access control is a known issue in PLCs. Software Defined Automation addresses this by securely managing access via the cloud.
• Server-Based PLC Logic: By using real-time operating systems, hypervisors, and Codesys runtime, Software Defined Automation allows for PLC programming to run on servers. "Why are we running logic right next to the factory line? Why not virtualize it and run the logic on a HP or Dell server instead?"

Take a moment to watch the full interview with Dr. Josef Waltl to grasp the depth of this paradigm shift. You'll hear firsthand about the challenges, opportunities, and next steps for turning factories into software systems. The full transcript is also available for those who prefer to read in detail.

Full transcription

Hi, this is Jeremy, CTO and co-founder of the United Manufacturing Hub. With me is Josef Waltl, founder and CEO of Software Defined Automation. Hi, Josef.

Hi, Jeremy.

Jeremy: To start, for our listeners to get to know you, what do you like to do in your spare time?

Josef: In my spare time, I try to get away from the computer and spend time outdoors, cycling, and being in the mountains. I'm Austrian, so that's pretty much in my genes, I think. Apart from that, I'm keen on understanding where technology is heading. I listen to podcasts and read books to catch up on the next wave of innovations.

Jeremy: What's something that currently interests you the most, unrelated to your work field?

Josef: I'm fascinated by the capabilities of generative AI at the moment. I was surprised by how powerful these models have become and am interested in combining them with semantic models to answer domain-specific questions more accurately.

Jeremy: Could you elaborate on what you mean by semantic models?

Josef: Semantic models provide context to pure text by defining what certain elements mean within a specific domain. By combining these models with text-based AI, we can deliver better answers to specialized questions.

Jeremy: Moving on to your career, you've had an impressive journey. You started with a Ph.D., I believe during your time at Siemens, and now you've founded a company. Could you share more about this path?

Josef: Sure, although I wouldn't necessarily call it "impressive." I began as an industrial electronics guy in Austria, where you can complete your A-levels while also learning a trade. Then I studied computer science and a bit of business. At Siemens, I quickly moved into the industrial software space, where I contributed to building up an impressive portfolio that includes CAD systems, simulation systems, and industrial control systems. Later, I was asked by AWS to help build their partner ecosystem for industrial software, which ranged from companies like Autodesk to Mitsubishi Electric. I also worked in product management for Azure's Industrial IoT, AI, and analytics sectors. My background in industrial automation and software, combined with my experience in cloud technologies, led to the idea for Software Defined Automation.

Jeremy: Given your experience in automation and cloud technologies like AWS, where do you think the industry is heading? What's your vision for the factory of the future?

Josef: I'll start by answering at a high level and then get into specifics. In Operational Technology (OT), we've seen microprocessors replace relay control logic, leading to the creation of programmable logic controllers (PLCs), robot controllers, and so on. This trajectory has continued. Meanwhile, in IT, we've moved from monolithic systems to modern software development practices, including microservices.

In both IT and OT, hardware has improved: PLCs have gotten stronger and data centers have moved to the cloud. However, some best practices from the IT world, like DevOps and continuous integration, haven't been fully adopted in OT. To put it another way, while we see powerful systems in OT, we haven't witnessed a paradigm shift in how these systems are developed and managed.

To directly address your question, we're at a crucial moment. We're on the brink of a major shift. If we don't address current challenges, such as the labor shortage of skilled automation engineers, it will be difficult to ramp up essential production processes, whether it's in battery manufacturing or other sectors that we want to bring back to Europe and the U.S.

Jeremy: I recently spoke with someone from the OT side who argued that IT should adjust to OT, rather than the other way around. What's your take on this perspective?

Josef: While there's some truth to the idea that OT is less flexible due to its basis in physics, I don't fully agree with the statement. Instead, I advocate for first-principle thinking: identify the core problem and use the best tools from both IT and OT to solve it. That's why we focus on providing immediate benefits with our solutions. For example, we offer a PLC management system that makes it easier to manage, deploy, and secure PLCs. We aim to meet OT engineers where they are, offering tools that are immediately useful and then gradually introducing applicable IT concepts.

Jeremy: You mentioned your focus on immediate benefit and the practical concerns of OT, especially where stopping production is a significant risk. How would you briefly explain what Software Defined Automation does, for those who might check out your website?

Josef: In essence, we try to combine the best of both worlds—IT and OT—to create solutions that bring immediate value. For instance, we streamline PLC management so you know exactly what programs are running, and their versions. Our goal is to make life easier for automation engineers and PLC programmers from day one, while also introducing IT best practices where they make sense. We're driven by what adds real value now, especially given the industry's reluctance to adopt untested technologies that could halt production. Our software is designed to minimize those risks.

Jeremy: What's your mission and vision?

Josef: Our vision is straightforward: We want to do for factory computers what cloud providers did for data center computers. Back in the early 2000s, managing data centers involved a lot of manual work—like using DVDs and CDs to install systems. Right now, factories are similar; they're often manually managed, lacking standardization and oversight. Our goal is to transform these factories into streamlined software systems, much like how data centers have evolved into cloud-based infrastructures. This is critical given various challenges, such as a shrinking skilled workforce and geopolitical factors. We're committed to building the best tools to meet these challenges.

Jeremy: How are you achieving this vision?

Josef: Achieving this vision is complex, and here's why: Industrial computers are reliable, cost-efficient, and capable of real-time control. They aren't going away anytime soon. Moreover, most factories are designed to last for decades before requiring significant upgrades. That's why our initial focus is on what we call "PLC Ops," a software-as-a-service tool for managing existing Programmable Logic Controllers (PLCs). This tool connects PLCs to the cloud, giving you real-time information on their status, the programs running on them, and the option to back them up.

We've designed PLC Ops to work across different PLC vendors, making it easier to manage a diverse range of systems. Whether it's Siemens, Rockwell, or other technologies, our solution simplifies their management. Companies like Hengel are already using it to manage their varied PLC technologies. This initial step helps eliminate the labor-intensive, non-productive work that often comes with managing PLCs manually.

So, in summary, our first major effort is PLC Ops, aimed at making PLC management simpler and more efficient. This sets the stage for further developments as we work towards transforming factories into more agile, software-driven systems

Jeremy: What do you mean by "management" of PLCs? Is it just reprogramming them or something more? For those coming from an IT background, what typically needs to be done with these PLCs?

Josef: When we talk about managing PLCs, it's much more dynamic than just setting them up and forgetting them for decades. Production changes frequently. This could mean simple tasks like adding a new data point to be analyzed or swapping out a motor. These tasks require reprogramming the PLCs. Often in PLCs, you'll find hardcoded values instead of adjustable settings, which isn't ideal but is the current reality.

Jeremy: So, making changes that would simply involve altering an environment variable in an IT setting often require reprogramming in the PLC environment?

Josef: Yes, exactly. A lot of what would be adjustable settings in IT is hardcoded in PLCs, making them less flexible. Take changing the speed of a conveyor belt as an example; that often requires reprogramming the PLC.

Jeremy: Could you also touch upon the connectivity aspect? How is access to these PLCs controlled?

Josef: Traditionally, access is via a VPN, and credentials are given to external parties like machine builders or system integrators. This becomes a security risk as you lose track of who all has these credentials. To address this, we've invested in creating a secure cloud environment and an innovative connectivity stack that simplifies access while enhancing security. With our setup, you can manage access on a user-by-user basis, streamlining the process and improving security.

Jeremy: Does this new approach have other benefits?

Josef: Absolutely. It minimizes costs and reduces the environmental impact by cutting down on the need for programmers to physically travel to different sites. This is not only better for the planet but also improves the work-life balance for these professionals. Companies like Encl are already using our system to manage access for their machine builders, leading to more efficient and secure operations.

Jeremy: So you're focusing on security and access control. Traditionally, everyone has root access to the PLC and must physically visit the factory for programming. You solve this by connecting the PLC to the cloud and putting the IDE software there. This seems to address several issues:

1. The IDE is now browser-based, so you don't need a dedicated Windows PC.
2. You can provide continuous access.
3. Access rights are now managed centrally through the cloud.

Is that accurate?

Josef: Yes, you've got it right. We're actually tackling several problems in one go. First, we're centralizing access control. In the old system, most PLCs didn't even have usernames and passwords, mainly to avoid shutdown risks. Now, we control access through the cloud. This lets us specify who can read or write to specific PLCs. We also sync this with corporate directories, so we can grant or revoke access based on project timelines or staff changes.

On top of that, we're simplifying version control for PLC programs. These used to be hidden in proprietary files, but we've made it possible to manage these centrally. You can see who changed what and when, and we even offer rollback features to reduce risks when updating. Our system works with various programming languages, not just text-based ones.

Finally, we're reducing hardware dependencies by allowing PLC software to run on general-purpose x86 servers. We've partnered with VMware for this. It's a big deal, especially given how the pandemic disrupted supply chains, making it hard to get specific hardware. This new approach mitigates that risk by making it possible to switch hardware more easily.

Jeremy: I'm interested in the real-time aspect. I know Linux has a real-time kernel patch to guarantee cycles every few milliseconds. Can you explain this from a technical standpoint? How do you ensure real-time operations, especially when networking and using hypervisors?

Josef: "Real-time" means being just in time for a physical process. The goal is to be faster than the process's reaction time. In process industries like oil rigs, if a valve takes three minutes to close, real-time can be two minutes. In discrete manufacturing, like making consumer goods or fighter jets, our target is under 10 milliseconds.

On the tech side, you need a real-time operating system. While some versions of Unix may not reliably perform this, real-time hypervisors can. VMware, for instance, has a real-time patch for their ESXi hypervisor that we've found to be very efficient, with latencies around 3-5 milliseconds.

Then you need a way to get the processor's logic to the actuator's signal. Field buses like Profinet or EtherCAT help here. They're like a bus that comes by every minute; you always jump in the same seat because you have a designated place in the protocol for an I/O signal.

Our system solves these issues. It's reliable, tested by customers, and also friendly for OT engineers. Setting up a virtual PLC and a real PLC is the same process. You don't need special training to work with our solution.

Jeremy: From an IT perspective, how do factories work without DevOps tools like Git, version control, and authentication systems? Can you give an example?

Josef: Sure. Let's say a car factory needs to make changes to its production line. They shut down the factory on Friday. About 70 people come in and manually update the software on each PLC (Programmable Logic Controller) by plugging in their laptops. They finish by Monday and then fix issues during the week. This approach relies heavily on the skill of the engineers, but it's risky and inefficient.

Jeremy: Why do we put that risk on people? In tech companies like Netflix or Amazon, changes are frequent and automated. Shouldn't factories do the same?

Josef: Absolutely. Our goal is to move from no control to automated control, deployment, and backup. More automation means less risk, which in turn means more room for innovation.

Jeremy: How can our community, mostly IT and OT architects, benefit from using software-defined automation? How can they get started?

Josef: They can start by going to our website, software-defined-automation.io, and signing up for a free two-week trial. All they need is a gateway with Ademia Linux, internet connectivity, and a PLC programmer.

Jeremy: Great, thank you for your insights. If anyone has questions, they can reach out to Josef or try the free version of the software. Thanks, Josef!

Josef: Thank you, Jeremy.