MuSE: Clustering, Extensibility & Almost Infinite I/O Power with OPAL-RT Simulations

OPAL-RT has improved the user experience when using multiple simulators, remote targets or when requiring clustered machines for increased I/O or other capabilities. This high-speed link is referred to as MuSE (Multi-System Expansion link).

Irène Pérès, MuSE’s Product Owner at OPAL-RT, recently joined us to speak about the incredible power, flexibility, versatility and user-friendliness of this special add-on—that is so much more than just an expansion kit.

Interviewer (IV)Thanks for joining us, Irène! I have a partial understanding that MuSE is sort of a hardware expansion kit, allowing clustering of multiple machines, and frequently used for augmenting available I/O. Is that how you’d describe it?

Irène Pérès (IP): “MuSE does indeed involve some hardware components: optical fiber and SFP (Small Form-factor Pluggable) transceivers. However, the hardware is just the tip of the iceberg, as the essence of MuSE lies in the software layers we’ve developed. The problem we needed to solve was: when a user needs more I/O channels than are available in one chassis, how can we facilitate connection with other chassis to add that additional I/O capacity?”

IV: MuSE has been used for simulation in industries where they tend to use a lot of I/O, like aviation and aeronautics—is that right?

IP: “Yes! We first developed the link with these industries in mind. But actually, we see an emerging trend to use MuSE in very diverse applications and even for smaller systems. The user-friendliness of the modular joining of the systems is a very persuasive selling point for its usability. It makes things a great deal easier. So, for example, if you have an OP4510 simulator with its 128 I/O channels and need more I/O, with PCIe you could add an OP4520 expansion chassis and double the capacity to 256 I/Os–but what if you needed even more? The OP4510 is a fairly small box, so it can’t take more than one PCIe card. But now, using the four SFP ports in the front of the chassis, we can connect four I/O expansion chassis to that OP4510, and these chassis can be of any type–even bigger ones like the OP5607 with 256 I/O channels, increasing the I/O capability exponentially.”

IV: So you mean you can mix any type of OPAL-RT chassis with MuSE?

IP: “Indeed! As I mentioned, we were using SFPs to connect to third-party devices (amplifiers, MMC controllers), and already had SFP ports on all our newer systems. So the software we developed is compatible with all systems, which allows the user to build a network of simulators and expansion chassis of different types, depending on their requirements and budget. On our OP5707 simulator, for example, we have 16 SFP ports, so we can connect 16 OP5607 chassis, each with up to 256 I/Os. We have even recently upgraded the OP5600 family to add MuSE capability, with the new OP5650 just released, which has an Artix-7 FPGA and 4 SFP ports.”

IV: So really: up to 4,096 I/O connections? Have we ever done anything like that as far as you know?

IP: “Well, let’s run through some pretty exceptional simulation challenges, and it’ll give you an idea of the flexibility and expandability we can provide. Configurations like this simply weren’t possible with PCIe in a user-friendly way, if at all:

  • One of our clients needed to expand their OP4510 simulator (running with RT-LAB) to perform real-time control of power electronics and electrical machines on a test bench. In that case, the addition of four OP4200s clustered allowed them to run the control on a real-time calculator of the OP4510 while piloting the inverters and collecting data from measurements on the OP4200s;
  • Another client has two setups, one OP5707 with four OP4520s, and another OP5707 with four OP4200s, running with HYPERSIM. In this case, the remote units are located ~100 meters from the simulator, and the customer needs to use the optical fiber pre-installed in their building. We could not add real-time synchronization cables between the units, as we had done previously, and with this setup in mind, we integrated our real-time synchronization signal into the MuSE link, so that all communications are now over one single link.
  • Finally, one of our long term clients had bought several of our chassis (OP5600, OP4520, OP5707, OP4510) and wanted to reuse these for a larger project they were preparing. Adding the MuSE capability allows them to connect their OP4510s and OP4520s to the OP5707 to collect data from measurements, yet still be able to easily disconnect an OP4510 to use it as standalone chassis in other experiments when needed.”

IV: Well we’ve talked a lot about hardware, but you mentioned the software layer was a large part of the link. What exactly does it bring, in term of expandability?

IP: “Usability, both during model preparation and during execution of the simulation, was our main objective. In the past, we had developed a generic Xilinx Aurora interface that users could use to prepare FPGA bitstreams for our chassis. But managing the data packing/unpacking and the timing constraints of the protocol was cumbersome, and we understood that our users were obviously more interested in developing their control algorithms than tackling these low-level problems. Now the Aurora integration in the bitstream is completely taken care of by our RT-XSG toolbox during the generation of the bitstream.”

“And for the CPU model, we made it easy through the RT-LAB and HYPERSIM interfaces: the user has only to add remote units through the GUIs. The simulator’s bitstream will not need to be changed if the users need to use one more remote unit later on —everything is taken care of. Much more user-friendly, and a huge step forward!”

IV: Pretty exceptional, and we helped them all out. Okay, so what we refer to as MEA (More Electric Aircraft), or electric boats, for example–it’s not hard to imagine a scenario where we actually need 4,056 connectors! Given the increasing electrical/electronic complexity of some of the transportation and other electronic networks and infrastructures out there, that is.

IP: “We know, through the installations I’ve outlined to you above, that these setups work reliably. Knowing this and going forward, in terms of expandability, it’s more an issue of time, of latency—how much data can you transfer in a time-step? But you have the same considerations as PCIe. We start to hit the ceiling at some point–but that’s not a limitation distinct to MuSE, or imposed by OPAL-RT.”

IV: And in term of performances, do the optical fibers reduce latency?

IP: “Actually it is not the link itself which removes latency, but the fact that we no longer need to use one or more PCIe slots managed by the chassis’ motherboard, and different PCIe interconnection cards and cables with various chassis. We thus gained better control by using the Aurora link, and the only PCIe connection remaining is between the simulator’s motherboard and the simulator’s internal FPGA. The rest of the communication happens between the FPGAs of the various chassis directly. We also control the network enumeration, the detection of the number and type of chassis connected, etc.”

IV: In your experience, is this a highly specialized configuration for a certain type of user…?

IP: “Not at all, perhaps even the opposite. Before, when we had PCIe connections, the users didn’t have to worry about it, the motherboard did. So we wanted to have the same ease of use with the SFP and optical fibers. Generally, all the user has to do is cable the SFPs, and configure the simulator and its remote chassis in RT-LAB or HYPERSIM. The way the users interact in the model, it’s the same as with any other chassis–there’s nothing special to program.”

IV: So I understand this doesn’t just increase the I/O of the machine, but it also allows many other connection schemes. So this is not just a linear improvement in use, but an exponential one. Are there other developments coming?

IP: Of course! We achieved ease of use for the CPU model and the bitstream preparation. But we are now tackling other aspects of FPGA to FPGA communications required by our users: models distributed over multiple FPGAs, communication with power amplifiers, with MMC controllers, etc. I think we’re just in the first phase of this MuSE project, and there are plenty of interesting features still to come!

About the Interviewee

Irène Pérès joined OPAL-RT in 2001, and she has been involved in the development of OPAL-RT simulators, including software drivers, firmware and aspects of hardware management.
She is now a Product Manager and technical fellow and focuses on the evolution of OPAL-RT multi-FPGA platforms.
From her early career years (she received a Ph.D. in Plasma Physics from Paul Sabatier University–in Toulouse, France–in 1990), she retains a strong interest in challenging, complex technical projects, and strives to bring simplicity to these challenges.

eHS: The Nanosecond Power Electronics Solver Whose Time has Come

What is eHS: The Nanosecond Power Solver?

“The OPAL-RT electric Hardware Solver (eHS) is a powerful floating-point solver… that enables users to simulate an electric circuit on an FPGA automatically, without having to write the mathematical equations.”

“It combines the simplicity of building electric circuit models using the new OPAL-RT Schematic Editor, SimPowerSystems Toolbox, PSIM, the PLECS Blockset, or NI Multisim software with the strength of OPAL-RT FPGA-based simulators to solve the currents and voltages within the circuit in real-time, with a sample time below 1 µs.”

“The eHS solver uses modified nodal analysis. It solves a conductance matrix to find the voltage at each node of the circuit, and the current in each branch. The conductance matrix of the circuit is made independent of the switch control signals through the implementation of the Pejovic method representing the switch impedance. With this method, a conducting switch is represented as an inductance and an open switch is represented as a capacitance so the conductance matrix does not change during the simulation.”

–eHS User Guide

Commentary: Luc-André Grégoire, one of eHS’ Developers

eHS—our electrical/electronic Hardware Solveris like all simulation tools, but uses a smaller time-step, and so an FPGA is required. An FPGA may not be intrinsically any faster than a CPU, but since it’s dedicated to a sole task and closer to the hardware layer, it is able to act a great deal faster than a CPU can. We also use FPGAs to acquire I/O closer to where the computation is required, and this reduces electrical latency. eHS is like an impartial third party—it’s an electronics solver, but it can be run on various kinds of hardware: on RT-LAB, HYPERSIM, or on a National Instruments (NI) platform. 

Having said all that, yes: it takes an FPGA because of the much-reduced time-step. When we talk about eHS x128, that signifies sub-microsecond simulation up to about 125 states in 1 microsecond. That’s equivalent to 20 three-phase two-level inverters on a single FPGA with no delays (since it’s all within a single solver core)—and the ‘no delays’ part is obviously crucial when you’re working with extremely short timesteps. 

“In general, the more operators in the solver, the bigger and faster the circuit that can be solved. Additionally, the more operators in the solver, the more resources and hardware that are then required—and thus the footprint required by the resources for the solver is also larger. We’ve looked at clustering these FPGAs: for a Québec client, they were considering three FPGAs, with each of these having two times x128s. This is significant computing availability. 

What we used to do about a decade ago on CPU, since the time-step was larger, was to open and close a switch at a very precise time. Since the time-step can be larger than the time in which you want to close and open the switch, we interpolated between the time-steps to ensure that—although the time-step is much slower than the speed at which the switch is going on and off—we still get accurate results.  

As we started to work more on FPGA, we first thought, ‘this is a much smaller time-step; we no longer need to interpolate’. However, when we went to a smaller time step, customers told us they wanted to go with a much faster switching frequency. A higher switching frequency meant that we needed to be much more accurate on the pulse. So we re-introduced the interpolation, but now on FPGA.”

The interpolation tool was applied to CPU, and now we use the same kind of trick on FPGA—but we’ve modified it to reach a higher performance than it used to achieve, and to support more things. This ‘trick’ is similar to what our RT-EVENTS block set does, which, in turn, is part of the CPU-Based Electrical Toolbox in RT-LAB (and soon in HYPERSIM, as well)The intended use for this is converters switching at faster than 20 kHz, so resonant converters–or enhanced accuracy on the switching events.  

So if you have some cascaded H-bridges that are supposed to be phaseshifted, and although they’re switching at low frequency, you still need to be very accurate on the switching events: this could be an application. You could also talk about ultra-high frequency DC-DC converters using Ga(gallium nitride) devices, with maybe 100 kHz switching frequency—this eHS implementation is a solution to some of these problems. 

“You mentioned MMC in HVDC to me earlier, with mixed converters. Ultimately, eHS is for high-switching converters, and eHS is very good because it’s a fixed matrix solverbut there are some drawbacks with that topology. Converters that use multiple switches in series may not be the best candidates for eHS, but with the technology we’re proposing, we use a number of switches, and manage to get around that as well.  

What is genuinely new about this is that OPAL-RT was the first company to introduce a time-stamped bridge around 20 years ago. This method allowed sampling of a PWM signal at a much higher rate than the simulated model. This new technology is now available in our latest eHS solver, where a PWM signal can now be sampled at a rate of 200 MHz, or every 5 ns.  

Sampling of the PWM going up to 1 MHz, while maintaining 99.5% accuracy, can be achieved. (For comparison, in earlier versions of our software, PWM frequency was limited to 20 kHz to obtain similar fidelity.) Additionally, using our patentpending modelling approach, any 2-level or 3-level converter topology (fullbridge, NPC, T-Type, Pack-U cell, flying capacitors) can be represented, even during blocked or discontinued operations. 

OPAL-RT Award for eHS: Prix Innovation PME 2018 @ ADRIQ-RCTi Gala

We are very pleased to announce that OPAL-RT TECHNOLOGIES has received the prestigious Prix Innovation PME 2018 yesterday evening at the ADRIQ-RCTi Gala, Montreal, for its innovative product eHS–used by our most innovative clients all around the globe.

OPAL-RT TECHNOLOGIES would like to give a big thanks to all of our developers and researchers, and to all of those who worked actively on the product. Special kudos to Sébastien Cense, Tarek Ould Bachir, Luc-Andre Gregoire and Christian Dufour who have worked on eHS since its creation.


Traveling Wave Relay Testing

When we last spoke with Shijia Li, in November, she told us about Protection Relay Testing. She has since been made team leader for Protection and Smart Grid team within OPAL-RT’s AXES (Application, eXpertise and Electrical Simulation) division. This time, she is speaking to OPAL-RT Product News about OPAL-RT’s HIL Traveling Wave Test System.

Interviewer (IV): “Hello Shijia. First, can you tell us when we introduced the Traveling Wave test system?”

Shijia Li (SL): “We developed it about 1.5 years ago.”

IV: “Our software has been simulating faults on FPGAs for a while; why hadn’t we used this method previously?”

SL: “Previously, the FPGA had not been used for protection system testing. It was used for simulating power electronics devices, or motors or drives, but not to simulate a power system with transmission lines, etc. This was the first time we’d tested the power system components on the FPGA; it was a new way to use the FPGA. It has the fast time step required to precisely locate (within a few meters) and diagnose faults on power system lines.”

IV: “So prior to that, all protection was run on CPU? How did we make this breakthrough?”

SL: “This actually came about because of a request from a client. They built a device containing an algorithm and needed a way to test it. The conventional tests [Editor’s note: CPU-based] wouldn’t work with their device, so we had to use an FPGA model to achieve a much smaller time step. We had an engineer developing a model–more of a mathematical model–to make it run much faster on an FPGA. That innovation also prompted us to improve our solver. The client’s engineers were so impressed with the results from our constant parameter (CP) line model that they’re eager to see our frequency dependent (FD) line model.”

IV: “So we currently have two different line models?”

SL: “As of now, we only have the CP line model, but our R&D department is finalizing the FD line model.”

IV: “What’s the difference between the two models?”

SL: “The FD line model more accurately represents overhead lines than the CP model. It has a richer harmonic content, which represents with higher fidelity a line during a fault; with the FD line model, we’ll be able to test single-ended TW fault locating algorithms, which is more challenging.”

IV: “Impressive. This is a fairly new innovation, then, FPGAs being used to do work this precise, in this context?”

SL: “Yes. The travelling wave is a very high-frequency phenomenon, so it requires faster simulation as well as faster hardware. Our usual I/O boards take one sample every microsecond, which is sufficient for simulations in the range of 10 to 50 µs, but when simulating the travelling wave phenomenon at 500 ns on the FPGA, we need I/O boards that can follow at this speed, to get better accuracy. Fortunately, we already have a board with a sampling rate of 2 MS/s.”

IV: “What did utilities do before this? Did they simply say, ‘there’s a fault somewhere between kilometre 364 and 365’, for example?”

SL: “We could say that. There are other ways of detecting the fault location that are not as accurate as this one; it really depends on the manufacturer. It is, for example, often expressed in a percentage of the setting, which relates to the length of the line, so the longer the line, the lower the accuracy.”

IV: “This was a breakthrough in terms of narrowing the range?”

SL: “Yes, absolutely. And this idea was floated a long time ago, but, at the time, the relay itself didn’t have enough computational power. The processor wasn’t fast enough to run the algorithm. But since technology has evolved, they can implement it on the hardware. To understand how much of a breakthrough that is, you have to look at it in the operational context. When there’s a fault on a line, there are some strategies that can be used to avoid sending out a team to investigate. These strategies vary from one utility to the next and are based on the environment around the fault location. Since there aren’t cameras everywhere, some assumptions must be made.”

“For example, in rural areas, some might successively reclose and reopen breakers to try and clear a fault (in case a tree fell on a line, for example) to liberate the line. If every automated strategy fails, or, in dense urban areas, it is most often necessary to send out a team to investigate, which can be very costly. If the team has to search over a few kilometres for the location of the fault, it can take a lot of time. It’s even more difficult, for obvious reasons, with cables that are buried underground. Travelling wave relays might mean a high-cost reduction in many cases. This is the breakthrough.”

IV: “This is an HIL process, right?”

SL: “Yes, this is a hardware-in-the-loop (HIL) process, but obviously not on the lines themselves. There are testers that can be used in the field to perform simple signal injection tests. But what we’re doing is more in the lab: we’re using the same devices, the same settings. The device we are testing is monitoring the line. What we’re doing is we’re replacing that actual power line with our simulator: we send signals to the device, but the device is monitoring the lines on the simulator.”

IV: “And the larger context for this is control and protection, one of your specialties. Was travelling wave testing something people had wanted to do for a while?”

SL: “Well, it is not a new idea, but it’s not that long since it has actually been put into use. It is a new feature, and we do have customers expressing interest in it. Generally speaking, in the context of the protection industry, this would be considered an innovation: it’s not been widely used or adopted by most utilities. And we’re seeing some of our clients out there in the early stages, trying to convince people to adopt this technology.”

IV: “How does this technology fit in, in terms of the industry in general?”

SL: “The protection and control sectors are very well-established, mature sectors or fields, within the power system industry. The current devices and schemes or implementations we have are good enough to protect most power systems. For now, there are some new perspectives—the broader introduction of renewable energy—that may introduce some new challenges. And travelling wave technology, which brings a challenge in terms of testing: this is ultimately why we’re developing this solution.”

“The microgrid also, and its protection, is a very hot topic in this field. Other than that, from the communication-aided protections: we use more and more fibre optics, so that’s something we could test as well. And that brings us to the IEC61850 digital substation concepts [Editor’s note: Product News blog post to come]: so let’s say, with one relay now, we can do a lot of complex functions and so, of course, the testing becomes exponentially more complex as well.”

IV: “Thanks for speaking with us again, Shijia.”

About the Interviewee

Shijia Li received her Bachelor’s degree from Zhejiang University, China in 2012 and Master’s degree from McGill University, Canada in 2015, both in the field of power engineering. She joined OPAL-RT in March 2015, where her work focuses on power system modelling and real-time simulation applications with protective relays and PMUs. Shijia is actively involved in developing technical solutions and providing advanced training to help users better utilize real-time simulation techniques for exploring the latest P&C/smart grid technologies. Currently, Shijia leads the Protection and Smart Grid team in OPAL-RT’s AXES (Application, eXpertise and Electrical Simulation division).

Libraries for HVDC & Modular Multilevel Converters (MMC/SM)

“The Modular Multilevel Converter (MMC) system offers many advantages over conventional voltage source converters and in DC power transmission, microgrid or renewable energy applications. MMC’s distinctive topology provides a wide variety of new features, necessitating the use of a sophisticated controller for extra control requirements.”

For this article, we spoke with Wei Li, currently working in OPAL-RT’s AXES (Application eXpertise and Electrical Simulation) department, with the team focusing on HVDC tools. He is working on developing MMC (modular multi-level converter) modelling tools for those simulating this converter type and mixed topology.

Interviewer (IV): “Hello and thanks for speaking with us. Would it be right to say, Wei Li, that HVDC and MMC are something that the market wants right now?”

Wei Li (WL): “Yes, MMC (Modular Multilevel Converter) is a technical term for a type of converters, whether they’re from AC to DC systems or vice versa. Normally we say they’re a subset of HVDC—which itself is kind of a link to connect two AC systems. MMC is looking like the future of HVDC technology. There are currently several huge engineering projects commissioned using this MMC technology for HVDC.”

IV: “So why is the market necessarily ready for this now?”

WL: “Because it has several technical advantages–people have been doing their research. And now, people are using these technologies to build. Compared to previous technologies, it’s a bit difficult, and there are several challenges for the controller. For our systems, we provide real-time Hardware-in-the-Loop (HiL) simulation and test benches. The main function of the HiL real-time test bench is so that these manufacturers can validate their controllers before they commission them for real in the field. Especially when they test high voltage, high power–it’s hard, and not necessarily what you want to start with, to test in the field.”

“Five years ago it was difficult, but now when they view the success of the current HVDC projects, they will choose to test their controllers in our systems. This has several advantages for them: the first advantage is that they test at an earlier stage—for prototyping and before there’s even hardware involved. Second, even if you’ve already built your controller, it’s expensive, potentially dangerous, and you can’t test all the scenarios—so being able to prototype-test is another advantage. You can’t test fault scenarios on purpose, because you could interrupt service or damage equipment, among other things. And you have to know your controller or protection system, for example, is going to work well in the worst case scenarios—this is another prototype-stage testing advantage.”

“We already have several HVDC MMC projects under our belts, and they’ve already used our models and simulators as hardware test benches. So we’ve been doing this for several years, and it’s been very successful. But as the technology evolves, for real systems, users have tried other, different technologies. Because of this, our real-time simulator has to follow the trends going on in the markets. So in China, for example, the bid for a contract had already been made by several manufacturers, and now it’s being built next year. With MMC, there are hundreds of modules stacked up for high voltage usage. Before, in one system or one project, all modules would have been the same: either all half-bridge or all full-bridge. And now they’re mixed for some technical advantages. People might have had difficulty simulating these before. Maybe they had a workaround, but it wasn’t straightforward.”

“Now what we do in our models, we can have mixed submodule types: 80% one, 20% the other, etc. So to replicate these scenarios—because in the real world it’ll be mixed types–they have to be able to simulate the mix and to change the ratio of the mix at will. The behaviour of the system during the fault is new and different, and we need to be able to simulate that. So when they design, they test various scenarios to see which is optimal.”

“We already have this new feature, we’re going to announce it, to tell people we can do this. For these new features, people can test mixed modules in one project, and during the simulation, you can change the ratio of modules in a simulation and see what changes, and validate the design based on those changes. Eventually, it ends up with the controller at a hardware-in-the-loop (HiL) test bench.”

IV: “It sounds like we already have this product or test bench?”

WL: “We have a library called MMC in both eFPGASIM and HYPERSIM. Each box can have hundreds of modules. We support both platforms, and we use hardware and software, and now we have full MMC libraries. We knew how we had to do it, but we now have a product ready–a test bench–that is MMC/mixed SM capable that deals with the advances the market has made.”

“So it’s more like a new feature announcement. I’ve been working here for more than 10 years, and I have an electrical engineering background. My specialty is power systems, and when I joined, we had the EMS (electromagnetic systems) team: we dealt with special needs for customers with power system modelling, but at that time it was not specific to MMC modelling. As the team grew, each team member started to be responsible for various specific areas. This is maybe 6-7 years ago. Now my team has joined AXES. Now we’re the FACTS & HVDC team.”

IV: “How do we hear about these new developments in the market, as the market starts to make technological advances?”

WL: “First, we closely watch what the markets need. We try to understand what are the research topics that are current in academia, then we write and read papers on these topics; we go to conferences; we keep our ear to the ground. For this particular topic, it’s a hot topic. There are several MMC projects going on around the world, and many conventional HVDC projects. We have close contact with those utilities and manufacturers, so we’re pretty easily able to stay current.”

“With the industry people, we try to understand what is their planned project. For academic people, we want to see what they’re researching. And understand it into the future by 5-10 years, or however far as we’re able to. Now most MMC is for HVDC use, and STATCOM is another device that provides reactive power. So we also provide the solutions, the tools, etc., for academia, because besides the obvious benefits, it lets us see into the future a little bit with the research topics.”

“When we provide real-time simulation solutions, the first try is always to use the general method: to build the models using the various basic components in the power systems. These theories have existed for many years. When people have to validate their controllers of MMC in real time, this becomes complex and difficult. Thousands of modules and we have to simulate it in real time. It’s almost impossible using conventional simulation methods. We have to look at this specific circuit, do optimization in the model, and do research ahead of time. This is another way to stay ahead of the market. Then we can go beyond conventional methods because we’re already optimized for the mixed topology. Then people will say, ‘I want to build different topologies’ in the field. So when they do this, we have to make it possible for them to simulate all their changes in real time.”

IV: “It also sounds like when we get huge projects, everything scales so much. Is that what you mean?”

WL: “Yes, because there are so many components and they all work together, and they have to do real-time simulation, which is fast. This is why we came up with an optimized way to do it, and we can do it in real time, and with much better resolution and detail. You have to optimize and look at specific operations when you get scaled all the way up. When people come to you and tell you there are mixed modules, you know you have work to do. Luckily we have great R&D people and customers who’ve identified their needs. When they start to take the bids and make the systems, we’ve already done the background work, so we’re caught up and ahead of the curve.”

“Different customers have different needs. So some customers will build HVDC, and they have to validate their controller with HiL. So when they design their systems, the topology, etc., they normally do it by themselves. And for commercial reasons, the design phase is usually confidential. Once this system is ready, they need some HiL: test benches that can precisely simulate their system. We know what they want, we prepare the model, and we have to meet their requirements to simulate their system in real time. Rarely, if we can’t do it right away with available tools, we have to figure out a solution, optimize a model, or develop something. So our main goal or point of success is to simulate their system in real time. We need a solution that is not only fast enough but with enough detail and accuracy–these are the three key points.”

“reaIn academia, it’s a little bit different, they try to figure out what are the possibilities, sometimes with very creative designs. They’ll work with us, it’s usually more iterative, and they may want us to provide some models, or to try something new. But a complete test cycle for them, as I understand it, is they design the system and controller using offline simulation; then they will try to build scaled-down equivalent systems, with the controller that is directly downloaded to a real-time simulator. This is rapid control prototyping (RCP)–it’s only a prototype controller. So during this cycle, they try to optimize their parameters and validate that the controller works. And then before the controller goes to the field, they’ll hook it up to a simulator. We call this HiL. And then finally they bring the controller to connect to the actual device.”

IV: “And we have products in all those categories, right?”

WL: “Exactly. From the beginning, our products provide offline simulation and then fully digital real-time simulation. Then our real-time simulator can be used either as a prototype controller or as a plant. As a prototype controller, we have all this control logic downloaded to a real-time simulator and connected to some hardware device to control. As a plant, it’s used not only for steady-state systems but also if not mostly to simulate the behavior during faults. We have a specific type of testing mode that is really good for customers that use two independent simulators: one acts as the controller, one as the plant. There is no closer way to simulate the real-life behavior, with noise in I/Os and delayed latency.”

“Considering this very important thing that are I/Os is essential as they can cause trouble in many cases. So testing all this together in the same simulation adds more certainty, it saves money, it solves problems ahead of time, and with HiL you can easily do many tests in one day—when you finish one test you just reset the model to steady-state and test again. This can be done within only a few seconds. In the field, you need to de-energize all the equipment and start again. For the first multi-terminal HVDC in the world, they had to do 650-1,000 tests before they commissioned it into the field in order for it to be viable.”

“You can imagine how, if you do so many tests, and if each day you can save some time, you can save an awful lot of time. And time is money in this field for sure. In China, they do these tests within one year. And if they do it in the field, 1,000 tests, at a rate of one a day, it would be three years. But they do all this in a few months. And they do each test several times.”

IV: “The scale of all that is just staggering.”

WL: ”Yeah, it’s literally awesome. And we’re pleased to be able to help people out.”

IV: “Wei Li, thanks for speaking with us today.”

About the Interviewee

Wei Li
received his B.Eng. degree from Zhejiang University, China, and an M. Eng. degree from the National University of Singapore, as well as a Ph.D. degree from McGill University, Canada. He is a Senior Power System Simulation Specialist with OPAL-RT Technologies, Montréal. His fields of interest are in power electronics, renewable energy, and distributed generation. His current research focuses mainly on real-time simulation and controls of modular multilevel converter HVDC systems and FACTS devices.


OPAL-RT Introduces Nanosecond Power Electronics Solver on HYPERSIM®

OPAL-RT Introduces Nanosecond Power Electronics Solver on HYPERSIM® 

OPAL-RT introduced two new offerings for PES and CIGRE 2018 in late-Summer/Fall, 2018: our nanosecond power electronics solver, eHS, on HYPERSIM and our first cloud offering, HYPERSIM on Demand, for full-featured EMT simulation on our premiere platform before going to real time.  

(A forthcoming blog post will talk about HYPERSIM on Demand—stay tuned!) 

We spoke with Simulation Specialist Weihua Wang, from OPAL-RT’s R&D organization and an invaluable connection with our Asian sales offices, about why both OPAL-RT and the market are excited about this new Power Electronics on HYPERSIM introduction. 

Interviewer (IV):  

Weihua, it’s nice to meet you. Your name comes up again and again in R&D conversations. So: can you speak briefly, please, about what is so exciting about this new product launch? 

Weihua (WH): 

“Well, we’ve just launched eHS on HYPERSIM, which now makes HYPERSIM much more than just a real-time power system simulator. It was already incredibly strong, scalable and versatile, and a market leader. So, it’s only made a great thing even better–and now it also does real-time power electronics. We’ve seen exponentially more power systems over the last few years equipped with power electronics converters at the low to medium voltage levels. So, when we think about the power converters used for renewable energies, microgrids, or HVDC transmissions, for example, all these contexts can benefit from faster power electronics simulation.” 

IV: “How exactly does FPGA simulation help our customers? 

WH: “As I was saying, we’re seeing more power electronics implemented in power systems, for starters. At the same time, converters’ switching frequencies have become higher, and these high-speed switching events are beyond the simulation capabilities of a CPU platform alone. We already had power electronics simulation on FPGA  in our catalog (known as eHS to OPAL-RT clients), and now we’ve brought this core strength to HYPERSIM, so we get the best of both worlds, really. HYPERSIM, augmented with an FPGA (and with its existing CPUs), can now simulate high-frequency phenomena with astonishing time precision and hence electrical accuracy—and all of this within the context of really very large power systems.” 

“So, there are these breakthroughs, which I think both our new and existing customers will find exciting. But there’s also usability, interface, and workflow improvements. The new Schematic Editor removes the need for 3rd party modelling software. And HYPERSIM has all the necessary tools in one place, under one roof as it were: report generation, automatic task splitting to use all available cores, an advanced digital scope with built-in analysis functions, an API, and more, all of this for real-time HIL and test automation, power electronics/power systems, etc. Everything is all there in one suite in HYPERSIM.”  

“Before, if we had simulated a large high-speed power system/power electronics environment, we’d have to interface with MATLAB, and we had at least three files, two environments, and two real-time simulators. Now we need just a single HYPERSIM project, a single environment and a single simulator. When we used to co-simulate, we’d have to have eHS on RT-LAB and HYPERSIM running separately, we’d start things running with scripts, we had some tricks—and it worked, but it was an elaborate workaround. And now we have everything in one box, so to speak.” 

IV: “Weihua, could you give a real-life example where eHS on HYPERSIM—the porting of our nanosecond power electronics solver to our premiere platformhas enabled a previously unachievable goal for our customers?” 

WH: “We can now simulate from DC up to 100 kHz, and we also simulate generation, transmission, pretty much the whole architecture of power systems, from the lowest level—your home—to the highest, up to 750 kV or more than 1,000 kV; HYPERSIM with eHS vastly expands the spectrum of what we can simulate.” 

“Previously, the solutions for power electronics, in general, were unclear in HYPERSIM—it always excelled as a tool for power systems. It was also okay for LCC, which was mature, and the first generation of HVDC converters. But for the newer topologies, e.g. the new VSC-based converters, we didn’t have clear solutions. We were either using average models which were sufficient from a high-level point of view—with PQ and the general dynamics—or custom solutions for the more demanding projects such as MMC converters…” 

“…But we could not simulate amplification of harmonics or harmonic penetration. So, when we’d study the complex dynamics of renewable energy interfacing with traditional power systems, the solution was great, but it wasn’t perfect. With the advance of eHS in this new release, we’ll considerably strengthen this part. You know, we’ve innovated with the eHS solver—it was a big breakthrough six years ago—at that time, no one had thought about simulating power circuits on FPGA. It’s been very well-accepted by many customers, but because of previous limitations, when people wanted to simulate both renewable energy and huge power systems, we had to rely on the MATLAB compiler, which was and is still incredibly slow compared to HYPERSIM.“ 

“When clients had these particular needs in the past, they used to have to do co-simulation with the eHS model in MATLAB and the power system in HYPERSIM, both connected by an optical fibre link—the workflow wasn’t great, we were doing it, but the workflow required case by case customization. We had to consider many issues such as incompatibility between software versions and delays over the optical fibre link. 

IV: “In your opinion, what does bringing FPGA simulation to HYPERSIM represent for this product? 

WH: “The introduction of our nanosecond power electronics solver to HYPERSIM is intended for an audience and a sector that’s growing hugely: all of the renewable energy technologies, all of the power system and power electronics users. Now we have another processor more suitable for power electronics simulation—the FPGA. We can save a lot of resources with the FPGA—the speed of it—and then use the CPU for the simulation of AC or slower dynamics systems. So, we expand our skill base, our strengths, and our resource usage, as well as the scalability. And we replace a lot of manual or customized work with a stronger, more generalized platform that meshes perfectly with what the market is looking for now. It’s really a win-win situation for customers.” 

“The FPGA itself represents another level of integration of OPAL-RT products. Across our product line, we had strong, world-class products that each addressed different needs. Historically, each one of the applications did one thing very well. And now we see all our strengths merging together, showing a dedication to a more diverse, demanding audience. We’ve married our former strengths to our new, current strengths and come up with some exciting hybrid products. 

IV: “In closing, what do you think this product and platform innovation means for the industry?” 

WH: “In the electric power systems industry, it obviously evolves as with any other industry. In the 1960s, there wasn’t much DC; everything was AC. Then in the late 1960s and early 1970s, traditional HVDC systems entered the picture—and HYPERSIM emerged from that background. HYPERSIM has obviously evolved a great deal from that date on an ongoing basis. 

“But since this century started—the 21st century—we’ve seen exciting new energy sources and renewable and evolving sources. Before this year, there wasn’t any one complete solution that could simulate all the current power systems with great accuracy—there was no perfect tool, at a large scale. There were customized solutions, but nothing turnkey or practical at higher speeds, with larger networks. Our competitors all started to develop solvers on FPGA much later than we did—so we have a big, strong head-start in that sense. And now we find ourselves with deep knowledge of power electronics, unrivalled in the market. And eHS on HYPERSIM represents a breakthrough at that scale and at that speed, tackling today’s and tomorrow’s challenges.”

IV: “Weihua, it’s been a pleasure speaking with you, and I wanted to thank you for your time. Some very valuable insights being shared here, so thanks again.”

About the Interviewee:

Weihua Wang
was born in Beijing, China in 1982. He received his M.Sc.E degree from the University of New Brunswick, Canada in 2007. He joined OPAL-RT technologies in 2009, and is currently working as a simulation specialist as well as the chief representative of the Asia-Pacific Technical Center.

Since 2010, Weihua has been involved in the development of real-time models for MMCs. He developed and commissioned eight VSC-HVDC Hardware-in-the-loop test benches in the past five years. He also participated in, or supervised the factory acceptance tests of, control and protection systems using real-time simulators for all five MMC projects in China using OPAL-RT simulators: Nanhui in 2011, Nan’ao and Zhoushan in 2013, and Xiamen and Lu-xi in 2015. He is currently participating in a project with CEPRI to build one of the largest HIL test benches, in order to simulate the main backbone of the Chinese State Grid, which includes more than 18 LCC or VSC-based HVDC links.

His fields of interest are real-time simulation of VSC-based HVDC systems, DC distribution networks and smart grids, as well as complex medium voltage drives.