> For the complete documentation index, see [llms.txt](https://www.opal-rt.com/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.opal-rt.com/docs/rt-lab/model-and-simulation/co-simulation/connect-real-time-simulations-over-fpga-optical-links.md).

# Connect Real-Time Simulations over FPGA Optical Links

## Introduction

Real-time simulations running on separate systems can exchange data directly through high-speed optical links connected to their FPGA resources.

Each simulator executes its own model. Data to be exchanged is transferred from the simulation to the local FPGA, serialized and transmitted through an SFP optical interface, then received and processed by the FPGA in the remote system.

OPAL-RT FPGA-based systems support **Generic Aurora communication** through their SFP interfaces. Generic Aurora can be used to exchange data with another OPAL-RT system or a compatible third-party FPGA device. Depending on the hardware platform, configurable communication rates up to 5 Gbps are supported.

This architecture provides a dedicated, high-speed communication path without relying on an IP network.

## Architecture

A typical configuration consists of two independent real-time simulation systems.

Each system includes:

* an OPAL-RT real-time simulator;
* an FPGA with an available SFP interface;
* compatible FPGA firmware;
* an SFP optical transceiver; and
* a fiber-optic connection.

The communication path is:

**Simulation A → FPGA A → SFP → Optical fiber → SFP → FPGA B → Simulation B**

and, for bidirectional communication:

**Simulation B → FPGA B → SFP → Optical fiber → SFP → FPGA A → Simulation A**

The models remain independent simulations. The FPGA optical link provides the communication path between them.

### Fiber-optic connection

Use duplex optical fiber compatible with the selected SFP transceivers.

The supported distance is determined by the complete optical link, including:

* SFP transmitter power;
* receiver sensitivity;
* wavelength;
* fiber type;
* fiber attenuation;
* connector and splice losses; and
* operating line rate.

OPAL-RT delivers configurations with multimode duplex fibers and lengths up to at least 30 m, but does **not establish a universal maximum distance for Generic Aurora**.

For longer links, validate the SFP modules, fiber infrastructure, optical power budget, and simulator compatibility before deployment.

## Design the Data Exchange

Before configuring the optical link, define the data exchanged between the simulations.

For each exchanged item, determine:

* direction;
* data type;
* size;
* update rate;
* required latency;
* validity behavior; and
* fallback behavior.

Both endpoints must use compatible data representations.

### Define the Data Frame

Generic Aurora provides the communication transport. The application-specific data structure must be defined in the FPGA implementation. [\[OP5707XG2...escription | Confluence\]](https://opal-rt.atlassian.net/wiki/spaces/PHDGD/pages/1590100131/OP5707XG2+System+Description), [\[OP4300 - S...escription | Confluence\]](https://opal-rt.atlassian.net/wiki/spaces/PHDGD/pages/1308591775/OP4300+-+System+Description)

Depending on the application, a frame can include:

* header or synchronization information;
* sequence number;
* timestamp;
* payload length;
* status or validity information;
* application data; and
* integrity information.

When communicating with a third-party FPGA device, the data structure expected by that device must be known before generating the OPAL-RT FPGA firmware. [\[Understand...ledge Base | PDF\]](https://opalrttechnologies104.sharepoint.com/sites/opalrtdynamics365/incident/RT-XSG%20FPGA_IO_Aurora_8b10b%20Block_B06EC16819A8F01194376045BDFA0885/Understand%20Better%20SFP,%20the%20Aurora%20Protocol%20and%20How%20to%20Design%20FPGA%20Firmware%20with%20Such%20Capabilities%20_%20OPAL-RT%20Knowledge%20Base.pdf?web=1)

### Define the Update Rate

Determine when data is transmitted.

For example, data can be exchanged:

* every simulation step;
* every specified number of steps;
* when new data becomes available; or
* following an FPGA trigger.

The FPGA communication rate does not have to be identical to the CPU simulation time step.

A documented OPAL-RT implementation, for example, triggered SFP communication every 5 μs when new data was available while the CPU simulation operated with a 50 μs time step. [\[CC-N/A - R...interface | Confluence\]](https://opal-rt.atlassian.net/wiki/spaces/SUP/pages/2651160617/CC-N+A+-+Regatron+G5+amplifier+tests+with+the+Aurora+SFP+interface)

***

## Configure the FPGA Communication

### 1. Configure the Aurora Link

Select an Aurora line rate supported by:

* both FPGA endpoints;
* both SFP transceivers; and
* the generated FPGA firmware.

Depending on the OPAL-RT hardware platform, documented Generic Aurora configurations support rates up to 5 Gbps. [\[OP5707XG2...escription | Confluence\]](https://opal-rt.atlassian.net/wiki/spaces/PHDGD/pages/1590100131/OP5707XG2+System+Description), [\[OP4300 - S...escription | Confluence\]](https://opal-rt.atlassian.net/wiki/spaces/PHDGD/pages/1308591775/OP4300+-+System+Description)

Both endpoints must use compatible physical-link settings.

### 2. Add the Generic Aurora Interface

In each RT-XSG FPGA model:

1. Add the appropriate Generic Aurora communication block.
2. Select the SFP channel.
3. Configure the line rate.
4. Configure the required Aurora interface parameters.
5. Connect the transmit interface.
6. Connect the receive interface.

The `GenericHighSpeedComm_Aurora_8B10B` interface exposes the transmit and receive data, validity, readiness, and frame-boundary signals required to implement the communication path. [\[Understand...ledge Base | PDF\]](https://opalrttechnologies104.sharepoint.com/sites/opalrtdynamics365/incident/RT-XSG%20FPGA_IO_Aurora_8b10b%20Block_B06EC16819A8F01194376045BDFA0885/Understand%20Better%20SFP,%20the%20Aurora%20Protocol%20and%20How%20to%20Design%20FPGA%20Firmware%20with%20Such%20Capabilities%20_%20OPAL-RT%20Knowledge%20Base.pdf?web=1)

Refer to the Generic Aurora block documentation for the complete parameter and port definitions.

### 3. Implement Data Packing

Convert the values transmitted by the simulation into the data structure expected by the remote FPGA.

Define consistently:

* word order;
* byte order;
* data types;
* signed representation;
* fixed-point or floating-point representation;
* array order; and
* frame boundaries.

### 4. Implement Data Unpacking

At the receiving endpoint:

1. Detect valid incoming data.
2. Reconstruct the data frame.
3. Verify its structure.
4. Convert the received words to the required application values.
5. Expose those values to the FPGA model or CPU simulation.

If sequence numbers, timestamps, or validity information are included, verify them before using the received application data.

### 5. Generate the FPGA Firmware

Generate compatible FPGA firmware for both endpoints.

The two FPGA implementations must agree on:

* Aurora configuration;
* line rate;
* SFP channel assignment;
* frame format;
* data representation; and
* communication timing.

***

## Connect the Optical Link

### 1. Install the SFP Transceivers

Install validated SFP transceivers in the selected FPGA SFP ports.

Verify that both transceivers use compatible:

* speeds;
* wavelengths; and
* fiber types.

### 2. Connect the Fiber

Connect the optical fiber between the two systems.

For each link, document:

* simulator and SFP port;
* SFP transceiver model;
* wavelength;
* fiber type;
* connector type; and
* fiber length.

### 3. Verify the Link

Before exchanging simulation data, verify that:

* both FPGA endpoints detect the link;
* the Aurora channel initializes successfully;
* the required lanes are active; and
* no persistent link or alignment errors are reported.

***

## Configure the Simulation Models

Configure each RT-LAB or HYPERSIM simulation to exchange the required data with its local FPGA.

The interface typically performs two functions:

* transfers outgoing simulation data from the CPU to the FPGA; and
* transfers incoming optical-link data from the FPGA to the CPU.

Configure the required CPU-to-FPGA and FPGA-to-CPU interfaces according to the simulator and FPGA architecture.

Where appropriate, also expose:

* link status;
* received-data validity;
* sequence status;
* communication errors; and
* timestamp information

to the CPU model for monitoring and fallback handling.

***

## Timing Considerations

### Configure Time Synchronization

The FPGA optical link and simulator time synchronization serve separate purposes:

* **Generic Aurora** transfers application data between the systems.
* **Time synchronization** provides the simulators with a common time reference.

If the application requires timestamped measurements, event correlation, or comparison against absolute time, synchronize each simulator using an appropriate supported timing source.

For example:

* IEEE 1588 PTP;
* IRIG-B;
* 1 PPS; or
* a GPS-derived timing source.

Refer to **Time Synchronization Driver** to configure synchronization.

> **Important:** Do not assume that an active Aurora connection automatically synchronizes the absolute clocks of the participating simulators.

### Account for Link Latency

The end-to-end communication latency includes more than propagation through the optical fiber:

```latex
T_{\mathrm{link}} =
T_{\mathrm{CPU\rightarrow FPGA}} +
T_{\mathrm{FPGA,TX}} +
T_{\mathrm{fiber}} +
T_{\mathrm{FPGA,RX}} +
T_{\mathrm{FPGA\rightarrow CPU}}
```

The actual latency can therefore include:

* CPU-to-FPGA transfers;
* FPGA processing;
* buffering;
* serialization;
* optical propagation;
* deserialization;
* receiving FPGA processing; and
* FPGA-to-CPU transfers.

Measure the complete data path when determining whether the connection meets the application's timing requirements.

***

## Validate the Communication

Validate each part of the link before closing the complete simulation loop.

### 1. Validate Each Simulation

Run each simulation independently and verify:

* real-time execution;
* CPU-to-FPGA communication;
* FPGA-to-CPU communication; and
* FPGA firmware operation.

### 2. Test Known Data Patterns

Send known patterns across the optical link.

For example:

* constant values;
* incrementing counters;
* alternating bit patterns; or
* predefined numerical sequences.

Verify that the values received by the remote system exactly match the transmitted data.

An incrementing sequence number is useful for detecting missing, duplicated, or unexpected frames.

### 3. Test Dynamic Data

Transmit a predictable dynamic signal, such as a ramp or sine wave.

Compare the transmitted and received signals to detect:

* corruption;
* discontinuities;
* unexpected values; or
* timing inconsistencies.

End-to-end validation is important because an active physical link does not, by itself, prove that the application data is correctly transmitted. OPAL-RT support history includes an Aurora application where occasional unexpected spikes appeared in a transmitted dynamic signal. [\[Jayme Lee...clavardage | Teams\]](https://teams.microsoft.com/l/message/19:03b22ace-6e2a-4ce8-85b9-e9eea0217c9b_7416294d-7ba7-40d0-9866-0bcb26f34959@unq.gbl.spaces/1753915048331?context=%7B%22contextType%22:%22chat%22%7D)

### 4. Measure Communication Latency

Measure the time required for data to travel through the complete simulation and FPGA communication path.

If the simulations form a closed loop, include the return communication path when evaluating the total loop delay.

### 5. Close the Simulation Loop

Enable full bidirectional interaction only after verifying:

* reliable physical communication;
* correct data representation;
* stable update rates;
* acceptable latency;
* correct validity handling; and
* real-time execution on both systems.

***

## Handle Communication Failures

Define how each simulation responds when communication fails or remote data becomes invalid.

Consider cases where:

* the optical link is lost;
* a frame is missing;
* a sequence number is skipped;
* the received frame is invalid;
* data becomes stale;
* one FPGA resets; or
* one simulation stops.

Depending on the application, possible responses include:

* holding the last valid value;
* using a default value;
* rejecting stale data;
* placing the application in a safe state;
* opening the distributed control loop; or
* stopping the simulation.

For critical applications, consider including sequence numbers, timestamps, and validity information in the application data frame.

***

## Distance Considerations

Optical fiber enables substantially greater physical separation than typical local electrical interconnects, but there is no single Generic Aurora maximum distance.

The achievable distance depends on:

* SFP transceiver reach;
* wavelength;
* multimode or single-mode fiber;
* line rate;
* fiber attenuation;
* connector and splice losses;
* receiver sensitivity; and
* the supported FPGA configuration.

The OPAL-RT configurations retrieved for this documentation include 850 nm transceivers with multimode fiber and cables up to 30 m. [\[OPBQ6412-0...PALRT\_9-30 | PDF\]](https://opalrttechnologies104.sharepoint.com/sites/clt-AC-005556/Orders/CC-224472/Quote_spec_client/OPBQ6412-01_ANL_OPALRT_9-30.pdf?web=1), [\[Mod. 1 to...No. 229930 | PDF\]](https://opalrttechnologies104.sharepoint.com/sites/clt-AC-008512/Orders/CC-222976/Quote_spec_client/Mod.%201%20to%20Purchase%20Order%20No.%20229930.pdf?web=1)

These documented cable lengths **must not be interpreted as the maximum technical reach of Aurora**. Conversely, a longer-reach SFP specification alone does not establish that the resulting configuration is supported by OPAL-RT.

For longer connections, validate the complete FPGA, SFP, fiber, and line-rate configuration.

> **Important:** Generic Aurora is a direct serial link. It cannot be routed through conventional Ethernet switches, IP routers, VPNs, or the Internet.

## Limitations and considerations

* **Custom FPGA implementation is required:** The application is responsible for data packing, unpacking, framing, timing, and error handling.
* **Distance depends on the optical configuration:** Do not specify a supported maximum distance without validating the SFP modules, fiber, line rate, optical power budget, and FPGA platform.
* **Absolute-time synchronization is separate:** The optical communication link does not automatically provide a shared absolute-time reference.
* **Both endpoints must be compatible:** Aurora configuration, line rate, transceivers, frame format, and data representation must be compatible between the two systems.
* **Communication delay affects closed-loop applications:** Include the complete communication path when evaluating the stability and real-time performance of a distributed control loop.

## When to use this architecture

Use an FPGA optical link when:

* the simulation systems can be connected by dedicated optical fiber;
* low and predictable communication latency is important;
* the participating hardware provides compatible FPGA-based SFP interfaces;
* the required communication interface can be implemented in FPGA firmware; and
* the physical distance is supported by the validated optical configuration.

For simulation sites connected through routed institutional, research, or public networks, use **Geographically Distributed Real-Time Simulation (GD-RTS)** over an appropriate Ethernet/IP application protocol instead.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://www.opal-rt.com/docs/rt-lab/model-and-simulation/co-simulation/connect-real-time-simulations-over-fpga-optical-links.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
