Drop-Copy Gateway (pm-dc-gwy)

Learning objectives

After reading this page you will understand:

• what pm-dc-gwy does and why it exists alongside direct port-5557 access
• how to configure and start it
• the session lifecycle: HELLO → WELCOME → unsolicited DC_FILL → EXIT
• the DC1 wire protocol: every message type and field
• how pm-dc-gwy compares to pm-dc-spy and the DC relay in pm-alf-gwy/pm-alf-console
• what it deliberately does not do (no auth, no replay-by-sequence)

What this process is

pm-dc-gwy is a small TCP gateway that exposes the engine's drop-copy feed (DropCopyPublisher, ZMQ PUB :5557 — see Drop Copy) to plain TCP clients that cannot or should not speak ZeroMQ: a risk system, a clearing broker's ingestion pipeline, a compliance monitor, or any process on another host that only has a sockets library.

It is the TCP counterpart to pm-dc-spy: both connect a zmq.SUB socket to port 5557 and read drop_copy.event.<gateway_id> messages. pm-dc-spy prints them to a terminal for a human. pm-dc-gwy relays them as DC1 text lines to any number of concurrently connected TCP clients, each scoped to whichever gateway ID it asked for at connect time.

flowchart LR
    ENG["pm-engine\nDropCopyPublisher\n(PUB :5557)"]

    subgraph GWY["pm-dc-gwy  (TCP :5590)"]
        direction TB
        SUB["Internal SUB\n(refcounted per gateway_id)"]
        ACC["TCP accept loop"]
        SES["Session manager\n(HELLO/WELCOME · heartbeat · idle)"]
        RELAY["DC_FILL relay"]
        ACC --> SES
        SUB --> RELAY --> SES
    end

    subgraph External["External DC1 clients"]
        R1["Risk system\n(watching TRADER01)"]
        R2["Compliance monitor\n(watching TRADER01)"]
        R3["Clearing pipeline\n(watching TRADER02)"]
    end

    ENG -- "PUB :5557" --> SUB
    SES -->|TCP| R1
    SES -->|TCP| R2
    SES -->|TCP| R3

What this is not

pm-dc-gwy is deliberately minimal compared to pm-alf-gwy or pm-ralf-gwy:

Not supported	Why
Authentication / entitlement checks	Same scope decision as the raw port-5557 socket — see Drop Copy → Architecture. Any client may request any gateway ID.
Replay-by-sequence over the wire	DropCopyPublisher.replay() is in-process only with no external trigger — see Drop Copy → Replay. pm-dc-gwy does not attempt to expose it.
Order entry, quoting, or any command that mutates state	This is a read-only relay. Use pm-alf-gwy/pm-alf-console for trading.
Role/channel model (CLEARING/AUDIT/etc.)	That is RALF's DROP_COPY channel, a different mechanism — see RALF DROP_COPY channel vs. this feed.

A client simply says which gateway ID it wants in HELLO and receives that gateway's live fills for as long as it stays connected. More than one client may request the same gateway ID simultaneously — unlike pm-alf-gwy sessions, a pm-dc-gwy connection does not "occupy" a trading identity.

Prerequisites

• pm-engine running with its drop-copy publisher bound on :5557 (the default; see Drop Copy → Startup and shutdown).
• Optional: add a dc_gateway: section to engine_config.yaml to customise port and limits.

Configuration

pm-dc-gwy reads an optional dc_gateway: section from engine_config.yaml. You can generate this section with pm-config-gen --dc-gateway (all fields are optional and default to the values shown below), or hand-edit it directly:

dc_gateway:
  name: "dc-gwy01"
  bind_address: "0.0.0.0"
  port: 5590
  heartbeat_interval_sec: 5
  idle_timeout_sec: 30
  max_client_queue: 10000

Field	Default	Description
name	dc-gwy01	Process name echoed in WELCOME
bind_address	0.0.0.0	Network interface to listen on (127.0.0.1 for local-only)
port	5590	TCP listen port
heartbeat_interval_sec	5	Seconds between HB lines when no other outbound traffic
idle_timeout_sec	30	Disconnect after this many seconds of inbound silence
max_client_queue	10000	Per-client outbound line buffer capacity before the client is treated as slow and dropped

The drop-copy source address (tcp://127.0.0.1:5557 by default) is not configurable via YAML — it always comes from the same DROP_COPY_PUB_ADDR constant the rest of the system uses. Override it per-run with --engine-dc-pub (see below) if you need to point at a non-default engine address.

TLS

pm-dc-gwy does not terminate TLS. For remote deployments, put it behind a reverse proxy (nginx, stunnel, or similar).

Start the gateway

Installed mode:

pm-engine --verbose
pm-dc-gwy

Developer mode:

poetry run pm-engine --verbose
poetry run pm-dc-gwy

CLI override options:

Option	Default	Description
--bind ADDR	from config / 0.0.0.0	Override TCP bind address
--port PORT	from config / 5590	Override TCP listen port
--engine-dc-pub ADDR	from config / tcp://127.0.0.1:5557	Override the engine drop-copy PUB address this gateway subscribes to
--log-level	WARNING	Explicit level: CRITICAL, ERROR, WARNING, INFO, DEBUG
-v / --verbose	off	Increase verbosity (-v → INFO, -vv → DEBUG)
-q / --quiet	off	Reduce output to warnings/errors

Config file location

The engine configuration is read from <EDUMATCHER_DATA_DIR>/ref_data/engine_config.json. There is no --config flag: every process reads that one file, so none of them can be started against a configuration the others have not seen. Install one with pm-config-deploy.

Quick connect test

Use nc or telnet to validate the session lifecycle before writing any code:

nc 127.0.0.1 5590

Type the following line, pressing Enter:

HELLO|CLIENT=test|PROTO=DC1|ID=TRADER01

Expected response:

WELCOME|PROTO=DC1|GW=dc-gwy01|ID=TRADER01|HBINT=5|IDLE=30

Any fill published for TRADER01 on port 5557 now arrives as an unsolicited DC_FILL line. Type EXIT to close cleanly.

TCP is a byte stream

Never assume one recv() equals one line. Always buffer and split on \n.

Session lifecycle

sequenceDiagram
    participant C as DC1 Client
    participant G as pm-dc-gwy
    participant D as DropCopyPublisher (:5557)

    C->>G: TCP connect :5590
    C->>G: HELLO|CLIENT=risksys|PROTO=DC1|ID=TRADER01
    G->>G: subscribe drop_copy.event.TRADER01 (refcounted)
    G-->>C: WELCOME|PROTO=DC1|GW=dc-gwy01|ID=TRADER01|HBINT=5|IDLE=30
    Note over C,G: Session active — no further commands needed
    D-->>G: drop_copy.event.TRADER01 {fill payload}
    G-->>C: DC_FILL|SEQ=..|ORDER_ID=..|SYMBOL=..|FILL_QTY=..|FILL_PRICE=..|LIQUIDITY=..
    C->>G: PING
    G-->>C: PONG|TS=...
    C->>G: EXIT
    Note over G: TCP closed, subscription refcount decremented

Step 1 — Send HELLO

The first line must be a HELLO:

HELLO|CLIENT=risksys|PROTO=DC1|ID=TRADER01

Field	Required	Notes
CLIENT	Yes	Free-text label for logging
PROTO	Yes	Must be exactly DC1
ID	Yes	Gateway ID to receive fills for. Not validated against any allowlist — see the auth warning below.

On any other first line, or a HELLO missing CLIENT/PROTO/ID, the gateway sends ERR|CODE=AUTH_REQUIRED|... and closes the connection.

No authentication

Unlike pm-alf-gwy, the ID in HELLO is not checked against gateways.alf or any other allowlist. Any client may request fills for any gateway ID — this mirrors the lack of authentication on the raw port-5557 socket itself. See Drop Copy → Architecture.

Step 2 — Receive WELCOME

WELCOME|PROTO=DC1|GW=dc-gwy01|ID=TRADER01|HBINT=5|IDLE=30

The gateway subscribes this session to drop_copy.event.TRADER01 on its internal SUB socket (refcounted — sharing the ZMQ subscription across every connected session watching the same gateway ID) before sending WELCOME.

Step 3 — Receive fills

No further command is needed. Every fill published on drop_copy.event.<ID> for the requested ID arrives as an unsolicited DC_FILL line for as long as the session stays connected.

Step 4 — Disconnect

Send EXIT for a graceful close, or simply close the TCP connection. The gateway decrements the internal subscription refcount and, if no other session still wants that gateway ID, unsubscribes from the topic.

Protocol reference

All messages follow the same line format as ALF/RALF/CALF: MSGTYPE|FIELD=VALUE|FIELD=VALUE\n. DC1 is intentionally the simplest of EduMatcher's TCP protocols — there is exactly one thing a client can ask for (fills for a given gateway ID), so there is no subscription grammar, no role model, and no replay-by-sequence.

Client → Gateway

Message	Fields	Notes
HELLO	CLIENT, PROTO, ID	Must be the first line. See Step 1.
PING	—	Gateway replies PONG
EXIT	—	Graceful disconnect

Gateway → Client

Message	Fields	Trigger
WELCOME	PROTO, GW, ID, HBINT, IDLE	Reply to a valid HELLO
DC_FILL	SEQ, ORDER_ID, SYMBOL, FILL_QTY, FILL_PRICE, LIQUIDITY	Unsolicited, one per fill for the session's gateway ID
PONG	TS	Reply to PING
HB	TS	Periodic heartbeat when no other outbound traffic
ERR	CODE, DETAIL	See Error codes

DC_FILL fields map directly from the drop-copy payload — see Drop Copy → order.fill event for the full source payload and field semantics. SEQ and LIQUIDITY (MAKER/TAKER) come from the drop-copy envelope; DC_FILL does not carry order state fields like REMAINING/STATUS.

> HELLO|CLIENT=risksys|PROTO=DC1|ID=TRADER01
WELCOME|PROTO=DC1|GW=dc-gwy01|ID=TRADER01|HBINT=5|IDLE=30
DC_FILL|SEQ=42|ORDER_ID=ORD-001|SYMBOL=AAPL|FILL_QTY=100|FILL_PRICE=150.05|LIQUIDITY=TAKER

Error codes

Every error arrives as ERR|CODE=<CODE>|DETAIL=<message>.

Code	When it occurs	Connection kept?
AUTH_REQUIRED	First line was not HELLO, or HELLO was missing CLIENT/PROTO/ID	No — closed immediately
BAD_MESSAGE	Empty line, non-UTF-8, unparseable line, or line > 4096 bytes	Yes (or No if unrecoverable — see detail)
SLOW_CLIENT	Outbound queue exceeded max_client_queue	No
IDLE_TIMEOUT	No inbound traffic for idle_timeout_sec	No

Minimal zero-dependency client (Python)

import socket

def dc_connect(host: str, port: int, gateway_id: str, client_name: str = "risksys"):
    sock = socket.create_connection((host, port), timeout=5)
    buf = bytearray()

    def send(line: str) -> None:
        sock.sendall((line + "\n").encode("utf-8"))

    def recv_line() -> str:
        while True:
            nl = buf.find(b"\n")
            if nl >= 0:
                line = bytes(buf[:nl]).decode("utf-8", errors="replace")
                del buf[:nl + 1]
                return line
            chunk = sock.recv(4096)
            if not chunk:
                raise RuntimeError("gateway closed connection")
            buf.extend(chunk)

    send(f"HELLO|CLIENT={client_name}|PROTO=DC1|ID={gateway_id}")
    line = recv_line()
    if not line.startswith("WELCOME"):
        raise RuntimeError(f"handshake failed: {line}")

    return sock, send, recv_line


sock, send, recv_line = dc_connect("127.0.0.1", 5590, "TRADER01")
while True:
    line = recv_line()
    if line.startswith("DC_FILL"):
        print(line)

TCP is a byte stream

Never assume one recv() equals one line. Always buffer and split on \n.

When to use pm-dc-gwy vs. the alternatives

Scenario	Best choice
Ad-hoc inspection from a terminal, human-readable or JSON	pm-dc-spy
External process, any language, no ZeroMQ dependency allowed	pm-dc-gwy
Process already has ZeroMQ / pyzmq available	Connect directly to port 5557 — see Drop Copy
You already have an ALF trading session open and only want your own fills	DC\|STATE=ON on that session — see ALF TCP Gateway → DC
Role-gated (CLEARING/AUDIT) trade feed with replay-by-sequence	pm-ralf-gwy — see Post-trade

Troubleshooting

Check whether the port is in use

macOS:

sudo lsof -iTCP:5590 -sTCP:LISTEN
netstat -an | grep LISTEN | grep 5590

Linux:

ss -tlnp 'sport = :5590'
sudo lsof -iTCP:5590 -sTCP:LISTEN
netstat -tlnp | grep 5590

Test the TCP connection from the command line

printf 'HELLO|CLIENT=test|PROTO=DC1|ID=TRADER01\nEXIT\n' | nc 127.0.0.1 5590

Expected output: WELCOME|... followed by a clean connection close.

Common problems

Symptom	Likely cause	Fix
Connection refused	Gateway not started or wrong port	Confirm pm-dc-gwy is running; check dc_gateway.port in config
ERR\|CODE=AUTH_REQUIRED immediately	First line was not HELLO, or a required field was missing	Ensure the first line is HELLO\|CLIENT=..\|PROTO=DC1\|ID=..
WELCOME arrives but no DC_FILL ever does	No fills have occurred for that gateway ID yet, or pm-engine's drop-copy publisher failed to bind	Check pm-engine logs for drop-copy bind warnings; confirm trades are occurring for that gateway ID
Gateway closes after ~30s of silence	idle_timeout_sec elapsed	Send PING periodically, or increase idle_timeout_sec in config
Gateway not reachable from another host	bind_address: 127.0.0.1	Change bind_address to 0.0.0.0 (or the specific interface IP)

See also

• Drop Copy — the engine's drop-copy feed (:5557) this gateway relays
• Drop-Copy Spy (pm-dc-spy) — read-only CLI for the same feed
• ALF TCP Gateway → DC — relay your own fills through an existing ALF session instead
• Processes — process topology and ZMQ message tables
• External Protocols Overview — protocol comparison and selection guide