diff --git a/docs/used-resources.md b/docs/used-resources.md new file mode 100644 index 0000000000..7e49631301 --- /dev/null +++ b/docs/used-resources.md @@ -0,0 +1,275 @@ +# Used Resources Registry + +A central registry of the **statically occupied exclusive resources** of adapter instances — resources that +can only be claimed by a single instance at a time (serial ports, TCP/UDP ports, USB devices, Bluetooth/HCI +adapters, GPIO pins, …). + +**Static** is the important word: the registry answers "which resource is spoken for, so which one can I still +choose?". That only works for resources an instance occupies by configuration or by a fixed decision — a +listening port taken from the configuration, the serial port a device is wired to. Resources that are assigned +dynamically at runtime (an ephemeral source port of an outgoing connection, a port the OS hands out when you +bind to `0`) do not belong in here: nobody can collide with them, and they would only add noise. + +## Why + +Without this registry there is no reliable way to tell which serial port, network port, or hardware device is +already occupied by an existing instance. When a user configures a **new** instance they have to guess, which +leads to conflicts, silent failures, and hard-to-debug "device busy" errors. + +The registry gives the user (and the admin UI) a clear overview of occupied vs. free resources per host, so a +free one can be picked confidently when creating or reconfiguring an instance. + +## Who fills the registry + +There are two ways an instance's resources end up in the registry: + +- **Controller-managed (the default)** — `native.port` is the established ioBroker convention for the port an + instance listens on, and js-controller uses it: without any adapter change, the configured `native.port` + (plus `native.bind`, if set) is listed as a `tcpPort`. Because the source is the **configuration** and not a + running process, the port is listed for an instance that was never started or is currently stopped as well, + and a changed `native.port` is picked up immediately instead of at the next restart. +- **Adapter-declared** — the adapter sets `common.declareUsedResources: true` in its `io-package.json` and + calls `registerUsedResource(...)` / `freeUsedResource(...)` itself. Use this whenever the occupied resources + are something other than the configured `native.port`: a serial port, several ports, a UDP port, a USB + device. The controller then derives nothing for that instance — the adapter knows best what it really opens. + +Whether the controller supports the registry at all can be checked with +`adapter.supportsFeature('CONTROLLER_USED_RESOURCES')`. + +## How it works + +1. A running adapter declares the resources it occupies by calling `registerUsedResource(...)`; for an + instance without `common.declareUsedResources` the host derives the entries from the instance object instead. +2. The call is forwarded to the **host the instance runs on**. Only the host mutates the registry, which keeps + it consistent across all instances (no races between adapters). +3. The host keeps the registry in memory and mirrors it into the state's DB under + `system.host..usedResources.` (one state per resource type, a JSON array of entries). +4. Reading is done directly from those states — no round-trip to the host — via `getHostUsedResources(...)`. + +``` + Adapter Host (js-controller) States DB + ─────── ──────────────────── ───────── + registerUsedResource() ───push──▶ registry (in memory) ──persist──▶ system.host..usedResources. + freeUsedResource() ───push──▶ registry (in memory) ──persist──▶ ▲ + getHostUsedResources() ───────────────read states directly──────────────────────┘ +``` + +## Adapter API + +```ts +registerUsedResource( + type: T, + data: ioBroker.UsedResourceData, +): Promise; + +freeUsedResource( + type: T, + data?: Partial>, +): Promise; + +clearUsedResources(): Promise; + +getHostUsedResources( + type: T, +): Promise[]>; +getHostUsedResources(): Promise; +``` + +The `type` selects the resource kind; `data` is the **strictly typed** payload for that kind (see +[Resource types](#resource-types)). Passing a payload that does not match the `type` is a compile-time error. + +The three mutating calls always act on **this instance** — an instance can neither register nor free anything +in the name of another one. `getHostUsedResources` is the one that reads across the whole **host**, which is +why it carries `Host` in its name. + +### `registerUsedResource(type, data)` + +Registers a resource as occupied by this instance. Call it on adapter start-up once the resource is actually +open. + +Registering is **additive**: one call per occupied resource, in any order, from any number of async init +paths. There is nothing to reset by hand — the host drops what this instance registered before **whenever the +instance starts**, so a registration from a previous configuration cannot survive a restart: + +```ts +// on adapter start, in any order and from wherever the resource actually opens: +await this.registerUsedResource('serialPort', { port: '/dev/ttyUSB0', baudRate: 9600 }); +await this.registerUsedResource('tcpPort', { port: 1883 }); +await this.registerUsedResource('tcpPort', { port: 8081 }); +``` + +### `freeUsedResource(type, data?)` + +Frees previously registered resources of this instance. `data` is a **filter, not the exact payload**: every +field it names must match, fields it does not name are ignored. If `data` is omitted (or empty), **all** +resources of the given `type` for this instance are freed. + +```ts +await this.freeUsedResource('tcpPort', { port: 8081 }); // every tcpPort 8081 of this instance, +// whatever it was registered with besides the port +await this.freeUsedResource('tcpPort', { port: 8081, bind: '127.0.0.1' }); // only the one on that address +await this.freeUsedResource('serialPort'); // all serial ports of this instance +``` + +That the payload is a filter matters in practice: you do not have to repeat optional fields you may not even +know about — the controller adds `bind` to the resources it derives from `native.bind` itself, and a `free` +call that had to match it byte for byte would silently free nothing. + +A filter that matches nothing is **logged as a warning by the host** (`freed no used resource of type …`). +The adapter call itself does not wait for the host, so its promise resolves either way — the log is where a +wrong filter becomes visible. + +You normally do not need to call this on shutdown — the host handles stop/crash automatically (see +[Lifecycle](#lifecycle)). Use it when an instance releases a resource while it keeps running. + +### `clearUsedResources()` + +Frees **all** resources this instance registered, across every type. Needed neither on start-up (the host +already resets the registrations of a starting instance) nor on shutdown; use it when the instance drops +everything it occupied while it keeps running, e.g. on a reconfiguration. + +### `getHostUsedResources(type?)` + +Returns the resources currently registered on the host this instance runs on, across **all** instances of that +host, so an overview can be presented. With a `type` only that kind is read, without one the resources of every +type. Reads directly from the state's DB. + +```ts +const all = await this.getHostUsedResources(); // every occupied resource on this host +const serial = await this.getHostUsedResources('serialPort'); // only serial ports + +const wantedPort = 1883; +const inUse = (await this.getHostUsedResources('tcpPort')).some(r => r.data.port === wantedPort && r.isBlocked); +``` + +## Registered resource shape + +The read method returns entries of type `ioBroker.RegisteredResource`: the typed payload in `data` plus these +bookkeeping fields: + +| Field | Type | Meaning | +| ----------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| `type` | string | The resource type, e.g. `"serialPort"`. | +| `data` | object | The type-specific payload, exactly as passed to `registerUsedResource`, e.g. `{ port: '/dev/ttyUSB0' }`. | +| `instance` | string | The instance that occupies the resource, e.g. `"mqtt.0"`. | +| `ts` | number | Timestamp (ms) when the resource was registered. | +| `isBlocked` | boolean | `true`: the instance is running and actively holding the resource. `false`: the instance is not running and would maybe occupy it when started — "maybe", because its configuration can still change before that. | + +```ts +{ type: 'tcpPort', data: { port: 1883 }, instance: 'mqtt.0', ts: 1723632000000, isBlocked: true } +``` + +The payload is **nested** and not merged into the entry, so a payload key can never shadow a bookkeeping +field: whatever keys a resource type uses now or in the future, `type`, `instance`, `ts` and `isBlocked` +always describe the registration itself. `type` also stays a reliable discriminator — narrowing on it narrows +`data` to the matching payload type. + +`isBlocked` lets the UI distinguish an **actively used** resource from one that is merely **reserved** by a +currently stopped instance. + +## Lifecycle + +The host (js-controller) keeps the registry and `isBlocked` in sync with the instance lifecycle: + +| Event | Effect on the registry | +| -------------------------------------------------- | ------------------------------------------------------------------------------------------- | +| Instance created / `native` changed (controller-managed) | entries are re-derived from the configuration, `isBlocked` reflects whether it runs | +| Instance start (adapter-managed) | previous registrations of this instance are dropped; the adapter registers what it uses now | +| adapter `registerUsedResource(...)` | entry added with `isBlocked = true` (additive) | +| Instance start (controller-managed) | the derived entries are set to `isBlocked = true` | +| Instance stop / crash (process exit) | entries are **kept**, but set to `isBlocked = false` | +| Instance deleted or moved to another host | all entries of the instance are removed by the host | +| Instance deleted via CLI | the CLI removes the instance's entries from the registry states of the hosts that are **down** - a running host does that itself | +| Controller restart | registry is restored from states, an assessment runs (see below) and the controller-managed entries are re-derived | + +An adapter-managed resource is only recorded once the instance **runs** and its adapter registers it — the +adapter is the only one that knows what it really opened. A controller-managed resource comes from the +instance configuration and is therefore listed as soon as it is configured, which is what makes "pick a port +that is still free" work while setting up a new instance. + +### Controller start assessment + +On controller start the registry is restored from `system.host..usedResources.*` and then cleaned up: + +- **all `isBlocked` flags are reset to `false`** — at start no instance is running yet; each instance re-blocks + its resources when it starts; +- **entries of instances that no longer exist are removed** — e.g. an instance deleted via the CLI while the + controller was down (belt-and-suspenders together with the CLI cleanup); +- **malformed entries are dropped** — anything that does not have the shape of a `RegisteredResource`; +- **controller-managed entries are re-derived** from the instance objects of this host, so the registry + matches the current configuration even if it changed while the controller was down. + +## Storage layout + +For every resource type in use there is one state on the host: + +``` +system.host..usedResources. +``` + +The state value is a JSON-serialized array of `RegisteredResource` entries (object type `state`, +`common.type = 'array'`, `common.role = 'json'`, read-only). The object is created the first time the host +writes that type; afterwards only the state is written. + +When the last entry of a type disappears, the state stays and holds `[]` — a reader that subscribed to it +still sees the change. Writes of the same type are serialized by the host, so a state never falls back to +older content. + +## Resource types + +Each resource type has its own strictly typed payload. The types are defined in +`@iobroker/types-dev` (`packages/types-dev/index.d.ts`) via the `UsedResourceDataMap` interface: + +| Type | Payload (`UsedResourceData`) | +| ------------ | --------------------------------------------------------- | +| `serialPort` | `{ port: string; baudRate?: number }` | +| `tcpPort` | `{ port: number; bind?: string }` | +| `udpPort` | `{ port: number; bind?: string }` | +| `usb` | `{ path: string; vendorId?: string; productId?: string }` | +| `bluetooth` | `{ hci: string }` | +| `gpio` | `{ pin: number }` | + +### Adding a new resource type + +1. Add a payload interface and a `UsedResourceDataMap` entry in `packages/types-dev/index.d.ts`: + + ```ts + /** A CAN bus interface occupied by an instance */ + interface CanBusResourceData { + /** Interface name, e.g. "can0" */ + iface: string; + } + + interface UsedResourceDataMap { + // ...existing entries... + canBus: CanBusResourceData; + } + ``` + +2. No controller or adapter change is required — `UsedResourceType`, `UsedResourceData` and + `RegisteredResource` derive from the map, and the host stores/reads the type generically. + +The map is also open for module augmentation, so an adapter that owns a custom resource can extend it in its +own type declarations. + +## Message protocol (internal) + +`registerUsedResource` and `freeUsedResource` are sent to the host via the states message box (`pushMessage` to +`system.host.`), because only the host may mutate the registry: + +| Command | Message payload | Answer (only if a callback is passed) | +| ---------------------- | --------------------------- | ----------------------------------------- | +| `registerUsedResource` | `{ type, data, instance }` | `{ result: 'ok' }` | +| `freeUsedResource` | `{ type, data?, instance }` | `{ result: 'ok', freed: boolean }` | +| `clearUsedResources` | `{ instance }` | `{ result: 'ok' }` | + +Every command answers with `{ error }` instead if it was rejected. The adapter API does not pass a callback — +it sends and returns — so the host also logs what went wrong. + +The host **derives the instance from the sender** (`from: system.adapter.`) instead of trusting +`message.instance`, so no instance can register resources in the name of another one or free another one's +registrations; a message whose `instance` does not match the sender is rejected. The `type` is validated as +well, because it becomes the last segment of the state id. + +`getHostUsedResources` does **not** use a message — the adapter reads the +`usedResources.` states directly. diff --git a/packages/adapter/src/lib/adapter/adapter.ts b/packages/adapter/src/lib/adapter/adapter.ts index 9082dbcc28..f70f7c707f 100644 --- a/packages/adapter/src/lib/adapter/adapter.ts +++ b/packages/adapter/src/lib/adapter/adapter.ts @@ -9097,6 +9097,88 @@ export class AdapterClass extends EventEmitter { return this.#async.registerNotification(scope, category, message, options); } + /** + * Register an exclusive resource (serial port, TCP/UDP port, USB device, ...) as used by this instance. + * + * Exclusive resources are the ones that cannot be occupied by more than one instance at the same time. + * The information is forwarded to the host this instance runs on and stored under + * `system.host..usedResources.`, so the user gets an overview of the occupied resources + * and can pick a free one when configuring a new instance. + * + * Registering is **additive**: call it once per occupied resource, in any order and from any number of + * async init paths. The host drops what this instance registered before whenever the instance starts, so + * a stale registration from a previous configuration cannot survive a restart. + * + * @param type the kind of resource, e.g. "serialPort" or "tcpPort" + * @param data the strictly typed payload describing the resource, e.g. `{ port: '/dev/ttyUSB0' }` + */ + async registerUsedResource( + type: T, + data: ioBroker.UsedResourceData, + ): Promise { + return this.#async.registerUsedResource(type, data); + } + + /** + * Free all exclusive resources this instance registered, across all types. + * + * This is not needed on start-up (the host already resets the registrations of a starting instance) nor + * on shutdown (the host marks them as no longer held). Use it when the instance drops everything it + * occupied while it keeps running, e.g. on a reconfiguration. + */ + async clearUsedResources(): Promise { + return this.#async.clearUsedResources(); + } + + /** + * Free previously registered exclusive resources of this instance. + * + * `data` is a **filter, not the exact payload**: every field it names must match, fields it does not name + * are ignored. `freeUsedResource('tcpPort', { port: 8080 })` therefore also frees an entry registered as + * `{ port: 8080, bind: '0.0.0.0' }`, and if `data` is omitted, all registered resources of the given + * `type` for this instance are freed. The change is forwarded to the host this instance runs on and + * reflected in `system.host..usedResources.`; a filter that matches nothing is logged by + * the host. + * + * @param type the kind of resource, e.g. "serialPort" or "tcpPort" + * @param data the fields identifying the resources to free; if omitted, all resources of `type` are freed + */ + async freeUsedResource( + type: T, + data?: Partial>, + ): Promise { + return this.#async.freeUsedResource(type, data); + } + + /** + * Query the exclusive resources currently registered as used on the **host** this instance runs on. + * + * Unlike `registerUsedResource`/`freeUsedResource`/`clearUsedResources`, which only ever touch the + * resources of this instance, this returns the resources of **all** instances of this host, so the user + * (or an admin UI) can present an overview of what is occupied and pick something free. + * + * Reading is done directly from the state's DB (`system.host..usedResources.`), which the + * host keeps up to date; only the mutating calls go through the host to keep the registry consistent. + * + * @param type resource type to read, e.g. "serialPort" + * @returns the list of registered resources of that type (across all instances of this host) + */ + async getHostUsedResources(type: T): Promise[]>; + /** + * Query the exclusive resources of every type currently registered as used on the host this instance runs on. + * + * @returns the list of registered resources (across all instances and types of this host) + */ + async getHostUsedResources(): Promise; + + /** + * @param type resource type to read; if omitted, the resources of every type are returned + * @returns the list of registered resources (across all instances of this host) + */ + async getHostUsedResources(type?: ioBroker.UsedResourceType): Promise { + return type === undefined ? this.#async.getHostUsedResources() : this.#async.getHostUsedResources(type); + } + // external signatures /** * Writes value into states DB. diff --git a/packages/adapter/src/lib/adapter/asyncAdapter.ts b/packages/adapter/src/lib/adapter/asyncAdapter.ts index 0a1fc1a4fe..3949f51769 100644 --- a/packages/adapter/src/lib/adapter/asyncAdapter.ts +++ b/packages/adapter/src/lib/adapter/asyncAdapter.ts @@ -9,6 +9,7 @@ import type { import type { AdapterContext } from '@/lib/adapter/context.js'; import { CertificateManager } from '@/lib/adapter/managers/CertificateManager.js'; import { MessagingManager } from '@/lib/adapter/managers/MessagingManager.js'; +import { ResourceManager } from '@/lib/adapter/managers/ResourceManager.js'; import { Validator } from '@/lib/adapter/validator.js'; /** @@ -19,6 +20,7 @@ export class AsyncAdapter { readonly #ctx: AdapterContext; #messagingInstance?: MessagingManager; #certificatesInstance?: CertificateManager; + #resourcesInstance?: ResourceManager; /** * @param ctx Shared adapter context providing live runtime state @@ -37,6 +39,11 @@ export class AsyncAdapter { return (this.#certificatesInstance ??= new CertificateManager(this.#ctx)); } + /** Lazily-constructed exclusive-resource manager. */ + get #resources(): ResourceManager { + return (this.#resourcesInstance ??= new ResourceManager(this.#ctx)); + } + /** * Sends a message to another adapter instance and, unless `options.expectReply` is `false`, * resolves with the reply when it arrives (or rejects with `Error('Timeout exceeded')` on timeout). @@ -269,6 +276,69 @@ export class AsyncAdapter { this.#certificatesInstance?.stopWatching(); } + /** + * Registers an exclusive resource (serial port, TCP/UDP port, USB device, ...) as used by this + * instance. The registration is forwarded to the host, which stores it under + * `system.host..usedResources.`. + * + * @param type the kind of resource, e.g. "serialPort" or "tcpPort" + * @param data payload describing the resource + */ + async registerUsedResource( + type: T, + data: ioBroker.UsedResourceData, + ): Promise { + Validator.assertString(type, 'type'); + Validator.assertObject(data, 'data'); + return this.#resources.registerUsedResource(type, data); + } + + /** + * Frees all exclusive resources this instance registered, across all types. + */ + async clearUsedResources(): Promise { + return this.#resources.clearUsedResources(); + } + + /** + * Frees previously registered exclusive resources of this instance. `data` is a filter: every field it + * names must match. If it is omitted, all registered resources of the given `type` are freed. + * + * @param type the kind of resource, e.g. "serialPort" or "tcpPort" + * @param data fields identifying the resources to free; if omitted, all resources of `type` are freed + */ + async freeUsedResource( + type: T, + data?: Partial>, + ): Promise { + Validator.assertString(type, 'type'); + if (data !== undefined) { + Validator.assertObject(data, 'data'); + } + return this.#resources.freeUsedResource(type, data); + } + + /** + * Reads the exclusive resources of the given type registered on this instance's host, across all + * instances of that host. + * + * @param type resource type to read, e.g. "serialPort" + */ + getHostUsedResources(type: T): Promise[]>; + /** Reads the exclusive resources of every type registered on this instance's host. */ + getHostUsedResources(): Promise; + + /** + * @param type resource type to read; if omitted, the resources of every type are read + */ + getHostUsedResources(type?: ioBroker.UsedResourceType): Promise { + if (type !== undefined) { + Validator.assertString(type, 'type'); + return this.#resources.getHostUsedResources(type); + } + return this.#resources.getHostUsedResources(); + } + /** * Resolves a pending reply promise for an acked messagebox message. * Returns true if a pending entry was found and consumed. diff --git a/packages/adapter/src/lib/adapter/managers/ResourceManager.test.ts b/packages/adapter/src/lib/adapter/managers/ResourceManager.test.ts new file mode 100644 index 0000000000..06403d6817 --- /dev/null +++ b/packages/adapter/src/lib/adapter/managers/ResourceManager.test.ts @@ -0,0 +1,178 @@ +import assert from 'node:assert/strict'; +import sinon from 'sinon'; +import { tools } from '@iobroker/js-controller-common'; +import { ResourceManager } from './ResourceManager.js'; +import type { AdapterContext } from '../context.js'; + +function makeContext(over: Partial = {}): AdapterContext { + return { + namespace: 'test.0', + namespaceLog: 'test.0', + logger: { silly() {}, debug() {}, info() {}, warn() {}, error() {} } as any, + uiMessagingController: {} as any, + states: null, + objects: null, + common: undefined, + config: {} as ioBroker.AdapterConfig, + host: 'localhost', + ...over, + }; +} + +describe('ResourceManager.registerUsedResource', () => { + it('rejects with ERROR_DB_CLOSED when states is not connected', async () => { + const mgr = new ResourceManager(makeContext({ states: null })); + await assert.rejects( + () => mgr.registerUsedResource('serialPort', { port: '/dev/ttyUSB0' }), + new RegExp(tools.ERRORS.ERROR_DB_CLOSED), + ); + }); + + it('forwards the resource to the host with the instance', async () => { + const pushMessage = sinon.stub().resolves(); + const mgr = new ResourceManager(makeContext({ states: { pushMessage } as any })); + + await mgr.registerUsedResource('serialPort', { port: '/dev/ttyUSB0' }); + + assert.equal(pushMessage.callCount, 1); + const [target, obj] = pushMessage.firstCall.args; + assert.equal(target, 'system.host.localhost'); + assert.equal(obj.command, 'registerUsedResource'); + assert.equal(obj.from, 'system.adapter.test.0'); + assert.deepEqual(obj.message, { + type: 'serialPort', + data: { port: '/dev/ttyUSB0' }, + instance: 'test.0', + }); + }); + + it('is additive: every call forwards its own resource', async () => { + const pushMessage = sinon.stub().resolves(); + const mgr = new ResourceManager(makeContext({ states: { pushMessage } as any })); + + await mgr.registerUsedResource('serialPort', { port: '/dev/ttyUSB0' }); + await mgr.registerUsedResource('tcpPort', { port: 1883 }); + + assert.equal(pushMessage.callCount, 2); + assert.deepEqual( + pushMessage.getCalls().map(call => call.args[1].message.type), + ['serialPort', 'tcpPort'], + ); + }); +}); + +describe('ResourceManager.clearUsedResources', () => { + it('rejects with ERROR_DB_CLOSED when states is not connected', async () => { + const mgr = new ResourceManager(makeContext({ states: null })); + await assert.rejects(() => mgr.clearUsedResources(), new RegExp(tools.ERRORS.ERROR_DB_CLOSED)); + }); + + it('forwards the request to the host', async () => { + const pushMessage = sinon.stub().resolves(); + const mgr = new ResourceManager(makeContext({ states: { pushMessage } as any })); + + await mgr.clearUsedResources(); + + const [target, obj] = pushMessage.firstCall.args; + assert.equal(target, 'system.host.localhost'); + assert.equal(obj.command, 'clearUsedResources'); + assert.equal(obj.from, 'system.adapter.test.0'); + assert.deepEqual(obj.message, { instance: 'test.0' }); + }); +}); + +describe('ResourceManager.freeUsedResource', () => { + it('rejects with ERROR_DB_CLOSED when states is not connected', async () => { + const mgr = new ResourceManager(makeContext({ states: null })); + await assert.rejects(() => mgr.freeUsedResource('serialPort'), new RegExp(tools.ERRORS.ERROR_DB_CLOSED)); + }); + + it('forwards a specific resource to free to the host', async () => { + const pushMessage = sinon.stub().resolves(); + const mgr = new ResourceManager(makeContext({ states: { pushMessage } as any })); + + await mgr.freeUsedResource('serialPort', { port: '/dev/ttyUSB0' }); + + const [target, obj] = pushMessage.firstCall.args; + assert.equal(target, 'system.host.localhost'); + assert.equal(obj.command, 'freeUsedResource'); + assert.deepEqual(obj.message, { + type: 'serialPort', + data: { port: '/dev/ttyUSB0' }, + instance: 'test.0', + }); + }); + + it('forwards a free-all request (no data) to the host', async () => { + const pushMessage = sinon.stub().resolves(); + const mgr = new ResourceManager(makeContext({ states: { pushMessage } as any })); + + await mgr.freeUsedResource('serialPort'); + + assert.equal(pushMessage.firstCall.args[1].message.data, undefined); + }); +}); + +describe('ResourceManager.getHostUsedResources', () => { + it('throws when the host is unknown', async () => { + const mgr = new ResourceManager(makeContext({ host: undefined, states: {} as any })); + await assert.rejects(() => mgr.getHostUsedResources('serialPort'), /host of this instance is unknown/); + await assert.rejects(() => mgr.getHostUsedResources(), /host of this instance is unknown/); + }); + + it('reads and parses the resources of the given type from the host state', async () => { + const entries = [ + { type: 'serialPort', data: { port: '/dev/ttyUSB0' }, instance: 'test.0', ts: 1, isBlocked: true }, + ]; + const getState = sinon.stub().resolves({ val: JSON.stringify(entries) }); + const mgr = new ResourceManager(makeContext({ states: { getState } as any })); + + const res = await mgr.getHostUsedResources('serialPort'); + + assert.equal(getState.firstCall.args[0], 'system.host.localhost.usedResources.serialPort'); + assert.deepEqual(res, entries); + }); + + it('returns an empty list for a missing, empty or malformed state', async () => { + const getState = sinon.stub(); + getState.onCall(0).resolves(null); + getState.onCall(1).resolves({ val: '' }); + getState.onCall(2).resolves({ val: 'not-json' }); + getState.onCall(3).resolves({ val: '{"not":"an array"}' }); + const mgr = new ResourceManager(makeContext({ states: { getState } as any })); + + assert.deepEqual(await mgr.getHostUsedResources('serialPort'), []); + assert.deepEqual(await mgr.getHostUsedResources('serialPort'), []); + assert.deepEqual(await mgr.getHostUsedResources('serialPort'), []); + assert.deepEqual(await mgr.getHostUsedResources('serialPort'), []); + }); + + it('collects and flattens resources across all types when no type is given', async () => { + const serial = [ + { type: 'serialPort', data: { port: '/dev/ttyUSB0' }, instance: 'test.0', ts: 1, isBlocked: true }, + ]; + const tcp = [{ type: 'tcpPort', data: { port: 8080 }, instance: 'web.0', ts: 2, isBlocked: false }]; + const getKeys = sinon + .stub() + .resolves([ + 'system.host.localhost.usedResources.serialPort', + 'system.host.localhost.usedResources.tcpPort', + ]); + const getStates = sinon.stub().resolves([{ val: JSON.stringify(serial) }, { val: JSON.stringify(tcp) }]); + const mgr = new ResourceManager(makeContext({ states: { getKeys, getStates } as any })); + + const res = await mgr.getHostUsedResources(); + + assert.equal(getKeys.firstCall.args[0], 'system.host.localhost.usedResources.*'); + assert.deepEqual(res, [...serial, ...tcp]); + }); + + it('returns an empty list when the host has no resource states', async () => { + const getKeys = sinon.stub().resolves([]); + const getStates = sinon.stub().resolves([]); + const mgr = new ResourceManager(makeContext({ states: { getKeys, getStates } as any })); + + assert.deepEqual(await mgr.getHostUsedResources(), []); + assert.equal(getStates.callCount, 0); + }); +}); diff --git a/packages/adapter/src/lib/adapter/managers/ResourceManager.ts b/packages/adapter/src/lib/adapter/managers/ResourceManager.ts new file mode 100644 index 0000000000..142725fa1d --- /dev/null +++ b/packages/adapter/src/lib/adapter/managers/ResourceManager.ts @@ -0,0 +1,141 @@ +import type { AdapterContext } from '@/lib/adapter/context.js'; +import { AdapterContextBase } from '@/lib/adapter/managers/AdapterContextBase.js'; + +/** Sub-id under a host holding the exclusive resources registered as used by its instances. */ +const USED_RESOURCES_ID = 'usedResources'; + +/** + * Owns the adapter's exclusive-resource registry. Register/free requests are forwarded to the host + * this instance runs on, which keeps `system.host..usedResources.` up to date; reads + * go straight to those states. + */ +export class ResourceManager extends AdapterContextBase { + /** + * @param ctx Shared adapter context providing live runtime state + */ + constructor(ctx: AdapterContext) { + super(ctx); + } + + /** + * Registers an exclusive resource as used by this instance by forwarding it to the host. + * + * Registering is additive - one call per occupied resource, in any order. The host drops what this + * instance registered before when it starts, so there is nothing to reset by hand. + * + * @param type the kind of resource, e.g. "serialPort" or "tcpPort" + * @param data payload describing the resource + */ + async registerUsedResource( + type: T, + data: ioBroker.UsedResourceData, + ): Promise { + const obj = { + command: 'registerUsedResource', + message: { + type, + data, + instance: this.namespace, + }, + from: `system.adapter.${this.namespace}`, + }; + + await this.states.pushMessage(`system.host.${this.host}`, obj); + } + + /** + * Frees all exclusive resources this instance registered, across all types, by forwarding it to the host. + */ + async clearUsedResources(): Promise { + const obj = { + command: 'clearUsedResources', + message: { instance: this.namespace }, + from: `system.adapter.${this.namespace}`, + }; + + await this.states.pushMessage(`system.host.${this.host}`, obj); + } + + /** + * Frees previously registered exclusive resources of this instance by forwarding it to the host. + * + * @param type the kind of resource, e.g. "serialPort" or "tcpPort" + * @param data fields identifying the resources to free; if omitted, all resources of `type` are freed + */ + async freeUsedResource( + type: T, + data?: Partial>, + ): Promise { + const obj = { + command: 'freeUsedResource', + message: { + type, + data, + instance: this.namespace, + }, + from: `system.adapter.${this.namespace}`, + }; + + await this.states.pushMessage(`system.host.${this.host}`, obj); + } + + /** + * Reads the exclusive resources of the given type currently registered on this instance's host, across + * all its instances. + * + * @param type resource type to read, e.g. "serialPort" + * @throws {Error} when the host of this instance is unknown + */ + async getHostUsedResources(type: T): Promise[]>; + /** Reads the exclusive resources of every type currently registered on this instance's host. */ + async getHostUsedResources(): Promise; + + /** + * @param type resource type to read; if omitted, the resources of every type are read + */ + async getHostUsedResources(type?: ioBroker.UsedResourceType): Promise { + if (!this.host) { + throw new Error('getHostUsedResources: host of this instance is unknown'); + } + + const states = this.states; + const prefix = `system.host.${this.host}.${USED_RESOURCES_ID}`; + + if (type !== undefined) { + const state = await states.getState(`${prefix}.${type}`); + return ResourceManager.#parseResources(state); + } + + const keys = await states.getKeys(`${prefix}.*`); + + const resources: ioBroker.RegisteredResource[] = []; + if (keys?.length) { + const values = await states.getStates(keys); + for (const state of values) { + resources.push(...ResourceManager.#parseResources(state)); + } + } + + return resources; + } + + /** + * Parses a `usedResources` state value (a JSON-encoded array) into a typed list, ignoring + * malformed or empty content. + * + * @param state the state holding the JSON-encoded resource array, if any + */ + static #parseResources(state: ioBroker.State | null | undefined): R[] { + if (state && typeof state.val === 'string' && state.val) { + try { + const parsed: unknown = JSON.parse(state.val); + if (Array.isArray(parsed)) { + return parsed as R[]; + } + } catch { + // ignore malformed content + } + } + return []; + } +} diff --git a/packages/cli/src/lib/setup/setupInstall.ts b/packages/cli/src/lib/setup/setupInstall.ts index 130079e3db..194513619e 100644 --- a/packages/cli/src/lib/setup/setupInstall.ts +++ b/packages/cli/src/lib/setup/setupInstall.ts @@ -1648,12 +1648,97 @@ export class Install { await this._deleteAdapterObjects(knownObjectIDs); await this._deleteAdapterStates(knownStateIDs); + await this._freeUsedResources(adapter, instance); if (this.params.custom) { // delete instance from custom await this._removeCustomFromObjects([`${adapter}.${instance}`]); } } + /** + * Free all exclusive resources (serial ports, TCP/UDP ports, ...) the given instance(s) had registered. + * + * This is needed because an instance can be deleted via the CLI while a js-controller is not running: it + * cannot notice that the instance object disappeared, so its `system.host..usedResources.` + * states would keep listing the resources of an instance that no longer exists. + * + * Hosts that are running are deliberately skipped. Such a host cleans its own registry up when it sees the + * instance object being deleted, and it holds the registry in memory: it rewrites the state from that copy + * on its next change, which would silently undo a write made here. + * + * @param adapter adapter name, e.g. "mqtt" + * @param instance instance number; if undefined, all instances of the adapter are freed + */ + private async _freeUsedResources(adapter: string, instance?: number): Promise { + const matches = (namespace: string): boolean => + instance !== undefined ? namespace === `${adapter}.${instance}` : namespace.startsWith(`${adapter}.`); + + let keys: string[] | null | undefined; + try { + keys = await this.states.getKeys('system.host.*.usedResources.*'); + } catch (e) { + console.warn(`Cannot read the used resources registry: ${e.message}`); + return; + } + if (!keys?.length) { + return; + } + + const hostPrefix = 'system.host.'; + const registryMarker = '.usedResources.'; + /** whether a host is currently running, queried once per host */ + const hostIsRunning = new Map(); + + for (const id of keys) { + const markerIndex = id.lastIndexOf(registryMarker); + if (!id.startsWith(hostPrefix) || markerIndex <= hostPrefix.length) { + continue; + } + const host = id.substring(hostPrefix.length, markerIndex); + + let isRunning = hostIsRunning.get(host); + if (isRunning === undefined) { + try { + const aliveState = await this.states.getState(`${hostPrefix}${host}.alive`); + isRunning = aliveState?.val === true; + } catch (e) { + // if it cannot be determined, assume the host is running: leaving an entry behind is + // harmless (its controller drops it on the next start) while fighting a live controller + // over the same state is not + console.warn(`Cannot check whether host "${host}" is running: ${e.message}`); + isRunning = true; + } + hostIsRunning.set(host, isRunning); + } + + if (isRunning) { + continue; + } + + try { + const state = await this.states.getState(id); + if (!state || typeof state.val !== 'string' || !state.val) { + continue; + } + const parsed: unknown = JSON.parse(state.val); + if (!Array.isArray(parsed)) { + continue; + } + const original = parsed as { instance?: unknown }[]; + // keep anything that is malformed: it cannot be attributed to the deleted instance, and one + // broken entry must not stop the cleanup of the sound ones in the same state + const filtered = original.filter( + entry => typeof entry?.instance !== 'string' || !matches(entry.instance), + ); + if (filtered.length !== original.length) { + await this.states.setStateAsync(id, { val: JSON.stringify(filtered), ack: true }); + } + } catch (e) { + console.warn(`Cannot free the used resources in "${id}": ${e.message}`); + } + } + } + /** * Remove all node modules that has been installed by this instance * diff --git a/packages/common/src/lib/common/constants.ts b/packages/common/src/lib/common/constants.ts index ffda25d868..e90d886546 100644 --- a/packages/common/src/lib/common/constants.ts +++ b/packages/common/src/lib/common/constants.ts @@ -23,6 +23,7 @@ const SUPPORTED_FEATURES_INTERNAL = [ 'ADAPTER_WEBSERVER_UPGRADE', // Controller supports upgrading adapter and provides a webserver (triggered via sendToHost). Since `js-controller` 5.0 'CONTROLLER_CMD_EXEC_FILES', // cmdExec host message supports sending files together with the command. Since `js-controller` 7.2 'CONTROLLER_FEATURE_REQUEST', // js-controller supports feature support requests via host messages. Since `js-controller` 7.2 + 'CONTROLLER_USED_RESOURCES', // js-controller maintains a registry of exclusive resources (serial/TCP/UDP ports, ...) used by instances. Since `js-controller` 7.2 ] as const; export const SUPPORTED_FEATURES = [...SUPPORTED_FEATURES_INTERNAL]; diff --git a/packages/controller/src/lib/usedResources.ts b/packages/controller/src/lib/usedResources.ts new file mode 100644 index 0000000000..02719cee42 --- /dev/null +++ b/packages/controller/src/lib/usedResources.ts @@ -0,0 +1,339 @@ +/** + * In-memory registry of exclusive resources (serial ports, TCP/UDP ports, USB devices, ...) occupied by the + * adapter instances running on a host. + * + * This module contains only the pure, side-effect-free bookkeeping logic so that it can be unit tested without + * a running controller. Persistence into `system.host..usedResources.` and the message handling + * live in `main.ts`, which owns a single {@link UsedResourcesRegistry} instance and persists the resource types + * that the mutating methods report as changed. + */ + +/** A resource type ends up as the last segment of `system.host..usedResources.`, so it has to be a plain identifier */ +const RESOURCE_TYPE_PATTERN = /^[A-Za-z0-9_-]{1,64}$/; + +/** + * Check that a value can be used as a resource type. + * + * `UsedResourceDataMap` is intentionally open for module augmentation, so unknown type names are accepted as + * long as they are usable as a state id segment. What this rejects is a missing, non-string or otherwise + * malformed type, which would create a `system.host..usedResources.undefined` state that + * {@link UsedResourcesRegistry} would happily read back as a real resource type on the next controller start. + * + * @param type the value to check + */ +export function isValidUsedResourceType(type: unknown): type is ioBroker.UsedResourceType { + return typeof type === 'string' && RESOURCE_TYPE_PATTERN.test(type); +} + +/** + * Check that a value has the shape of a registered resource. Used when reading entries back from the + * persisted state, so that malformed or outdated content cannot enter the registry. + * + * @param entry the value to check + */ +export function isRegisteredResource(entry: unknown): entry is ioBroker.RegisteredResource { + if (typeof entry !== 'object' || entry === null) { + return false; + } + const candidate = entry as Partial; + return ( + isValidUsedResourceType(candidate.type) && + typeof candidate.instance === 'string' && + !!candidate.instance && + typeof candidate.ts === 'number' && + typeof candidate.isBlocked === 'boolean' && + typeof candidate.data === 'object' && + candidate.data !== null && + !Array.isArray(candidate.data) + ); +} + +/** + * Build a stable comparison key for a registered resource so that duplicates can be detected and the correct + * entry can be freed. The key is composed of the instance, the type and the sorted payload; the bookkeeping + * fields (`ts`, `isBlocked`) are intentionally ignored so that re-registering or (un)blocking hits the same entry. + * + * Payload values are serialized with `JSON.stringify`, so `80` and `"80"` stay distinguishable, and keys with + * an `undefined` value are dropped, so passing an optional field explicitly as `undefined` produces the same + * key as omitting it (which is also what survives the JSON round-trip through the persisted state). + * + * @param resource the resource to build the key for + * @param resource.type the resource type, e.g. "serialPort" + * @param resource.instance the instance that occupies the resource, e.g. "mqtt.0" + * @param resource.data the type-specific payload describing the resource + */ +export function getUsedResourceKey(resource: { + type: ioBroker.UsedResourceType; + instance: string; + data: ioBroker.UsedResourceData | undefined; +}): string { + const { type, instance } = resource; + const data = (resource.data || {}) as Record; + const sorted = Object.keys(data) + .filter(key => data[key] !== undefined) + .sort() + .map(key => `${key}=${JSON.stringify(data[key])}`) + .join(','); + return `${instance}|${type}|${sorted}`; +} + +/** + * Check whether a registered payload matches a filter: every field the filter names must be equal, fields it + * does not name are ignored. An omitted or empty filter matches everything. + * + * Values are compared serialized, so `80` and `"80"` stay different and structured values are compared by + * content - the same rules {@link getUsedResourceKey} applies. A field explicitly set to `undefined` counts + * as not named, because that is also what survives the JSON round-trip through the persisted state. + * + * @param data the payload of a registered resource + * @param filter the fields that have to match + */ +export function matchesUsedResourceData( + data: ioBroker.UsedResourceData, + filter: Partial | undefined, +): boolean { + if (!filter) { + return true; + } + + const entries = data as unknown as Record; + for (const [key, value] of Object.entries(filter as Record)) { + if (value === undefined) { + continue; + } + if (JSON.stringify(entries[key]) !== JSON.stringify(value)) { + return false; + } + } + + return true; +} + +/** Options for the {@link UsedResourcesRegistry} */ +export interface UsedResourcesRegistryOptions { + /** Clock used for the `ts` of newly registered resources. Injectable for deterministic tests. Defaults to `Date.now`. */ + now?: () => number; +} + +/** + * Pure in-memory registry of the used resources of a single host. + * + * All mutating methods return the list of resource types they changed, so the caller can persist exactly those + * (and only those) types. Nothing here touches the databases. + */ +export class UsedResourcesRegistry { + private readonly resources = new Map(); + private readonly now: () => number; + + /** + * @param options optional configuration, e.g. an injectable clock for deterministic tests + */ + constructor(options: UsedResourcesRegistryOptions = {}) { + this.now = options.now ?? Date.now; + } + + /** + * Replace the whole list of a resource type. Used when loading the persisted state back into memory. + * + * @param type the resource type + * @param list the resources of that type + */ + setType(type: ioBroker.UsedResourceType, list: ioBroker.RegisteredResource[]): void { + this.setEntries(type, list); + } + + /** + * Store the entries of a resource type, dropping the type entirely when nothing is left. An empty bucket + * would otherwise be reported by {@link UsedResourcesRegistry.getTypes} and be persisted and reloaded + * forever, for every type any instance of this host ever used. + * + * @param type the resource type + * @param list the remaining resources of that type + */ + private setEntries(type: ioBroker.UsedResourceType, list: ioBroker.RegisteredResource[]): void { + if (list.length) { + this.resources.set(type, list); + } else { + this.resources.delete(type); + } + } + + /** All resource types that currently hold at least one entry. */ + getTypes(): ioBroker.UsedResourceType[] { + return [...this.resources.keys()]; + } + + /** + * Get the registered resources, optionally filtered by type. + * + * The entries are deep copies, so a caller cannot reach into the registry through the nested `data` of a + * returned entry. + * + * @param type optional resource type to filter for; if omitted, resources of all types are returned + */ + get(type?: ioBroker.UsedResourceType): ioBroker.RegisteredResource[] { + const clone = (r: ioBroker.RegisteredResource): ioBroker.RegisteredResource => structuredClone(r); + if (type) { + return (this.resources.get(type) || []).map(clone); + } + const all: ioBroker.RegisteredResource[] = []; + for (const list of this.resources.values()) { + all.push(...list.map(clone)); + } + return all; + } + + /** + * Register a resource as used by an instance. + * + * Registering is always **additive**: an instance registers one entry per resource it occupies, in any + * order and from any number of async init paths. Dropping what an instance registered earlier is a + * separate, explicit operation ({@link UsedResourcesRegistry.removeInstance}), which the controller + * performs once when the instance starts. + * + * @param type the resource type, e.g. "serialPort" + * @param data the type-specific payload describing the resource + * @param instance the instance that occupies the resource, e.g. "mqtt.0" + * @returns the resource types that changed and should be persisted + */ + register( + type: T, + data: ioBroker.UsedResourceData, + instance: string, + ): ioBroker.UsedResourceType[] { + // an instance can only register a resource while it is running, so it is actively blocking it. + // `satisfies` checks the shape against the entry for exactly this `type`; the cast afterwards only + // widens it to the stored union, which TypeScript cannot derive from the generic on its own. + const resource = { + type, + data, + instance, + ts: this.now(), + isBlocked: true, + } satisfies ioBroker.RegisteredResource as ioBroker.RegisteredResource; + + const list = this.resources.get(type) || []; + const key = getUsedResourceKey(resource); + const existingIndex = list.findIndex(entry => getUsedResourceKey(entry) === key); + + if (existingIndex === -1) { + list.push(resource); + } else { + // refresh the timestamp and blocking flag of an already known resource + list[existingIndex] = resource; + } + + this.setEntries(type, list); + return [type]; + } + + /** + * Free the resources of an instance that match a description. + * + * `data` is a **filter, not an exact payload**: every field it names must match, fields it does not name + * are ignored. So `free('tcpPort', { port: 8080 }, 'web.0')` also frees an entry that was registered as + * `{ port: 8080, bind: '0.0.0.0' }` - the caller does not have to repeat optional fields it may not even + * know about (the controller adds `bind` to the resources it derives itself). An omitted or empty `data` + * matches everything, which frees all resources of that type for the instance. + * + * Only the entries of `instance` are ever considered, so a filter can never reach a foreign registration. + * + * @param type the resource type, e.g. "serialPort" + * @param data the fields identifying the resources to free; if omitted, all resources of `type` for the instance are freed + * @param instance the instance that occupied the resource, e.g. "mqtt.0" + * @returns the resource types that changed and should be persisted - empty if nothing matched + */ + free( + type: T, + data: Partial> | undefined, + instance: string, + ): ioBroker.UsedResourceType[] { + const list = this.resources.get(type); + if (!list) { + return []; + } + + const filtered = list.filter( + entry => entry.instance !== instance || !matchesUsedResourceData(entry.data, data), + ); + + if (filtered.length === list.length) { + return []; + } + + this.setEntries(type, filtered); + return [type]; + } + + /** + * Update the `isBlocked` flag of all resources of an instance across all types. Called on instance start + * (blocked) and stop (not blocked): a stopped instance keeps its registrations, but they are no longer held. + * + * @param instance the instance whose resources should be updated, e.g. "mqtt.0" + * @param isBlocked whether the instance is currently running and actively using the resources + * @returns the resource types that changed and should be persisted + */ + setInstanceBlocked(instance: string, isBlocked: boolean): ioBroker.UsedResourceType[] { + const changed: ioBroker.UsedResourceType[] = []; + for (const [type, list] of this.resources) { + let typeChanged = false; + for (const entry of list) { + if (entry.instance === instance && entry.isBlocked !== isBlocked) { + entry.isBlocked = isBlocked; + typeChanged = true; + } + } + if (typeChanged) { + changed.push(type); + } + } + return changed; + } + + /** + * Remove all resources registered by the given instance across all types. + * + * Used when an instance is deleted or moved to another host, and once when an instance starts: the user + * may have changed the settings in between, so what the instance registered before is dropped and the + * instance (or the controller) declares from scratch what it really occupies now. + * + * @param instance the instance whose resources should be removed, e.g. "mqtt.0" + * @returns the resource types that changed and should be persisted + */ + removeInstance(instance: string): ioBroker.UsedResourceType[] { + const changed: ioBroker.UsedResourceType[] = []; + for (const [type, list] of this.resources) { + if (list.some(entry => entry.instance === instance)) { + this.setEntries( + type, + list.filter(entry => entry.instance !== instance), + ); + changed.push(type); + } + } + return changed; + } + + /** + * Assessment run on controller start: reset every `isBlocked` flag to `false` (no instance is running yet) + * and drop resources whose instance no longer exists (e.g. deleted via CLI while the controller was down). + * + * @param existingInstances the namespaces of the instances that currently exist, e.g. `new Set(['mqtt.0'])` + * @returns the resource types that changed and should be persisted + */ + assess(existingInstances: Set): ioBroker.UsedResourceType[] { + const changed: ioBroker.UsedResourceType[] = []; + for (const [type, list] of this.resources) { + const cleaned = list + .filter(entry => existingInstances.has(entry.instance)) + .map(entry => (entry.isBlocked ? { ...entry, isBlocked: false } : entry)); + + const wasChanged = cleaned.length !== list.length || list.some(entry => entry.isBlocked); + if (wasChanged) { + this.setEntries(type, cleaned); + changed.push(type); + } + } + return changed; + } +} diff --git a/packages/controller/src/main.ts b/packages/controller/src/main.ts index 34781b85d5..9e898ea7ce 100644 --- a/packages/controller/src/main.ts +++ b/packages/controller/src/main.ts @@ -54,6 +54,7 @@ import type { UpgradeArguments } from '@/lib/upgradeManager.js'; import { AdapterUpgradeManager } from '@/lib/adapterUpgradeManager.js'; import { setTimeout as wait } from 'node:timers/promises'; import { getHostObjects } from '@/lib/objects.js'; +import { isRegisteredResource, isValidUsedResourceType, UsedResourcesRegistry } from '@/lib/usedResources.js'; import * as url from 'node:url'; import { createRequire } from 'node:module'; // eslint-disable-next-line unicorn/prefer-module @@ -219,6 +220,11 @@ let compactGroupController = false; let compactGroup: null | number = null; const compactProcs: Record = {}; const scheduledInstances: Record = {}; +/** + * In-memory registry of the exclusive resources (serial ports, TCP/UDP ports, USB devices, ...) currently used + * by the instances running on this host. Mirrored into `system.host..usedResources.`. + */ +const usedResources = new UsedResourcesRegistry(); /** If less than this disk space free in %, generate a warning */ let diskWarningLevel = DEFAULT_DISK_WARNING_LEVEL; @@ -727,6 +733,7 @@ async function initializeController(): Promise { await checkHost(); await startMultihost(config); await setMeta(); + await loadUsedResources(); started = true; await getInstances(); } @@ -806,6 +813,11 @@ function createObjects(onConnect: () => void): void { try { logger.debug(`${hostLogPrefix} object change ${id} (from: ${obj ? obj.from : null})`); + + // the configuration of an instance defines which resources it occupies, so keep the registry + // in line with it - no matter whether the instance is known here, running, or was just deleted + await syncUsedResourcesOfInstance(id, obj); + // known adapter const proc = procs[id]; @@ -825,6 +837,7 @@ function createObjects(onConnect: () => void): void { } // instance removed -> remove all notifications + // (its used resources were already freed by syncUsedResourcesOfInstance above) await notificationHandler.clearNotifications(null, null, id); proc.config.common.enabled = false; // @ts-expect-error check if we can handle it differently @@ -2065,6 +2078,383 @@ async function uploadAdapter(task: UploadTask): Promise { } } +/** Resource types whose state object has already been created in this controller run */ +const createdUsedResourceObjects = new Set(); +/** The write currently in flight per resource type, so that two writes of the same type cannot interleave */ +const pendingUsedResourceWrites = new Map>(); + +/** + * Write the current content of a resource type into `system.host..usedResources.`. + * + * The object is only created the first time this type is written in this controller run - `extendObject` + * costs an objects-DB write plus a change event broadcast to every connected client, which is not worth + * paying on every resource change. + * + * Never rejects: a failed write is logged, because the callers are partly fire-and-forget. + * + * @param type the resource type to write + */ +async function writeUsedResources(type: ioBroker.UsedResourceType): Promise { + const id = `${hostObjectPrefix}.usedResources.${type}`; + + try { + if (!createdUsedResourceObjects.has(type)) { + await objects!.extendObject(id, { + type: 'state', + common: { + name: `Used resources: ${type}`, + type: 'array', + role: 'json', + read: true, + write: false, + }, + native: {}, + }); + createdUsedResourceObjects.add(type); + } + + // read the registry only now: everything before this point may have yielded to another write + const resources = usedResources.get(type); + await states!.setState(id, { val: JSON.stringify(resources), ack: true, from: hostObjectPrefix }); + } catch (e) { + logger.warn(`${hostLogPrefix} Cannot persist used resources of type "${type}": ${e.message}`); + } +} + +/** + * Persist the used resources of a given type into `system.host..usedResources.`. + * + * Writes of the same type are chained: several callers are deliberately fire-and-forget (instance start and + * exit handlers), so without the chain two of them could read the registry, interleave over their awaits and + * let the older content win - which would then also be what the next controller start reads back. + * + * @param type the resource type to persist + */ +function persistUsedResources(type: ioBroker.UsedResourceType): Promise { + if (compactGroupController) { + // the registry of a host is owned by its main controller: a compact group controller writes to its own + // `system.host.compactGroup` prefix, which nobody reads, and would compete with the real one. + // Guarding here covers every caller, including the lifecycle handlers shared with the main controller. + return Promise.resolve(); + } + + const pending = pendingUsedResourceWrites.get(type) || Promise.resolve(); + // `.catch` before chaining on purpose: a rejected link would otherwise never run the write of the + // next one, and the cleanup below would never fire either - so that resource type would silently + // stop being persisted for the rest of this controller's life. writeUsedResources() handles its + // own errors today, but that invariant lives in another function. + const next = pending.catch(() => {}).then(() => writeUsedResources(type)); + pendingUsedResourceWrites.set(type, next); + + // forget the chain again once nothing else is queued behind it + void next + .catch(() => {}) + .then(() => { + if (pendingUsedResourceWrites.get(type) === next) { + pendingUsedResourceWrites.delete(type); + } + }); + + return next; +} + +/** + * Persist all given resource types (as reported changed by a registry mutation). + * + * @param types the resource types to persist + */ +async function persistUsedResourceTypes(types: ioBroker.UsedResourceType[]): Promise { + for (const type of types) { + await persistUsedResources(type); + } +} + +/** + * Load the used resources persisted under `system.host..usedResources.*` back into the registry. + * Called once on controller start so that registrations survive a controller restart. + * + * On load an assessment is done (see {@link UsedResourcesRegistry.assess}): + * - all `isBlocked` flags are reset to `false`, because at controller start no instance is running yet + * (running instances re-register and thereby re-block their resources on their next start); + * - resources of instances that no longer exist (e.g. deleted via CLI while the controller was down) are removed. + */ +async function loadUsedResources(): Promise { + if (!states || !objects || compactGroupController) { + // the registry of a host is owned by its main controller + return; + } + try { + // collect the instances (namespaces, e.g. "mqtt.0") that currently exist and the ones of this host + // whose resources the controller derives from their configuration + const existingInstances = new Set(); + const controllerManaged: ioBroker.InstanceObject[] = []; + const instanceView = await objects.getObjectViewAsync('system', 'instance', { + startkey: SYSTEM_ADAPTER_PREFIX, + endkey: `${SYSTEM_ADAPTER_PREFIX}\u9999`, + }); + for (const row of instanceView.rows) { + const instance = row.value; + if (!instance?._id) { + continue; + } + // Only instances of this host count: the registry is per host, so an instance that was + // moved elsewhere while this controller was down must not keep its entries alive here - + // nothing would ever remove them, because no object change follows a move that already + // happened. + if (instance.common?.host !== hostname) { + continue; + } + + existingInstances.add(instance._id.substring(SYSTEM_ADAPTER_PREFIX.length)); + if (!instance.common.declareUsedResources) { + controllerManaged.push(instance); + } + } + + // types whose content changed while loading and must be written back + const changedTypes = new Set(); + + const keys = (await states.getKeys(`${hostObjectPrefix}.usedResources.*`)) || []; + const values = keys.length ? (await states.getStates(keys)) || [] : []; + for (let i = 0; i < keys.length; i++) { + const state = values[i]; + if (!state || typeof state.val !== 'string' || !state.val) { + continue; + } + const type = keys[i].split('.').pop(); + if (!isValidUsedResourceType(type)) { + logger.warn(`${hostLogPrefix} Ignoring used resources of invalid type in "${keys[i]}"`); + continue; + } + try { + const parsed: unknown = JSON.parse(state.val); + if (Array.isArray(parsed)) { + // drop entries that do not have the expected shape, so nothing malformed enters the registry + const valid = parsed.filter(entry => isRegisteredResource(entry)); + if (valid.length !== parsed.length) { + logger.warn( + `${hostLogPrefix} Ignoring ${parsed.length - valid.length} malformed used resource(s) of type "${type}"`, + ); + changedTypes.add(type); + } + usedResources.setType(type, valid); + } + } catch { + // ignore malformed content + } + } + + // reset blocking flags and drop resources of no longer existing instances + for (const type of usedResources.assess(existingInstances)) { + changedTypes.add(type); + } + + // (re)derive the resources of the instances the controller manages itself, so that their configured + // ports are listed no matter whether they were ever started + for (const instance of controllerManaged) { + for (const type of seedUsedResourcesOfInstance(instance)) { + changedTypes.add(type); + } + } + + await persistUsedResourceTypes([...changedTypes]); + } catch (e) { + logger.warn(`${hostLogPrefix} Cannot load used resources: ${e.message}`); + } +} + +/** + * Determine the TCP port an instance occupies according to its configuration. + * + * @param instance the instance object + * @returns the resource payload for `native.port` (plus `native.bind` if set), or null if no port is configured + */ +function getConfiguredTcpPort(instance: ioBroker.InstanceObject): ioBroker.TcpPortResourceData | null { + const port = instance.native?.port; + const portNumber = + typeof port === 'number' ? port : typeof port === 'string' && port.trim() !== '' ? Number(port) : Number.NaN; + // port 0 means "pick a free one at runtime", so it does not occupy anything + if (!Number.isInteger(portNumber) || portNumber <= 0 || portNumber > 65_535) { + return null; + } + + const data: ioBroker.TcpPortResourceData = { port: portNumber }; + // if the instance also binds to a specific interface, record it together with the port + const bind = instance.native?.bind; + if (typeof bind === 'string' && bind.trim() !== '') { + data.bind = bind; + } + + return data; +} + +/** + * Derive the used resources of an instance the controller manages itself (`common.declareUsedResources` not set) + * from its configuration and replace what was derived for it before. + * + * Deriving from the object instead of registering on process start is what makes the registry answer the + * question it exists for: a port configured for an instance that was never started, or that is currently + * stopped, is listed as well - and a changed `native.port` is picked up right away instead of at the next + * restart. + * + * @param instance the instance object + * @returns the resource types that changed and should be persisted + */ +function seedUsedResourcesOfInstance(instance: ioBroker.InstanceObject): ioBroker.UsedResourceType[] { + const namespace = instance._id.substring(SYSTEM_ADAPTER_PREFIX.length); + const changed = new Set(usedResources.removeInstance(namespace)); + + const data = getConfiguredTcpPort(instance); + if (data) { + for (const type of usedResources.register('tcpPort', data, namespace)) { + changed.add(type); + } + // register() marks a resource as actively held, which is only true while the instance runs + const isRunning = !!procs[instance._id]?.process; + for (const type of usedResources.setInstanceBlocked(namespace, isRunning)) { + changed.add(type); + } + } + + return [...changed]; +} + +/** + * Bring the used-resources registry in line with an instance that is about to run. + * + * Called from the paths that actually launch a process, not from {@link startInstance} as a whole: a + * redundant call for an instance that is already running must not touch the registry, because nothing + * would re-register afterwards and the live entries would simply be gone. + * + * @param id the instance id, e.g. "system.adapter.mqtt.0" + * @param instance the instance object + */ +async function markInstanceResourcesStarting(id: string, instance: ioBroker.InstanceObject): Promise { + if (compactGroupController) { + return; + } + + const namespace = id.substring(SYSTEM_ADAPTER_PREFIX.length); + + await persistUsedResourceTypes( + instance.common.declareUsedResources + ? // the adapter declares its resources itself: drop what it declared before this (re)start, + // because the settings may have changed in between. What it registers now is additive. + usedResources.removeInstance(namespace) + : // the resources derived from the configuration are held again as soon as the instance runs + usedResources.setInstanceBlocked(namespace, true), + ); +} + +/** + * Bring the used-resources registry in line with the current state of an instance object. This is the single + * place where the configuration of an instance enters the registry: + * + * - an instance that was deleted or moved to another host loses all its entries; + * - an instance the controller manages itself gets its entries derived from its configuration; + * - an adapter-managed instance (`common.declareUsedResources`) is left alone - it declares its resources itself + * while it runs, and {@link startInstance} drops the previous declarations when it starts. + * + * @param id the instance id, e.g. "system.adapter.mqtt.0" + * @param obj the current instance object, or null if the instance was deleted + */ +async function syncUsedResourcesOfInstance( + id: ioBroker.ObjectIDs.Instance, + obj: ioBroker.InstanceObject | null, +): Promise { + if (compactGroupController) { + // the registry of a host is owned by its main controller + return; + } + + const namespace = id.substring(SYSTEM_ADAPTER_PREFIX.length); + let changed: ioBroker.UsedResourceType[]; + + if (!obj?.common || obj.common.host !== hostname) { + // deleted or moved to another host: this host does not track its resources anymore + changed = usedResources.removeInstance(namespace); + } else if (!obj.common.declareUsedResources) { + changed = seedUsedResourcesOfInstance(obj); + } else { + return; + } + + await persistUsedResourceTypes(changed); +} + +/** + * Determine which instance an incoming used-resources host message belongs to. + * + * The instance is derived from `msg.from` and not taken from the message body: the host message box is + * reachable by everything that may `sendToHost`, so trusting `msg.message.instance` would let one instance + * register resources in the name of another - or free another one's registrations of a whole type. A body + * that claims a different instance is rejected instead of being silently corrected, so a caller that got it + * wrong notices. (`from` is written by the sender as well, so this is a plausibility check and not an + * authentication of the sender.) + * + * @param msg the received host message + * @returns the namespace of the instance the message belongs to, e.g. "mqtt.0" + */ +function getUsedResourceMessageInstance(msg: ioBroker.SendableMessage): string { + const from = typeof msg.from === 'string' ? msg.from : ''; + if (!from.startsWith(SYSTEM_ADAPTER_PREFIX) || from.length === SYSTEM_ADAPTER_PREFIX.length) { + throw new Error(`used resources can only be modified by an instance, but sender is "${from || 'unknown'}"`); + } + const instance = from.substring(SYSTEM_ADAPTER_PREFIX.length); + + const claimedInstance: unknown = msg.message?.instance; + if (claimedInstance !== undefined && claimedInstance !== instance) { + throw new Error( + `instance "${instance}" must not modify the used resources of ${JSON.stringify(claimedInstance)}`, + ); + } + + // An instance without the flag is controller-managed: its entries are derived from the instance + // object, and the next change to that object replaces whatever it registered here. Accepting the + // call would look like it worked and the entry would vanish later for an unrelated reason, so it + // is refused with something the adapter developer can act on. The config comes from `procs`, so + // this costs no database read. + const config = procs[`${SYSTEM_ADAPTER_PREFIX}${instance}` as ioBroker.ObjectIDs.Instance]?.config; + if (config && !config.common?.declareUsedResources) { + throw new Error( + `instance "${instance}" does not declare its used resources - add "common.declareUsedResources": true to its io-package.json`, + ); + } + + return instance; +} + +/** + * Validate an incoming `registerUsedResource` / `freeUsedResource` host message. + * + * @param msg the received host message + * @param dataRequired whether the payload is mandatory - it is for `registerUsedResource` + * @returns the validated instance, resource type and payload + */ +function parseUsedResourceMessage( + msg: ioBroker.SendableMessage, + dataRequired: boolean, +): { instance: string; type: ioBroker.UsedResourceType; data: ioBroker.UsedResourceData | undefined } { + const instance = getUsedResourceMessageInstance(msg); + + // the type becomes the last segment of "system.host..usedResources.", so it must be validated + const type: unknown = msg.message?.type; + if (!isValidUsedResourceType(type)) { + throw new Error(`invalid resource type ${JSON.stringify(type)}`); + } + + const data: unknown = msg.message?.data; + if (data === undefined) { + if (dataRequired) { + throw new Error(`missing payload for resource type "${type}"`); + } + } else if (typeof data !== 'object' || data === null || Array.isArray(data)) { + throw new Error(`invalid payload for resource type "${type}"`); + } + + return { instance, type, data: data as ioBroker.UsedResourceData | undefined }; +} + /** * Process message to controller, like execute some script * @@ -3165,6 +3555,61 @@ async function processMessage(msg: ioBroker.SendableMessage): Promise void): Promise { `${hostLogPrefix} instance ${instance._id} in version "${instance.common.version}"${!isNpm ? ` (non-npm: ${instance.common.installedFrom})` : ''} started with pid ${proc.process.pid}`, ); + // the scheduled run holds the resources of this instance until it exits again - same + // handling as any other start + markInstanceResourcesStarting(id, instance).catch(e => + logger.warn(`${hostLogPrefix} Cannot update used resources of ${id}: ${e.message}`), + ); + proc.process.on('exit', (code, signal) => { outputCount++; states! .setState(`${id}.alive`, { val: false, ack: true, from: hostObjectPrefix }) .catch(e => logger.error(`${hostLogPrefix} Cannot set ${id}.alive: ${e.message}`)); + + // the instance is no longer running: keep its resource registrations but mark them as not + // actively blocked, so the user still sees which resources it would occupy when started again + const instanceNamespace = id.startsWith(`${SYSTEM_ADAPTER_PREFIX}`) + ? id.substring(SYSTEM_ADAPTER_PREFIX.length) + : id; + persistUsedResourceTypes(usedResources.setInstanceBlocked(instanceNamespace, false)).catch(e => + logger.warn(`${hostLogPrefix} Cannot update used resources of ${id}: ${e.message}`), + ); + if (signal) { logger.warn(`${hostLogPrefix} instance ${id} terminated due to ${signal}`); } else if (code === null) { @@ -4263,6 +4724,8 @@ async function startInstance(id: ioBroker.ObjectIDs.Instance, wakeUp = false): P delete proc.stopping; } + await markInstanceResourcesStarting(id, instance); + logger.debug( `${hostLogPrefix} startInstance ${name}.${instanceNo} loglevel=${loglevel}, compact=${ instance.common.compact && instance.common.runAsCompactMode @@ -5030,6 +5493,9 @@ async function startInstance(id: ioBroker.ObjectIDs.Instance, wakeUp = false): P } if (proc.process) { storePids(); + // this forks an adapter right away, so it needs the same resource handling as any + // other start - the exit handler below unblocks them again + await markInstanceResourcesStarting(id, instance); const isNpm = isInstalledFromNpm({ installedFrom: instance.common.installedFrom, adapterName: instance.common.name, @@ -5047,6 +5513,16 @@ async function startInstance(id: ioBroker.ObjectIDs.Instance, wakeUp = false): P states! .setState(`${id}.alive`, { val: false, ack: true, from: hostObjectPrefix }) .catch(e => logger.error(`${hostLogPrefix} Cannot set ${id}.alive: ${e.message}`)); + + // the init run is over: keep the registrations but mark them as no longer held + if (!compactGroupController) { + persistUsedResourceTypes( + usedResources.setInstanceBlocked(id.substring(SYSTEM_ADAPTER_PREFIX.length), false), + ).catch(e => + logger.warn(`${hostLogPrefix} Cannot update used resources of ${id}: ${e.message}`), + ); + } + if (signal) { logger.warn(`${hostLogPrefix} instance ${id} terminated due to ${signal}`); } else if (code === null) { @@ -6036,6 +6512,9 @@ async function setInstanceOfflineStates(id: ioBroker.ObjectIDs.Instance): Promis const adapterInstance = id.substring(SYSTEM_ADAPTER_PREFIX.length); + // the instance is no longer running: keep its resource registrations but mark them as not actively blocked + await persistUsedResourceTypes(usedResources.setInstanceBlocked(adapterInstance, false)); + const state = await states!.getState(`${adapterInstance}.info.connection`); if (state?.val === true) { diff --git a/packages/controller/test/testUsedResources.ts b/packages/controller/test/testUsedResources.ts new file mode 100644 index 0000000000..4bd617083b --- /dev/null +++ b/packages/controller/test/testUsedResources.ts @@ -0,0 +1,459 @@ +import assert from 'node:assert/strict'; +import { + getUsedResourceKey, + isRegisteredResource, + isValidUsedResourceType, + matchesUsedResourceData, + UsedResourcesRegistry, +} from '../src/lib/usedResources.js'; + +/** Fixed clock so that the `ts` of registered resources is deterministic in the tests */ +const FIXED_TS = 1_700_000_000_000; +const newRegistry = (): UsedResourcesRegistry => new UsedResourcesRegistry({ now: () => FIXED_TS }); + +describe('lib/usedResources: getUsedResourceKey', () => { + it('ignores the bookkeeping fields ts and isBlocked', () => { + const a = { + type: 'serialPort', + instance: 'mqtt.0', + data: { port: '/dev/ttyUSB0' }, + ts: 1, + isBlocked: true, + } as any; + const b = { + type: 'serialPort', + instance: 'mqtt.0', + data: { port: '/dev/ttyUSB0' }, + ts: 999, + isBlocked: false, + } as any; + assert.strictEqual(getUsedResourceKey(a), getUsedResourceKey(b)); + }); + + it('is stable regardless of payload property order', () => { + const a = { type: 'tcpPort', instance: 'web.0', data: { port: 8081, bind: '0.0.0.0' } } as any; + const b = { type: 'tcpPort', instance: 'web.0', data: { bind: '0.0.0.0', port: 8081 } } as any; + assert.strictEqual(getUsedResourceKey(a), getUsedResourceKey(b)); + }); + + it('differs for different instance, type or payload', () => { + const base = { type: 'tcpPort', instance: 'web.0', data: { port: 8081 } } as any; + const otherInstance = { ...base, instance: 'web.1' }; + const otherType = { ...base, type: 'udpPort' }; + const otherPayload = { ...base, data: { port: 9090 } }; + const keys = new Set([ + getUsedResourceKey(base), + getUsedResourceKey(otherInstance), + getUsedResourceKey(otherType), + getUsedResourceKey(otherPayload), + ]); + assert.strictEqual(keys.size, 4); + }); + + it('keeps values of different types apart', () => { + const asNumber = { type: 'tcpPort', instance: 'web.0', data: { port: 80 } } as any; + const asString = { type: 'tcpPort', instance: 'web.0', data: { port: '80' } } as any; + assert.notStrictEqual(getUsedResourceKey(asNumber), getUsedResourceKey(asString)); + }); + + it('keeps structured payload values apart', () => { + const a = { type: 'usb', instance: 'a.0', data: { path: { bus: 1 } } } as any; + const b = { type: 'usb', instance: 'a.0', data: { path: { bus: 2 } } } as any; + assert.notStrictEqual(getUsedResourceKey(a), getUsedResourceKey(b)); + }); + + it('treats an explicitly undefined payload field like an omitted one', () => { + const explicit = { type: 'tcpPort', instance: 'web.0', data: { port: 8080, bind: undefined } } as any; + const omitted = { type: 'tcpPort', instance: 'web.0', data: { port: 8080 } } as any; + assert.strictEqual(getUsedResourceKey(explicit), getUsedResourceKey(omitted)); + }); + + it('handles a missing payload', () => { + const withoutData = { type: 'gpio', instance: 'rpi.0', data: undefined } as any; + assert.strictEqual(getUsedResourceKey(withoutData), 'rpi.0|gpio|'); + }); +}); + +describe('lib/usedResources: isValidUsedResourceType', () => { + it('accepts known and custom type names', () => { + assert.ok(isValidUsedResourceType('tcpPort')); + assert.ok(isValidUsedResourceType('canBus')); + assert.ok(isValidUsedResourceType('my-custom_1')); + }); + + it('rejects anything that would break the state id', () => { + assert.ok(!isValidUsedResourceType(undefined)); + assert.ok(!isValidUsedResourceType(null)); + assert.ok(!isValidUsedResourceType(42)); + assert.ok(!isValidUsedResourceType('')); + assert.ok(!isValidUsedResourceType('with.dot')); + assert.ok(!isValidUsedResourceType('with space')); + assert.ok(!isValidUsedResourceType('with*star')); + }); +}); + +describe('lib/usedResources: isRegisteredResource', () => { + const valid = { type: 'tcpPort', data: { port: 8080 }, instance: 'web.0', ts: 1, isBlocked: false }; + + it('accepts a complete entry', () => { + assert.ok(isRegisteredResource(valid)); + }); + + it('rejects entries with a missing or malformed field', () => { + assert.ok(!isRegisteredResource(null)); + assert.ok(!isRegisteredResource('nope')); + assert.ok(!isRegisteredResource({ ...valid, type: undefined })); + assert.ok(!isRegisteredResource({ ...valid, instance: '' })); + assert.ok(!isRegisteredResource({ ...valid, ts: 'now' })); + assert.ok(!isRegisteredResource({ ...valid, isBlocked: 'yes' })); + assert.ok(!isRegisteredResource({ ...valid, data: undefined })); + assert.ok(!isRegisteredResource({ ...valid, data: [] })); + }); + + it('rejects an entry in the old flat format', () => { + assert.ok(!isRegisteredResource({ type: 'tcpPort', port: 8080, instance: 'web.0', ts: 1, isBlocked: false })); + }); +}); + +describe('lib/usedResources: UsedResourcesRegistry.register', () => { + it('adds a resource as blocked with the injected timestamp', () => { + const reg = newRegistry(); + const changed = reg.register('serialPort', { port: '/dev/ttyUSB0' }, 'mqtt.0'); + + assert.deepStrictEqual(changed, ['serialPort']); + const all = reg.get(); + assert.strictEqual(all.length, 1); + assert.deepStrictEqual(all[0], { + type: 'serialPort', + data: { port: '/dev/ttyUSB0' }, + instance: 'mqtt.0', + ts: FIXED_TS, + isBlocked: true, + }); + }); + + it('does not let the payload overwrite the bookkeeping fields', () => { + const reg = newRegistry(); + // a payload carrying keys that also exist as bookkeeping fields must not take them over + reg.register('tcpPort', { port: 80, instance: 'evil.0', isBlocked: false, type: 'gpio' } as any, 'mqtt.0'); + + const [entry] = reg.get('tcpPort'); + assert.strictEqual(entry.type, 'tcpPort'); + assert.strictEqual(entry.instance, 'mqtt.0'); + assert.strictEqual(entry.isBlocked, true); + assert.strictEqual(entry.ts, FIXED_TS); + + // ... and the entry stays reachable for all by-instance operations + assert.deepStrictEqual(reg.setInstanceBlocked('mqtt.0', false), ['tcpPort']); + assert.deepStrictEqual(reg.removeInstance('mqtt.0'), ['tcpPort']); + assert.deepStrictEqual(reg.get(), []); + }); + + it('is additive and independent of the call order', () => { + const reg = newRegistry(); + reg.register('serialPort', { port: '/dev/ttyUSB0' }, 'mqtt.0'); + const changed = reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + reg.register('tcpPort', { port: 8081 }, 'mqtt.0'); + + // registering one type must not touch the entries of another one + assert.deepStrictEqual(changed, ['tcpPort']); + assert.strictEqual(reg.get('serialPort').length, 1); + assert.strictEqual(reg.get('tcpPort').length, 2); + assert.strictEqual(reg.get().length, 3); + }); + + it('drops the previous registrations only on the explicit removeInstance', () => { + const reg = newRegistry(); + reg.register('serialPort', { port: '/dev/ttyUSB0' }, 'mqtt.0'); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + + // what the controller does when the instance (re)starts with a possibly changed configuration + assert.deepStrictEqual(reg.removeInstance('mqtt.0').sort(), ['serialPort', 'tcpPort']); + assert.deepStrictEqual(reg.get(), []); + }); + + it('does not duplicate an identical registration but refreshes it', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + + assert.strictEqual(reg.get('tcpPort').length, 1); + }); + + it('keeps resources of different instances side by side', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + reg.register('tcpPort', { port: 1884 }, 'mqtt.1'); + + assert.strictEqual(reg.get('tcpPort').length, 2); + }); +}); + +describe('lib/usedResources: matchesUsedResourceData', () => { + const data = { port: 8080, bind: '0.0.0.0' } as any; + + it('matches when the filter is omitted or empty', () => { + assert.ok(matchesUsedResourceData(data, undefined)); + assert.ok(matchesUsedResourceData(data, {})); + }); + + it('ignores the fields the filter does not name', () => { + assert.ok(matchesUsedResourceData(data, { port: 8080 })); + assert.ok(matchesUsedResourceData(data, { bind: '0.0.0.0' })); + }); + + it('does not match when a named field differs', () => { + assert.ok(!matchesUsedResourceData(data, { port: 8081 })); + assert.ok(!matchesUsedResourceData(data, { bind: '127.0.0.1' })); + // a field the payload does not have at all + assert.ok(!matchesUsedResourceData(data, { family: 4 } as any)); + }); + + it('compares by value and type', () => { + assert.ok(!matchesUsedResourceData(data, { port: '8080' })); + assert.ok(matchesUsedResourceData({ path: { bus: 1 } } as any, { path: { bus: 1 } } as any)); + assert.ok(!matchesUsedResourceData({ path: { bus: 1 } } as any, { path: { bus: 2 } } as any)); + }); + + it('treats an explicitly undefined filter field as not named', () => { + assert.ok(matchesUsedResourceData(data, { port: 8080, family: undefined })); + }); +}); + +describe('lib/usedResources: UsedResourcesRegistry.free', () => { + it('frees a single resource identified by its payload', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + reg.register('tcpPort', { port: 8081 }, 'mqtt.0'); + + const changed = reg.free('tcpPort', { port: 8081 }, 'mqtt.0'); + assert.deepStrictEqual(changed, ['tcpPort']); + assert.deepStrictEqual( + reg.get('tcpPort').map(r => (r.data as ioBroker.TcpPortResourceData).port), + [1883], + ); + }); + + it('frees a resource that was registered with an explicitly undefined optional field', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 8080, bind: undefined }, 'web.0'); + + assert.deepStrictEqual(reg.free('tcpPort', { port: 8080 }, 'web.0'), ['tcpPort']); + assert.deepStrictEqual(reg.get('tcpPort'), []); + }); + + it('frees by a partial payload without repeating the optional fields', () => { + const reg = newRegistry(); + // this is what the controller derives itself when native.bind is set + reg.register('tcpPort', { port: 8080, bind: '0.0.0.0' }, 'web.0'); + + assert.deepStrictEqual(reg.free('tcpPort', { port: 8080 }, 'web.0'), ['tcpPort']); + assert.deepStrictEqual(reg.get('tcpPort'), []); + }); + + it('frees every entry the filter matches', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 8080, bind: '0.0.0.0' }, 'web.0'); + reg.register('tcpPort', { port: 8080, bind: '127.0.0.1' }, 'web.0'); + reg.register('tcpPort', { port: 9090 }, 'web.0'); + + assert.deepStrictEqual(reg.free('tcpPort', { port: 8080 }, 'web.0'), ['tcpPort']); + assert.deepStrictEqual( + reg.get('tcpPort').map(r => (r.data as ioBroker.TcpPortResourceData).port), + [9090], + ); + }); + + it('does not free more than the filter says', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 8080, bind: '0.0.0.0' }, 'web.0'); + + assert.deepStrictEqual(reg.free('tcpPort', { port: 8080, bind: '127.0.0.1' }, 'web.0'), []); + assert.strictEqual(reg.get('tcpPort').length, 1); + }); + + it('never reaches the resources of another instance', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 8080 }, 'web.0'); + reg.register('tcpPort', { port: 8080 }, 'web.1'); + + assert.deepStrictEqual(reg.free('tcpPort', { port: 8080 }, 'web.0'), ['tcpPort']); + assert.deepStrictEqual( + reg.get('tcpPort').map(r => r.instance), + ['web.1'], + ); + }); + + it('treats an empty filter like an omitted one', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 8080 }, 'web.0'); + reg.register('tcpPort', { port: 9090 }, 'web.0'); + + assert.deepStrictEqual(reg.free('tcpPort', {}, 'web.0'), ['tcpPort']); + assert.deepStrictEqual(reg.get('tcpPort'), []); + }); + + it('frees all resources of a type for the instance when no payload is given', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + reg.register('tcpPort', { port: 8081 }, 'mqtt.0'); + reg.register('tcpPort', { port: 1884 }, 'mqtt.1'); + + const changed = reg.free('tcpPort', undefined, 'mqtt.0'); + assert.deepStrictEqual(changed, ['tcpPort']); + assert.deepStrictEqual( + reg.get('tcpPort').map(r => r.instance), + ['mqtt.1'], + ); + }); + + it('reports no change when nothing matched', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + + assert.deepStrictEqual(reg.free('tcpPort', { port: 9999 }, 'mqtt.0'), []); + assert.deepStrictEqual(reg.free('serialPort', undefined, 'mqtt.0'), []); + assert.strictEqual(reg.get('tcpPort').length, 1); + }); +}); + +describe('lib/usedResources: UsedResourcesRegistry.setInstanceBlocked', () => { + it('toggles isBlocked across all types of an instance and reports changed types', () => { + const reg = newRegistry(); + reg.register('serialPort', { port: '/dev/ttyUSB0' }, 'mqtt.0'); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + reg.register('tcpPort', { port: 1884 }, 'other.0'); + + const changed = reg.setInstanceBlocked('mqtt.0', false); + assert.deepStrictEqual(changed.sort(), ['serialPort', 'tcpPort']); + for (const r of reg.get()) { + assert.strictEqual(r.isBlocked, r.instance !== 'mqtt.0'); + } + }); + + it('reports no change when the flag already has the desired value', () => { + const reg = newRegistry(); + reg.register('serialPort', { port: '/dev/ttyUSB0' }, 'mqtt.0'); // already blocked + assert.deepStrictEqual(reg.setInstanceBlocked('mqtt.0', true), []); + }); +}); + +describe('lib/usedResources: UsedResourcesRegistry.removeInstance', () => { + it('removes all resources of an instance across types', () => { + const reg = newRegistry(); + reg.register('serialPort', { port: '/dev/ttyUSB0' }, 'mqtt.0'); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + reg.register('tcpPort', { port: 1884 }, 'other.0'); + + const changed = reg.removeInstance('mqtt.0'); + assert.deepStrictEqual(changed.sort(), ['serialPort', 'tcpPort']); + assert.deepStrictEqual( + reg.get().map(r => r.instance), + ['other.0'], + ); + }); + + it('reports no change for an unknown instance', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + assert.deepStrictEqual(reg.removeInstance('nope.0'), []); + }); +}); + +describe('lib/usedResources: UsedResourcesRegistry.assess (controller start)', () => { + it('resets isBlocked and drops resources of no longer existing instances', () => { + const reg = newRegistry(); + // simulate a state loaded from persistence: two instances, blocked + reg.setType('tcpPort', [ + { type: 'tcpPort', data: { port: 1883 }, instance: 'mqtt.0', ts: 1, isBlocked: true }, + { type: 'tcpPort', data: { port: 1884 }, instance: 'deleted.0', ts: 1, isBlocked: true }, + ]); + + const changed = reg.assess(new Set(['mqtt.0'])); + assert.deepStrictEqual(changed, ['tcpPort']); + + const remaining = reg.get('tcpPort'); + assert.strictEqual(remaining.length, 1); + assert.strictEqual(remaining[0].instance, 'mqtt.0'); + assert.strictEqual(remaining[0].isBlocked, false); + }); + + it('reports no change when everything is already valid and unblocked', () => { + const reg = newRegistry(); + reg.setType('tcpPort', [ + { type: 'tcpPort', data: { port: 1883 }, instance: 'mqtt.0', ts: 1, isBlocked: false }, + ]); + + assert.deepStrictEqual(reg.assess(new Set(['mqtt.0'])), []); + }); +}); + +describe('lib/usedResources: UsedResourcesRegistry.get', () => { + it('returns copies so callers cannot mutate the internal state', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + + const list = reg.get('tcpPort'); + list.push({ type: 'tcpPort', data: { port: 1 }, instance: 'evil.0', ts: 0, isBlocked: true }); + assert.strictEqual(reg.get('tcpPort').length, 1); + }); + + it('returns deep copies so the nested payload cannot be mutated either', () => { + const reg = newRegistry(); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + + const [entry] = reg.get('tcpPort'); + (entry.data as ioBroker.TcpPortResourceData).port = 9999; + entry.instance = 'evil.0'; + entry.isBlocked = false; + + const [stored] = reg.get('tcpPort'); + assert.strictEqual((stored.data as ioBroker.TcpPortResourceData).port, 1883); + assert.strictEqual(stored.instance, 'mqtt.0'); + assert.strictEqual(stored.isBlocked, true); + }); + + it('getTypes lists the types that hold entries', () => { + const reg = newRegistry(); + reg.register('serialPort', { port: '/dev/ttyUSB0' }, 'mqtt.0'); + reg.register('tcpPort', { port: 1883 }, 'mqtt.0'); + assert.deepStrictEqual(reg.getTypes().sort(), ['serialPort', 'tcpPort']); + }); +}); + +describe('lib/usedResources: UsedResourcesRegistry empty types', () => { + it('drops a type when its last entry is freed', () => { + const reg = newRegistry(); + reg.register('gpio', { pin: 4 }, 'rpi.0'); + + assert.deepStrictEqual(reg.free('gpio', undefined, 'rpi.0'), ['gpio']); + assert.deepStrictEqual(reg.getTypes(), []); + assert.deepStrictEqual(reg.get(), []); + }); + + it('drops a type when its last entry is removed with the instance', () => { + const reg = newRegistry(); + reg.register('gpio', { pin: 4 }, 'rpi.0'); + reg.register('tcpPort', { port: 8080 }, 'web.0'); + + assert.deepStrictEqual(reg.removeInstance('rpi.0'), ['gpio']); + assert.deepStrictEqual(reg.getTypes(), ['tcpPort']); + }); + + it('drops a type whose entries are all dropped by the start assessment', () => { + const reg = newRegistry(); + reg.setType('tcpPort', [ + { type: 'tcpPort', data: { port: 1883 }, instance: 'deleted.0', ts: 1, isBlocked: true }, + ]); + + assert.deepStrictEqual(reg.assess(new Set(['mqtt.0'])), ['tcpPort']); + assert.deepStrictEqual(reg.getTypes(), []); + }); + + it('does not create a type when an empty list is loaded from the persisted state', () => { + const reg = newRegistry(); + reg.setType('tcpPort', []); + + assert.deepStrictEqual(reg.getTypes(), []); + }); +}); diff --git a/packages/types-dev/index.d.ts b/packages/types-dev/index.d.ts index 751533f27b..1108274199 100644 --- a/packages/types-dev/index.d.ts +++ b/packages/types-dev/index.d.ts @@ -79,7 +79,114 @@ declare global { | 'CONTROLLER_UI_UPGRADE' | 'ADAPTER_WEBSERVER_UPGRADE' | 'CONTROLLER_CMD_EXEC_FILES' - | 'CONTROLLER_FEATURE_REQUEST'; + | 'CONTROLLER_FEATURE_REQUEST' + | 'CONTROLLER_USED_RESOURCES'; + + // #region Used resources + // --------------------------------------------------------------------------------------------------- + // Exclusive resources occupied by adapter instances (serial ports, TCP/UDP ports, USB devices, ...). + // These are the resources that cannot be used by more than one instance at the same time. + // Each resource type has its own strictly typed payload; extend `UsedResourceDataMap` to add a new one. + // --------------------------------------------------------------------------------------------------- + + /** A serial port occupied by an instance */ + interface SerialPortResourceData { + /** System path or name of the serial port, e.g. "/dev/ttyUSB0" or "COM3" */ + port: string; + /** Baud rate the port is opened with, if known */ + baudRate?: number; + } + + /** A TCP port occupied by an instance */ + interface TcpPortResourceData { + /** TCP port number */ + port: number; + /** Address the socket is bound to. Default "0.0.0.0" (all interfaces) */ + bind?: string; + /** address family */ + family?: 4 | 6; + } + + /** A UDP port occupied by an instance */ + interface UdpPortResourceData { + /** UDP port number */ + port: number; + /** Address the socket is bound to. Default "0.0.0.0" (all interfaces) */ + bind?: string; + /** address family */ + family?: 4 | 6; + } + + /** A USB device occupied by an instance */ + interface UsbResourceData { + /** System path of the USB device, e.g. "/dev/bus/usb/001/004" or "\\\\.\\COM3" */ + path: string; + /** USB vendor id (hex string), e.g. "10c4" */ + vendorId?: string; + /** USB product id (hex string), e.g. "ea60" */ + productId?: string; + } + + /** A Bluetooth / HCI adapter occupied by an instance */ + interface BluetoothResourceData { + /** HCI device name or index, e.g. "hci0" */ + hci: string; + } + + /** A GPIO pin occupied by an instance */ + interface GpioResourceData { + /** GPIO pin number (BCM numbering) */ + pin: number; + } + + /** + * Maps every known resource type to its strictly typed payload. + * To introduce a new resource type, add its `RESOURCE_TYPE: RESOURCE_TYPE_Data` entry here + * (this map is intentionally open for module augmentation by adapters that own custom resources). + */ + interface UsedResourceDataMap { + serialPort: SerialPortResourceData; + tcpPort: TcpPortResourceData; + udpPort: UdpPortResourceData; + usb: UsbResourceData; + bluetooth: BluetoothResourceData; + gpio: GpioResourceData; + } + + /** Kind of an exclusive resource that can be occupied by only one instance at a time */ + type UsedResourceType = keyof UsedResourceDataMap; + + /** The type-specific payload for a given resource type (without bookkeeping fields) */ + type UsedResourceData = UsedResourceDataMap[T]; + + /** + * A registered resource as stored on the host: the discriminating `type`, the type-specific payload + * in `data` and the ownership/bookkeeping fields (`instance`, `ts`, `isBlocked`). + * + * The payload is nested on purpose. If it were merged into this object, a payload key could shadow a + * bookkeeping field - an entry could then claim a foreign `instance` or a `type` that does not match + * the bucket it is stored in, and would be unreachable for all by-instance operations. Nesting makes + * that impossible for every current and future payload type. + */ + type RegisteredResource = { + [K in T]: { + /** Kind of the occupied resource, e.g. "serialPort" */ + type: K; + /** The type-specific payload describing the resource, e.g. `{ port: '/dev/ttyUSB0' }` */ + data: UsedResourceDataMap[K]; + /** Instance that occupies the resource, e.g. "mqtt.0" */ + instance: string; + /** Timestamp (ms) when the resource was registered */ + ts: number; + /** + * If true, the instance is running and uses this resource. If false, the instance is not + * running and would maybe occupy this resource when started - "maybe", because its + * configuration can still change before the next start. + */ + isBlocked: boolean; + }; + }[T]; + // #endregion type StateValue = string | number | boolean | null; diff --git a/packages/types-dev/objects.d.ts b/packages/types-dev/objects.d.ts index c7c0ba4075..8677164398 100644 --- a/packages/types-dev/objects.d.ts +++ b/packages/types-dev/objects.d.ts @@ -842,6 +842,15 @@ declare global { | 'weather'; /** If `true`, the `npm` package must be installed with the `--unsafe-perm` flag */ unsafePerm?: true; + /** + * If `true`, the adapter declares the exclusive resources it occupies (serial ports, TCP/UDP ports, ...) + * itself via `adapter.registerUsedResource(...)`. Set this when the occupied resources are not simply + * the configured `native.port`. + * + * If not set, js-controller maintains the registry for this adapter and derives the entries from the + * instance's `native.port` / `native.bind`. + */ + declareUsedResources?: boolean; /** The available version in the ioBroker repo. */ version: string; /** Definition of the vis-2 widgets */