diff --git a/COLLABORATORS.md b/COLLABORATORS.md index 72472faed5..adca76c944 100644 --- a/COLLABORATORS.md +++ b/COLLABORATORS.md @@ -21,8 +21,8 @@ new major version, which the maintainers plan. What counts as breaking and how it maps to version numbers is already documented: -- [API development](docs/pages/development/3-api.rst) — API versioning rules. -- [Releasing — Semantic versioning](docs/pages/development/2-releasing.rst) — +- [API development](docs/README.md#api-development-guidelines) — API versioning rules. +- [Releasing — Semantic versioning](RELEASING.md#semantic-versioning) — how "API" is scoped (it also covers config and network protocols) and which version part a change bumps. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..6cf5d7b8fe --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,3 @@ +# Contributing to nuts-node + +See [COLLABORATORS.md](COLLABORATORS.md) for how to contribute to `nuts-node`, and [DEVELOPMENT.md](DEVELOPMENT.md) for setting up a development environment. diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 0000000000..032c12646a --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,105 @@ +# Developing nuts-node + +## Requirements + +![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/nuts-foundation/nuts-node) +or higher is required. + +## Building + +Just use `go build`. + +### ES256 Koblitz support + +To enable ES256K (Koblitz) support, you need to build with the `jwx_es256k` tag: + +```shell +go build -tags jwx_es256k +``` + +## Running tests + +Tests can be run by executing + +```shell +go test ./... +``` + +## Code generation + +Code generation is used for generating mocks, OpenAPI client- and servers, and gRPC services. +Make sure that `GOPATH/bin` is available on `PATH` and that the dependencies are installed. + +Install `protoc`: + +- MacOS: `brew install protobuf` +- Linux: `apt install -y protobuf-compiler` + +Install Go tools: + +```shell +make install-tools +``` + +Generating code: + +To regenerate all code run the `run-generators` target from the makefile, or use one of the following for a specific group: + +| Group | Command | +|-----------------|------------------------| +| Mocks | `make gen-mocks` | +| OpenApi | `make gen-api` | +| Protobuf + gRPC | `make gen-protobuf` | +| All | `make run-generators` | + +See [docs/README.md](docs/README.md) for API development guidelines (OpenAPI contract-first workflow, versioning, error responses). + +### Documentation + +The documentation is automatically built on readthedocs based on the config in `.readthedocs.yaml`. +All files to be included can be generated using: + +```shell +make cli-docs +``` + +This regenerates the config-option and CLI reference tables from code, and the root `README.rst` from `README_template.rst`. +Whenever you add, remove, or change a config flag or CLI command, run this and commit the regenerated files — +don't hand-edit `docs/pages/configuration/server_options*.rst` or `docs/pages/operations/cli-reference.rst`. + +If needed, you can also build the documentation locally in `/docs/_build` using Docker; see [docs/README.md](docs/README.md). + +## Developing with Vault + +You can start a development Vault server as follows: + +```shell +docker run --cap-add=IPC_LOCK -d -p 8200:8200 \ +-e 'VAULT_DEV_ROOT_TOKEN_ID=unsafe' -e 'VAULT_ADDRESS=http://localhost:8200' \ +--name=dev-vault \ +vault +``` + +The server will start unsealed, with root token `unsafe`. + +Now log in and enable a key-value secret engine named `kv`: + +```shell +docker exec -e 'VAULT_ADDR=http://0.0.0.0:8200' dev-vault vault login +``` + +Enter the root token `unsafe`, then enable the `kv` engine: + +```shell +docker exec -e 'VAULT_ADDR=http://0.0.0.0:8200' dev-vault vault secrets enable -path=kv kv +``` + +Then configure the Nuts node to use the Vault server: + +```yaml +crypto: + storage: vaultkv + vault: + address: http://localhost:8200 + token: unsafe +``` diff --git a/README.rst b/README.rst index 31d1448529..3f8030fb7e 100644 --- a/README.rst +++ b/README.rst @@ -46,82 +46,7 @@ Claude Code skill files for common maintenance tasks can be found in the `.claud Development ^^^^^^^^^^^ -.. |gover| image:: https://img.shields.io/github/go-mod/go-version/nuts-foundation/nuts-node - :alt: GitHub go.mod Go version - -|gover| or higher is required. - -Building -******** - -Just use ``go build``. - -ES256 Koblitz support -===================== - -To enable ES256K (Koblitz) support, you need to build with the ``jwx_es256k`` tag: - -.. code-block:: shell - - go build -tags jwx_es256k - -Running tests -************* - -Tests can be run by executing - -.. code-block:: shell - - go test ./... - -Code Generation -*************** - -Code generation is used for generating mocks, OpenAPI client- and servers, and gRPC services. -Make sure that ``GOPATH/bin`` is available on ``PATH`` and that the dependencies are installed - -Install ``protoc``: - - | MacOS: ``brew install protobuf`` - | Linux: ``apt install -y protobuf-compiler`` - -Install Go tools: - -.. code-block:: shell - - make install-tools - -Generating code: - -To regenerate all code run the ``run-generators`` target from the makefile or use one of the following for a specific group - -================ ======================= -Group Command -================ ======================= -Mocks ``make gen-mocks`` -OpenApi ``make gen-api`` -Protobuf + gRCP ``make gen-protobuf`` -All ``make run-generators`` -================ ======================= - -Documentation -============= - -The documentation is automatically build on readthedocs based on the config in ``.readthedocs.yaml``. -All files to be included can be generated using: - -.. code-block:: shell - - make cli-docs - -This regenerates files from code, and the ``README.rst`` file which requires python package ``rst-include`` (``pip install rst-include``). - -If needed, you can also build the documentation locally in ``/docs/_build`` using docker: - -.. code-block:: shell - - docker build -t local/nuts-node-docs ./docs - docker run --rm -v ./docs:/docs local/nuts-node-docs +See `DEVELOPMENT.md `_ for build, test and code generation instructions. Configuration ^^^^^^^^^^^^^ @@ -169,147 +94,223 @@ The following options can be configured on the server: .. marker-for-config-options -.. table:: Server Options +.. list-table:: Server Options :widths: 20 30 50 :class: options-table - - ======================================== =================================================================================================================================================================================================================================================================================================================================================================================================================================================================== ============================================================================================================================================================================================================================================================================================================================================ - Key Default Description - ======================================== =================================================================================================================================================================================================================================================================================================================================================================================================================================================================== ============================================================================================================================================================================================================================================================================================================================================ - configfile ./config/nuts.yaml Nuts config file - cpuprofile When set, a CPU profile is written to the given path. Ignored when strictmode is set. - datadir ./data Directory where the node stores its files. - didmethods [web,nuts] Comma-separated list of enabled DID methods (without did: prefix). It also controls the order in which DIDs are returned by APIs, and which DID is used for signing if the verifying party does not impose restrictions on the DID method used. - internalratelimiter true When set, expensive internal calls are rate-limited to protect the network. Always enabled in strict mode. - loggerformat text Log format (text, json) - strictmode true When set, insecure settings are forbidden. - url Public facing URL of the server (required). Must be HTTPS when strictmode is set. - verbosity info Log level (trace, debug, info, warn, error) - httpclient.timeout 30s Request time-out for HTTP clients, such as '10s'. Refer to Golang's 'time.Duration' syntax for a more elaborate description of the syntax. - **Auth** - auth.authorizationendpoint.enabled false enables the v2 API's OAuth2 Authorization Endpoint, used by OpenID4VP and OpenID4VCI. This flag might be removed in a future version (or its default become 'true') as the use cases and implementation of OpenID4VP and OpenID4VCI mature. - **Crypto** - crypto.storage Storage to use, 'fs' for file system (for development purposes), 'vaultkv' for HashiCorp Vault KV store, 'azure-keyvault' for Azure Key Vault, 'external' for an external backend (deprecated). - crypto.azurekv.hsm false Whether to store the key in a hardware security module (HSM). If true, the Azure Key Vault must be configured for HSM usage. Default: false - crypto.azurekv.timeout 10s Timeout of client calls to Azure Key Vault, in Golang time.Duration string format (e.g. 10s). - crypto.azurekv.url The URL of the Azure Key Vault. - crypto.azurekv.auth.type default Credential type to use when authenticating to the Azure Key Vault. Options: default, managed_identity (see https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/azidentity/README.md for an explanation of the options). - crypto.vault.address The Vault address. If set it overwrites the VAULT_ADDR env var. - crypto.vault.pathprefix kv The Vault path prefix. - crypto.vault.timeout 5s Timeout of client calls to Vault, in Golang time.Duration string format (e.g. 1s). - crypto.vault.token The Vault token. If set it overwrites the VAULT_TOKEN env var. - **Discovery** - discovery.client.refreshinterval 10m0s Interval at which the client synchronizes with the Discovery Server; refreshing Verifiable Presentations of local DIDs and loading changes, updating the local copy. It only will actually refresh registrations of local DIDs that about to expire (less than 1/4th of their lifetime left). Specified as Golang duration (e.g. 1m, 1h30m). - discovery.definitions.directory ./config/discovery Directory to load Discovery Service Definitions from. If not set, the discovery service will be disabled. If the directory contains JSON files that can't be parsed as service definition, the node will fail to start. - discovery.server.ids [] IDs of the Discovery Service for which to act as server. If an ID does not map to a loaded service definition, the node will fail to start. - **HTTP** - http.clientipheader X-Forwarded-For Case-sensitive HTTP Header that contains the client IP used for audit logs. For the X-Forwarded-For header only link-local, loopback, and private IPs are excluded. Switch to X-Real-IP or a custom header if you see your own proxy/infra in the logs. - http.log metadata What to log about HTTP requests. Options are 'nothing', 'metadata' (log request method, URI, IP and response code), and 'metadata-and-body' (log the request and response body, in addition to the metadata). When debug vebosity is set the authorization headers are also logged when the request is fully logged. - http.cache.maxbytes 10485760 HTTP client maximum size of the response cache in bytes. If 0, the HTTP client does not cache responses. - http.internal.address 127.0.0.1:8081 Address and port the server will be listening to for internal-facing endpoints. - http.internal.auth.audience Expected audience for JWT tokens (default: hostname) - http.internal.auth.authorizedkeyspath Path to an authorized_keys file for trusted JWT signers - http.internal.auth.type Whether to enable authentication for /internal endpoints, specify 'token_v2' for bearer token mode or 'token' for legacy bearer token mode. - http.public.address \:8080 Address and port the server will be listening to for public-facing endpoints. - **JSONLD** - jsonld.contexts.localmapping [https://nuts.nl/credentials/2024=assets/contexts/nuts-2024.ldjson,https://nuts.nl/credentials/v1=assets/contexts/nuts.ldjson,https://schema.org=assets/contexts/schema-org-v13.ldjson,https://w3c-ccg.github.io/lds-jws2020/contexts/lds-jws2020-v1.json=assets/contexts/lds-jws2020-v1.ldjson,https://w3id.org/vc/status-list/2021/v1=assets/contexts/w3c-statuslist2021.ldjson,https://www.w3.org/2018/credentials/v1=assets/contexts/w3c-credentials-v1.ldjson] This setting allows mapping external URLs to local files for e.g. preventing external dependencies. These mappings have precedence over those in remoteallowlist. - jsonld.contexts.remoteallowlist [https://schema.org,https://www.w3.org/2018/credentials/v1,https://w3c-ccg.github.io/lds-jws2020/contexts/lds-jws2020-v1.json,https://w3id.org/vc/status-list/2021/v1] In strict mode, fetching external JSON-LD contexts is not allowed except for context-URLs listed here. - **PKI** - pki.maxupdatefailhours 4 Maximum number of hours that a denylist update can fail - pki.softfail true Do not reject certificates if their revocation status cannot be established when softfail is true - **Storage** - storage.debug false When true, enables extra logging of storage-layer problems (e.g. performance issues). - storage.session.memcached.address [] List of Memcached server addresses. These can be a simple 'host:port' or a Memcached connection URL with scheme, auth and other options. - storage.session.redis.address Redis session database server address. This can be a simple 'host:port' or a Redis connection URL with scheme, auth and other options. If not set it, defaults to an in-memory database. - storage.session.redis.database Redis session database name, which is used as prefix every key. Can be used to have multiple instances use the same Redis instance. - storage.session.redis.password Redis session database password. If set, it overrides the username in the connection URL. - storage.session.redis.username Redis session database username. If set, it overrides the username in the connection URL. - storage.session.redis.sentinel.master Name of the Redis Sentinel master. Setting this property enables Redis Sentinel. - storage.session.redis.sentinel.nodes [] Addresses of the Redis Sentinels to connect to initially. Setting this property enables Redis Sentinel. - storage.session.redis.sentinel.password Password for authenticating to Redis Sentinels. - storage.session.redis.sentinel.username Username for authenticating to Redis Sentinels. - storage.session.redis.tls.truststorefile PEM file containing the trusted CA certificate(s) for authenticating remote Redis session servers. Can only be used when connecting over TLS (use 'rediss://' as scheme in address). - storage.sql.connection Connection string for the SQL database. If not set it, defaults to a SQLite database stored inside the configured data directory. Note: using SQLite is not recommended in production environments. If using SQLite anyways, remember to enable foreign keys ('_foreign_keys=on') and the write-ahead-log ('_journal_mode=WAL'). - storage.sql.rdsiam.dbuser Database username for IAM authentication. If not specified, the username from the connection string will be used. The database user must be created with IAM authentication enabled. - storage.sql.rdsiam.enabled false Enable AWS RDS IAM authentication for the SQL database connection. When enabled, the node will use temporary IAM tokens instead of passwords. Requires the connection string to be a PostgreSQL or MySQL RDS endpoint without a password. - storage.sql.rdsiam.region AWS region where the RDS instance is located (e.g., 'us-east-1). Required when RDS IAM authentication is enabled. - storage.sql.rdsiam.tokenrefreshinterval 14m0s Interval at which to refresh the IAM authentication token. RDS tokens are valid for 15 minutes, so set this to ensure tokens are refreshed before expiry. Specified as Golang duration (e.g. 10m, 1h). - **Tracing** - tracing.endpoint OTLP collector endpoint for OpenTelemetry tracing (e.g., 'localhost:4318'). When empty, tracing is disabled. - tracing.insecure false Disable TLS for the OTLP connection. - tracing.servicename Service name reported to the tracing backend. Defaults to 'nuts-node'. - **policy** - policy.directory ./config/policy Directory to read policy files from. Policy files are JSON files that contain a scope to PresentationDefinition mapping. - ======================================== =================================================================================================================================================================================================================================================================================================================================================================================================================================================================== ============================================================================================================================================================================================================================================================================================================================================ - -Options specific for ``did:nuts``/gRPC -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The following table contains additional (deprecated) options that are relevant for use cases that use ``did:nuts`` DIDs and/or the gRPC network. -If your use case does not use these features, you can ignore this table. - -.. table:: did:nuts/gRPC Server Options - :widths: 20 30 50 - :class: options-table - - ================================ =========================== ====================================================================================================================================================================================== - Key Default Description - ================================ =========================== ====================================================================================================================================================================================== - tls.certfile PEM file containing the certificate for the gRPC server (also used as client certificate). Required in strict mode. - tls.certheader Name of the HTTP header that will contain the client certificate when TLS is offloaded for gRPC. - tls.certkeyfile PEM file containing the private key of the gRPC server certificate. Required in strict mode. - tls.offload Whether to enable TLS offloading for incoming gRPC connections. Enable by setting it to 'incoming'. If enabled 'tls.certheader' must be configured as well. - tls.truststorefile ./config/ssl/truststore.pem PEM file containing the trusted CA certificates for authenticating remote gRPC servers. Required in strict mode. - **Auth** - auth.accesstokenlifespan 60 defines how long (in seconds) an access token is valid. Uses default in strict mode. - auth.clockskew 5000 allowed JWT Clock skew in milliseconds - auth.contractvalidators [irma,dummy,employeeid] sets the different contract validators to use - auth.irma.autoupdateschemas true set if you want automatically update the IRMA schemas every 60 minutes. - auth.irma.schememanager pbdf IRMA schemeManager to use for attributes. Can be either 'pbdf' or 'irma-demo'. - auth.irma.cors.origin [] sets the allowed CORS origins for the IRMA server - **Events** - events.nats.hostname 0.0.0.0 Hostname for the NATS server - events.nats.port 4222 Port where the NATS server listens on - events.nats.storagedir Directory where file-backed streams are stored in the NATS server - events.nats.timeout 30 Timeout for NATS server operations - **GoldenHammer** - goldenhammer.enabled true Whether to enable automatically fixing DID documents with the required endpoints. - goldenhammer.interval 10m0s The interval in which to check for DID documents to fix. - **Network** - network.bootstrapnodes [] List of bootstrap nodes (':') which the node initially connect to. - network.connectiontimeout 5000 Timeout before an outbound connection attempt times out (in milliseconds). - network.enablediscovery true Whether to enable automatic connecting to other nodes. - network.grpcaddr \:5555 Local address for gRPC to listen on. If empty the gRPC server won't be started and other nodes will not be able to connect to this node (outbound connections can still be made). - network.maxbackoff 24h0m0s Maximum between outbound connections attempts to unresponsive nodes (in Golang duration format, e.g. '1h', '30m'). - network.nodedid Specifies the DID of the party that operates this node. It is used to identify the node on the network. If the DID document does not exist of is deactivated, the node will not start. - network.protocols [] Specifies the list of network protocols to enable on the server. They are specified by version (1, 2). If not set, all protocols are enabled. - network.v2.diagnosticsinterval 5000 Interval (in milliseconds) that specifies how often the node should broadcast its diagnostic information to other nodes (specify 0 to disable). - network.v2.gossipinterval 5000 Interval (in milliseconds) that specifies how often the node should gossip its new hashes to other nodes. - **Storage** - storage.bbolt.locktimeout 1s Maximum time to wait for acquiring a lock on the BBolt database before giving up and returning an error. Formatted as Golang duration (e.g. 1s, 1m). - storage.bbolt.backup.directory Target directory for BBolt database backups. - storage.bbolt.backup.interval 0s Interval, formatted as Golang duration (e.g. 10m, 1h) at which BBolt database backups will be performed. - storage.redis.address Redis database server address. This can be a simple 'host:port' or a Redis connection URL with scheme, auth and other options. - storage.redis.database Redis database name, which is used as prefix every key. Can be used to have multiple instances use the same Redis instance. - storage.redis.password Redis database password. If set, it overrides the username in the connection URL. - storage.redis.username Redis database username. If set, it overrides the username in the connection URL. - storage.redis.sentinel.master Name of the Redis Sentinel master. Setting this property enables Redis Sentinel. - storage.redis.sentinel.nodes [] Addresses of the Redis Sentinels to connect to initially. Setting this property enables Redis Sentinel. - storage.redis.sentinel.password Password for authenticating to Redis Sentinels. - storage.redis.sentinel.username Username for authenticating to Redis Sentinels. - storage.redis.tls.truststorefile PEM file containing the trusted CA certificate(s) for authenticating remote Redis servers. Can only be used when connecting over TLS (use 'rediss://' as scheme in address). - **VCR** - vcr.openid4vci.definitionsdir Directory with the additional credential definitions the node could issue (experimental, may change without notice). - vcr.openid4vci.enabled true Enable issuing and receiving credentials over OpenID4VCI. - vcr.openid4vci.timeout 30s Time-out for OpenID4VCI HTTP client operations. - ================================ =========================== ====================================================================================================================================================================================== - -This table is automatically generated using the configuration flags in the core and engines. When they're changed -the options table must be regenerated using the Makefile: - -.. code-block:: shell - - $ make docs + :header-rows: 1 + + * - Key + - Default + - Description + * - configfile + - ./config/nuts.yaml + - Nuts config file + * - cpuprofile + - + - When set, a CPU profile is written to the given path. Ignored when strictmode is set. + * - datadir + - ./data + - Directory where the node stores its files. + * - didmethods + - [web,nuts] + - Comma-separated list of enabled DID methods (without did: prefix). It also controls the order in which DIDs are returned by APIs, and which DID is used for signing if the verifying party does not impose restrictions on the DID method used. + * - internalratelimiter + - true + - When set, expensive internal calls are rate-limited to protect the network. Always enabled in strict mode. + * - loggerformat + - text + - Log format (text, json) + * - strictmode + - true + - When set, insecure settings are forbidden. + * - url + - + - Public facing URL of the server (required). Must be HTTPS when strictmode is set. It's baked into every DID and OAuth identity the node issues, so choose a domain you own, that is stable (avoid TLDs that block re-registration after expiry, and cloud-provider subdomains that don't identify the owner), and that serves security.txt and robots.txt at its root. + * - verbosity + - info + - Log level (trace, debug, info, warn, error) + * - httpclient.timeout + - 30s + - Request time-out for HTTP clients, such as '10s'. Refer to Golang's 'time.Duration' syntax for a more elaborate description of the syntax. + * - **Auth** + - + - + * - auth.authorizationendpoint.enabled + - false + - enables the v2 API's OAuth2 Authorization Endpoint, used by OpenID4VP and OpenID4VCI. This flag might be removed in a future version (or its default become 'true') as the use cases and implementation of OpenID4VP and OpenID4VCI mature. + * - auth.experimental.jwtbearerclient + - false + - enables the experimental RFC 7523 jwt-bearer two-VP token request flow. While disabled (the default), requests carrying a service-provider subject identifier are rejected. Subject to change without notice. + * - **Crypto** + - + - + * - crypto.storage + - + - Storage to use, 'fs' for file system (for development purposes), 'vaultkv' for HashiCorp Vault KV store, 'azure-keyvault' for Azure Key Vault, 'external' for an external backend (deprecated). + * - crypto.azurekv.hsm + - false + - Whether to store the key in a hardware security module (HSM). If true, the Azure Key Vault must be configured for HSM usage. Default: false + * - crypto.azurekv.timeout + - 10s + - Timeout of client calls to Azure Key Vault, in Golang time.Duration string format (e.g. 10s). + * - crypto.azurekv.url + - + - The URL of the Azure Key Vault. + * - crypto.azurekv.auth.type + - default + - Credential type to use when authenticating to the Azure Key Vault. Options: default, managed_identity (see https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/azidentity/README.md for an explanation of the options). + * - crypto.vault.address + - + - The Vault address. If set it overwrites the VAULT_ADDR env var. + * - crypto.vault.pathprefix + - kv + - The Vault path prefix. + * - crypto.vault.timeout + - 5s + - Timeout of client calls to Vault, in Golang time.Duration string format (e.g. 1s). + * - crypto.vault.token + - + - The Vault token. If set it overwrites the VAULT_TOKEN env var. + * - **Discovery** + - + - + * - discovery.client.refreshinterval + - 10m0s + - Interval at which the client synchronizes with the Discovery Server; refreshing Verifiable Presentations of local DIDs and loading changes, updating the local copy. It only will actually refresh registrations of local DIDs that about to expire (less than 1/4th of their lifetime left). Specified as Golang duration (e.g. 1m, 1h30m). + * - discovery.definitions.directory + - ./config/discovery + - Directory to load Discovery Service Definitions from. If not set, the discovery service will be disabled. If the directory contains JSON files that can't be parsed as service definition, the node will fail to start. + * - discovery.server.ids + - [] + - IDs of the Discovery Service for which to act as server. If an ID does not map to a loaded service definition, the node will fail to start. + * - **HTTP** + - + - + * - http.clientipheader + - X-Forwarded-For + - Case-sensitive HTTP Header that contains the client IP used for audit logs. For the X-Forwarded-For header only link-local, loopback, and private IPs are excluded. Switch to X-Real-IP or a custom header if you see your own proxy/infra in the logs. + * - http.log + - metadata + - What to log about HTTP requests. Options are 'nothing', 'metadata' (log request method, URI, IP and response code), and 'metadata-and-body' (log the request and response body, in addition to the metadata). In strictmode, 'metadata-and-body' is not allowed and is changed to 'metadata' at startup. + * - http.cache.maxbytes + - 10485760 + - HTTP client maximum size of the response cache in bytes. If 0, the HTTP client does not cache responses. + * - http.client.allowedinternalcidrs + - [] + - IP ranges (CIDR notation, e.g. 10.0.0.0/8) exempted from the strict-mode SSRF guard, which otherwise blocks outbound requests to non-public networks. Use to permit internal flows that legitimately target a private address, such as an internal credential offering or an internal OAuth user flow. Leave empty to block all non-public addresses. + * - http.client.deniedcidrs + - [] + - IP ranges (CIDR notation) that outbound HTTP requests must never target in strict mode, in addition to the built-in blocked ranges (non-public addresses and cloud metadata endpoints). Use for publicly routable ranges that are internal-only in your infrastructure. Takes precedence over http.client.allowedinternalcidrs. + * - http.internal.address + - 127.0.0.1:8081 + - Address and port the server will be listening to for internal-facing endpoints. + * - http.internal.auth.audience + - + - Expected audience for JWT tokens (default: hostname) + * - http.internal.auth.authorizedkeyspath + - + - Path to an authorized_keys file for trusted JWT signers + * - http.internal.auth.type + - + - Whether to enable authentication for /internal endpoints, specify 'token_v2' for bearer token mode or 'token' for legacy bearer token mode. + * - http.public.address + - \:8080 + - Address and port the server will be listening to for public-facing endpoints. + * - **JSONLD** + - + - + * - jsonld.contexts.localmapping + - [https://nuts.nl/credentials/2024=assets/contexts/nuts-2024.ldjson,https://nuts.nl/credentials/v1=assets/contexts/nuts.ldjson,https://schema.org=assets/contexts/schema-org-v13.ldjson,https://w3c-ccg.github.io/lds-jws2020/contexts/lds-jws2020-v1.json=assets/contexts/lds-jws2020-v1.ldjson,https://w3id.org/vc/status-list/2021/v1=assets/contexts/w3c-statuslist2021.ldjson,https://www.w3.org/2018/credentials/v1=assets/contexts/w3c-credentials-v1.ldjson] + - This setting allows mapping external URLs to local files for e.g. preventing external dependencies. These mappings have precedence over those in remoteallowlist. + * - jsonld.contexts.remoteallowlist + - [https://schema.org,https://www.w3.org/2018/credentials/v1,https://w3c-ccg.github.io/lds-jws2020/contexts/lds-jws2020-v1.json,https://w3id.org/vc/status-list/2021/v1] + - In strict mode, fetching external JSON-LD contexts is not allowed except for context-URLs listed here. + * - **PKI** + - + - + * - pki.maxupdatefailhours + - 4 + - Maximum number of hours that a denylist update can fail + * - pki.softfail + - true + - Do not reject certificates if their revocation status cannot be established when softfail is true + * - **Storage** + - + - + * - storage.debug + - false + - When true, enables extra logging of storage-layer problems (e.g. performance issues). + * - storage.session.memcached.address + - [] + - List of Memcached server addresses. These can be a simple 'host:port' or a Memcached connection URL with scheme, auth and other options. + * - storage.session.redis.address + - + - Redis session database server address. This can be a simple 'host:port' or a Redis connection URL with scheme, auth and other options. If not set it, defaults to an in-memory database. + * - storage.session.redis.database + - + - Redis session database name, which is used as prefix every key. Can be used to have multiple instances use the same Redis instance. + * - storage.session.redis.password + - + - Redis session database password. If set, it overrides the username in the connection URL. + * - storage.session.redis.username + - + - Redis session database username. If set, it overrides the username in the connection URL. + * - storage.session.redis.sentinel.master + - + - Name of the Redis Sentinel master. Setting this property enables Redis Sentinel. + * - storage.session.redis.sentinel.nodes + - [] + - Addresses of the Redis Sentinels to connect to initially. Setting this property enables Redis Sentinel. + * - storage.session.redis.sentinel.password + - + - Password for authenticating to Redis Sentinels. + * - storage.session.redis.sentinel.username + - + - Username for authenticating to Redis Sentinels. + * - storage.session.redis.tls.truststorefile + - + - PEM file containing the trusted CA certificate(s) for authenticating remote Redis session servers. Can only be used when connecting over TLS (use 'rediss://' as scheme in address). + * - storage.sql.connection + - + - Connection string for the SQL database. If not set it, defaults to a SQLite database stored inside the configured data directory. Note: using SQLite is not recommended in production environments. If using SQLite anyways, remember to enable foreign keys ('_foreign_keys=on') and the write-ahead-log ('_journal_mode=WAL'). + * - storage.sql.rdsiam.dbuser + - + - Database username for IAM authentication. If not specified, the username from the connection string will be used. The database user must be created with IAM authentication enabled. + * - storage.sql.rdsiam.enabled + - false + - Enable AWS RDS IAM authentication for the SQL database connection. When enabled, the node will use temporary IAM tokens instead of passwords. Requires the connection string to be a PostgreSQL or MySQL RDS endpoint without a password. + * - storage.sql.rdsiam.region + - + - AWS region where the RDS instance is located (e.g., 'us-east-1). Required when RDS IAM authentication is enabled. + * - storage.sql.rdsiam.tokenrefreshinterval + - 14m0s + - Interval at which to refresh the IAM authentication token. RDS tokens are valid for 15 minutes, so set this to ensure tokens are refreshed before expiry. Specified as Golang duration (e.g. 10m, 1h). + * - **Tracing** + - + - + * - tracing.endpoint + - + - OTLP collector endpoint for OpenTelemetry tracing (e.g., 'localhost:4318'). When empty, tracing is disabled. + * - tracing.insecure + - false + - Disable TLS for the OTLP connection. + * - tracing.servicename + - + - Service name reported to the tracing backend. Defaults to 'nuts-node'. + * - **policy** + - + - + * - policy.directory + - ./config/policy + - Directory to read policy files from. Policy files are JSON files that contain a scope to PresentationDefinition mapping. + * - policy.authzen.endpoint + - + - Base URL of the AuthZen PDP endpoint. Required when any credential profile uses scope_policy 'dynamic'; the node refuses to start if such a profile is configured but this flag is empty. + +If your use case still uses ``did:nuts`` DIDs and/or the gRPC network, there's an additional (deprecated) options +table in :ref:`Legacy did:nuts configuration `. + +See :ref:`The node's URL ` for guidance on choosing the ``url`` option. Secrets ******* @@ -339,7 +340,10 @@ As a general safety precaution ``auth.contractvalidators`` ignores the ``dummy`` requesting an access token from another node on ``/n2n/auth/v1/accesstoken`` does not return any error details, ``auth.accesstokenlifespan`` is always 60 seconds, json-ld context can only be downloaded from trusted domains configured in ``jsonld.contexts.remoteallowlist``, +``http.log=metadata-and-body`` is not allowed and is changed to ``metadata`` at startup (request and response bodies on the OAuth endpoints contain credentials, which must not be written to logs), and the ``internalratelimiter`` is always on. Interacting with remote Nuts nodes requires HTTPS: it will refuse to connect to plain HTTP endpoints when in strict mode. +Strict mode additionally rejects outbound URLs whose host is an RFC 2606 reserved hostname/TLD (e.g. ``*.localhost``, ``*.test``, ``example.com/net/org``); non-public IP addresses are refused too, unless explicitly permitted via ``http.client.allowedinternalcidrs``. +This applies to every outbound HTTP call made via the shared HTTP client (OpenID4VCI, OAuth relying-party, IAM, Discovery, did:web resolution, etc.) and to every redirect target along the way. diff --git a/README_template.rst b/README_template.rst index c23b372dbd..eca61cc205 100644 --- a/README_template.rst +++ b/README_template.rst @@ -46,11 +46,10 @@ Claude Code skill files for common maintenance tasks can be found in the `.claud Development ^^^^^^^^^^^ -.. include:: docs/pages/development/1-development.rst - :start-after: .. marker-for-readme +See `DEVELOPMENT.md `_ for build, test and code generation instructions. Configuration ^^^^^^^^^^^^^ -.. include:: docs/pages/deployment/configuration.rst +.. include:: docs/pages/configuration/options.rst :start-after: .. marker-for-readme diff --git a/RELEASING.md b/RELEASING.md new file mode 100644 index 0000000000..f2fcd9d410 --- /dev/null +++ b/RELEASING.md @@ -0,0 +1,85 @@ +# Releasing Nuts Node + +## Semantic versioning + +Nuts Node and auxiliary tools/applications follow a semantic versioning scheme (`..(-rc.)`): + +> Given a version number MAJOR.MINOR.PATCH, increment the: +> +> 1. MAJOR version when you make incompatible API changes, +> 2. MINOR version when you add functionality in a backwards compatible manner, or +> 3. PATCH version when you make backwards compatible bug fixes. +> +> (Taken from [semver.org](https://semver.org/)) + +Note: "API" is a broad term, it covers every interface interacted with by applications or other nodes (including Nuts network protocols). + +When a new minor or major version is released, always create a release candidate first: `..0-rc.1`, e.g.: `v5.1.0-rc.1`. +This version will symbolize a feature freeze and will be used for the first tests. +All problems will be fixed and the release candidate version is increased on every bugfix release, e.g.: `v5.1.0-rc.2`. +When no more problems are found the major/minor version is released without a `-rc.` postfix. +This approach prevents the docker `latest` tag from being updated to a new version automatically. + +Aside from the Nuts Node itself, all projects that follow the same versions need to be released. +They follow the major version from the Nuts Node, but minor and patch versions may differ. + +## Major release + +A major release starts with version number `.0.0`. Every Nuts Node release has a name (e.g. "Brazil") and a version number. +A release consists of a Git tag and a release in Github with release notes. Releases are created according to the following format: + +- Git tag: `v..`, e.g. `v2.0.0` +- Release name: ` release ()`, e.g.: `Brazil release (v2.0.0)` (every release has a designated name) +- Release notes: auto-generated by Github. + +## Bugfix release/patches + +When an issue is fixed in a released version a bugfix/patch version must be released. +The bug must be fixed on a branch named after the major version, e.g. `v1` or `v2`. +The release name follows the release name, but is named "bugfix" instead of "release". E.g.: `Brazil bugfix (v2.0.1)`. + +### Backports + +Bugfixes often need to be backported, e.g. it's fixed on the `master` branch but also needs to be fixed in the last version, +and maybe even in the version before that. Bugfix releases stemming from backports follow the same versioning and naming scheme as regular bugfix releases. + +## Building a release + +Make sure all changes are on the relevant branch. +If it's for an older version (backport), cherry-pick all changes that need to be included and merge them into the correct branch (e.g., V5.4, V6.1). +Make sure to add the release notes to the branch *before* tagging a release or it will not be visible on read-the-docs. +For good measure, also run `make cli-docs` (requires the `rst_include` python package to be installed) to make sure we didn't forget to update the documentation. + +Go to [releases on github](https://github.com/nuts-foundation/nuts-node/releases) and perform the following steps: + +1. `Draft new release` +2. set the target branch +3. `Choose a tag` and create a new one according to the git tag convention above (e.g. v6.1.0) +4. `Generate release notes` +5. (optional) curate release text. For major/minor versions probably replace with the release notes written for read the docs. +6. Set the `Set as latest release` checkbox as needed +7. `Publish release` + +This will trigger github actions that publish a new release to [Docker Hub](https://hub.docker.com/r/nutsfoundation/nuts-node/tags), and a message will be posted in the `#releases` channel on Slack. + +### Major/Minor version updates + +**Make a new branch** + +Every new major or minor version has its own branch. +After creating a new release, a new branch should be made based off of the git tag for the release. +The naming convention for branches is `V.`, e.g. V6.0 or V5.4. (Yes, git/github tag/version uses lowercase `v`, branches use uppercase `V`, and Docker tags omit the prefix entirely since version 6.0) +Add branch protection to the new branch on Github. + +**Read the docs** + +Go to [app.readthedocs.org/projects/nuts-node](https://app.readthedocs.org/projects/nuts-node/) and click on `+ Add version` to add the new branch to the available documentation versions on [nuts-node.readthedocs.io](https://nuts-node.readthedocs.io/). + +**Automated tests** + +Testing is automated using Github workflows. +Some of the tests cannot handle branch patterns and require updating relevant major/minor version branches to the workflow file manually. +The current list of files that need to be updated are: + +- **Scheduled govulncheck** action: `.github/workflows/govulncheck-cron-schedule.yaml`. Runs every day and sends vulnerability warnings to the `#nuts-core-team` slack channel. +- **Scheduled CodeQL** action: `.github/workflows/codeql-analysis-cron-schedule.yml` diff --git a/core/server_config.go b/core/server_config.go index 434300e72b..d2f5ad6c22 100644 --- a/core/server_config.go +++ b/core/server_config.go @@ -265,7 +265,9 @@ func FlagSet() *pflag.FlagSet { flagSet.Bool("strictmode", defaultCfg.Strictmode, "When set, insecure settings are forbidden.") flagSet.Bool("internalratelimiter", defaultCfg.InternalRateLimiter, "When set, expensive internal calls are rate-limited to protect the network. Always enabled in strict mode.") flagSet.String("datadir", defaultCfg.Datadir, "Directory where the node stores its files.") - flagSet.String("url", defaultCfg.URL, "Public facing URL of the server (required). Must be HTTPS when strictmode is set.") + flagSet.String("url", defaultCfg.URL, "Public facing URL of the server (required). Must be HTTPS when strictmode is set. "+ + "It's baked into every DID and OAuth identity the node issues, so choose a domain you own, that is stable (avoid TLDs that block re-registration after expiry, "+ + "and cloud-provider subdomains that don't identify the owner), and that serves security.txt and robots.txt at its root.") flagSet.StringSlice("didmethods", defaultCfg.DIDMethods, "Comma-separated list of enabled DID methods (without did: prefix). "+ "It also controls the order in which DIDs are returned by APIs, and which DID is used for signing if the verifying party does not impose restrictions on the DID method used.") flagSet.Duration("httpclient.timeout", defaultCfg.HTTPClient.Timeout, "Request time-out for HTTP clients, such as '10s'. Refer to Golang's 'time.Duration' syntax for a more elaborate description of the syntax.") diff --git a/docs/README.md b/docs/README.md index 4de6b16962..c97555f966 100644 --- a/docs/README.md +++ b/docs/README.md @@ -8,3 +8,59 @@ To build the documentation locally, run `make cli-docs` in the parent directory docker build -t nuts-node-docs . docker run --rm -v $PWD:/docs nuts-node-docs ``` + +## API development guidelines + +When developing APIs, please follow these guidelines. + +### Contract first + +The Nuts node APIs are specified in [Open API Specification (OAS)](https://swagger.io/specification/). +The files are located under `/docs/_static//.yaml`, where `` is a specific module like `crypto` or `auth`, and `` defines the version of the API. +We use version `3.0.y` of the OAS. + +#### Versioning + +We use versioning of the APIs. This is reflected in both the OAS files and the HTTP paths. +Versions must follow the pattern `v` and start at `v1`. These are major versions: any breaking change results in a new major version of the API. +New additions, bug fixes and changes that are backwards compatible may be done in the current version. + +#### Code generation + +The OAS files are used for code generation. The makefile contains the `gen-api` target which generates the code. +The build target only needs to be extended when a new version or new engine is added. +Generated code is always placed in `//api//generated.go`. + +#### Return codes + +The error return values are generalized for all API calls. The return values follow [RFC7807](https://tools.ietf.org/html/rfc7807). +The definition is available under `/docs/_static/common/error_response.yaml`. The error definition can be used in an OAS file: + +```yaml +paths: + /some/path: + get: + responses: + default: + $ref: '../common/error_response.yaml' +``` + +The error responses will not be listed as responses in the online generated documentation. +To describe error responses, the specific responses need to be added to the API description: + +```yaml +paths: + /some/path: + post: + description: | + Some description on the API + + error returns: + * 400 - incorrect input +``` + +### Paths + +The API paths are designed so it's clear which APIs are to be blocked for external traffic. + +- `/internal/**` These APIs are meant to be behind a firewall and should only be available to the internal infrastructure. diff --git a/docs/_static/didman/v1.yaml b/docs/_static/didman/v1.yaml index 94a29df7a9..c30b8d17de 100644 --- a/docs/_static/didman/v1.yaml +++ b/docs/_static/didman/v1.yaml @@ -489,7 +489,7 @@ components: type: object description: > An object describing the found entity, modelled as a concept as specified by VCR's OpenAPI specification. - See https://nuts-node.readthedocs.io/en/latest/pages/development/3-vc.html for examples on which concepts are supported and how they're structured. + See https://nuts-node.readthedocs.io/en/latest/pages/api/verifiable-credentials.html for examples on which concepts are supported and how they're structured. didDocument: $ref: '../common/ssi_types.yaml#/components/schemas/DIDDocument' EndpointResponse: diff --git a/docs/_static/vdr/v1.yaml b/docs/_static/vdr/v1.yaml index 5808898ed4..a25745b323 100644 --- a/docs/_static/vdr/v1.yaml +++ b/docs/_static/vdr/v1.yaml @@ -1,7 +1,12 @@ openapi: "3.0.0" info: title: Nuts Verifiable Data Registry API spec - description: API specification for the Verifiable Data Registry + description: | + API specification for the Verifiable Data Registry. + + Deprecated: this is the v1 API, which only supports `did:nuts`. Use the v2 API for new integrations. + Do not mix v1 and v2 API usage on the same subject: they read from different stores, and switching from v2 back to v1 is not supported. + See the migration guide (Operations > Migration) for details. version: 1.0.0 license: name: GPLv3 diff --git a/docs/generate_docs.go b/docs/generate_docs.go index efb149ec88..fc16cfbb96 100644 --- a/docs/generate_docs.go +++ b/docs/generate_docs.go @@ -48,7 +48,7 @@ func (sl stringSlice) contains(s string) bool { } // serverCommands lists the commands that use the server config. The options server commands are only printed once, because the list is quite long. -var serverCommands stringSlice = []string{"nuts config", "nuts server", "nuts crypto fs2vault", "nuts crypto fs2external", "nuts http gen-token"} +var serverCommands stringSlice = []string{"nuts config", "nuts server", "nuts crypto fs2external", "nuts http gen-token"} func generateDocs() { system := cmd.CreateSystem(func() {}) @@ -63,7 +63,7 @@ func generateDocs() { // output path. generateDocs writes them to disk; the up-to-date test compares them against the committed files. func generatedDocFiles(system *core.System) map[string][]byte { files := renderServerOptions(system) - files["docs/pages/deployment/cli-reference.rst"] = renderCLICommands(system) + files["docs/pages/operations/cli-reference.rst"] = renderCLICommands(system) return files } @@ -148,8 +148,8 @@ func renderServerOptions(system *core.System) map[string][]byte { } return map[string][]byte{ - "docs/pages/deployment/server_options.rst": renderPartitionedConfigOptionsDocs("Server Options", filterFlags(flags, v5FlagsPredicates, true)), - "docs/pages/deployment/server_options_didnuts.rst": renderPartitionedConfigOptionsDocs("did:nuts/gRPC Server Options", filterFlags(flags, v5FlagsPredicates, false)), + "docs/pages/configuration/server_options.rst": renderPartitionedConfigOptionsDocs("Server Options", filterFlags(flags, v5FlagsPredicates, true)), + "docs/pages/configuration/server_options_didnuts.rst": renderPartitionedConfigOptionsDocs("did:nuts/gRPC Server Options", filterFlags(flags, v5FlagsPredicates, false)), } } diff --git a/docs/index.rst b/docs/index.rst index c6316da266..f25ac14e00 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -12,60 +12,56 @@ Nuts documentation .. toctree:: :maxdepth: 1 - :caption: Integrating: - :glob: + :caption: Configuration: - pages/integrating/api.rst - pages/integrating/api-authentication.rst - pages/integrating/vc.rst - pages/integrating/supported-protocols-formats.rst - pages/integrating/version-incompatibilities.rst - pages/release_notes.rst + pages/configuration/options.rst + pages/configuration/url.rst + pages/configuration/usecase/index.rst + pages/configuration/legacy-did-nuts.rst .. toctree:: :maxdepth: 1 - :caption: Deployment & Configuration: - :glob: + :caption: Deployment: - pages/deployment/domain.rst - pages/deployment/configuration.rst - pages/deployment/migration.rst pages/deployment/recommended-deployment.rst pages/deployment/clustering.rst - pages/deployment/certificates.rst pages/deployment/docker.rst pages/deployment/storage.rst - pages/deployment/verifiable-credentials.rst - pages/deployment/logging.rst pages/deployment/monitoring.rst - pages/deployment/cli-reference.rst - pages/deployment/discovery.rst - pages/deployment/policy.rst - pages/deployment/key-rotation.rst - pages/deployment/audit-logging.rst - pages/deployment/oauth.rst + pages/deployment/logging.rst pages/deployment/security-considerations.rst + pages/deployment/saas.rst .. toctree:: :maxdepth: 1 - :caption: Technology: - :glob: - - pages/technology/** + :caption: API: + pages/api/reference.rst + pages/api/authentication.rst + pages/api/verifiable-credentials.rst .. toctree:: :maxdepth: 1 - :caption: Development: - :glob: + :caption: Operations: - pages/development/** - pages/contribute.rst + pages/operations/migration.rst + pages/operations/key-rotation.rst + pages/operations/certificate-revocation.rst + pages/operations/base-url-change.rst + pages/operations/cli-reference.rst +.. toctree:: + :maxdepth: 1 + :caption: Background: + + pages/background/did.rst + pages/background/jsonld.rst + pages/background/security_model.rst + pages/background/oauth.rst + pages/background/supported-protocols-formats.rst .. toctree:: :maxdepth: 1 - :caption: Other: - :glob: + :caption: Release Notes: - pages/contact.rst + pages/release_notes.rst diff --git a/docs/pages/integrating/api-authentication.rst b/docs/pages/api/authentication.rst similarity index 100% rename from docs/pages/integrating/api-authentication.rst rename to docs/pages/api/authentication.rst diff --git a/docs/pages/integrating/api.rst b/docs/pages/api/reference.rst similarity index 100% rename from docs/pages/integrating/api.rst rename to docs/pages/api/reference.rst diff --git a/docs/pages/integrating/vc.rst b/docs/pages/api/verifiable-credentials.rst similarity index 100% rename from docs/pages/integrating/vc.rst rename to docs/pages/api/verifiable-credentials.rst diff --git a/docs/pages/technology/did.rst b/docs/pages/background/did.rst similarity index 68% rename from docs/pages/technology/did.rst rename to docs/pages/background/did.rst index 6b32ff83d6..e078517699 100644 --- a/docs/pages/technology/did.rst +++ b/docs/pages/background/did.rst @@ -29,3 +29,17 @@ Services ======== The **services** section is used to list service endpoints. Although still available, the preferred way is to register services via :ref:`discovery`. + +.. _did-x509-background: + +did:x509 +******** + +In ``did:x509`` a certificate is converted to a DID Document (that includes its entire certificate chain) so it can be used in the Verifiable Credentials ecosystem. +This DID Method provides a temporary bridge between the 'old' world of CAs/Certificates and the 'new' Verifiable Credential world. +With other DID Methods, certificates are only used to create a secure channel for communication and optionally for client authentication. +In ``did:x509`` the certificates are also used in the cryptographic proofs to obtain access-tokens. +This means the certificate chain now provides the root of trust and has stricter requirements than connection certificates. + +Trust in specific certificate CAs is configured per use-case in a :ref:`Discovery ` and :ref:`Access policies ` definition file. +CRLs from the trusted chains (per those definition files) are consulted when evaluating ``did:x509`` Verifiable Credentials; see :ref:`Certificate revocation (CRL) handling ` for how a failure to download a CRL is handled — for ``did:x509`` this always hard-fails. diff --git a/docs/pages/technology/jsonld.rst b/docs/pages/background/jsonld.rst similarity index 100% rename from docs/pages/technology/jsonld.rst rename to docs/pages/background/jsonld.rst diff --git a/docs/pages/deployment/oauth.rst b/docs/pages/background/oauth.rst similarity index 100% rename from docs/pages/deployment/oauth.rst rename to docs/pages/background/oauth.rst diff --git a/docs/pages/technology/security_model.rst b/docs/pages/background/security_model.rst similarity index 100% rename from docs/pages/technology/security_model.rst rename to docs/pages/background/security_model.rst diff --git a/docs/pages/integrating/supported-protocols-formats.rst b/docs/pages/background/supported-protocols-formats.rst similarity index 100% rename from docs/pages/integrating/supported-protocols-formats.rst rename to docs/pages/background/supported-protocols-formats.rst diff --git a/docs/pages/configuration/legacy-did-nuts.rst b/docs/pages/configuration/legacy-did-nuts.rst new file mode 100644 index 0000000000..7df4e866ad --- /dev/null +++ b/docs/pages/configuration/legacy-did-nuts.rst @@ -0,0 +1,38 @@ +.. _legacy-did-nuts-configuration: + +Legacy did:nuts configuration +############################## + +This page covers configuration that's only relevant for use cases that still use ``did:nuts`` DIDs and/or the Nuts gRPC network. +If your use case does not use these features, you can ignore this page. + +Options +******* + +The following table contains additional (deprecated) options that are relevant for use cases that use ``did:nuts`` DIDs and/or the gRPC network: + +.. include:: server_options_didnuts.rst + +Client authentication +********************** + +The ``/n2n`` endpoints and the ``gRPC Nuts network`` use TLS certificates for client authentication. +The Nuts-node validates the client certificates used by its peers on the ``gRPC network`` when a new connection is established, and periodically after that as long as the connection exists. +To do this, all trusted certificate chains must be configured in ``tls.truststorefile``. +See :ref:`Certificate revocation handling ` for how CRL checks and soft-fail behavior work for these certificates. +The ``gRPC Nuts network`` and ``/n2n`` endpoints are deprecated and will be removed in the future. + +Publishing services for use cases +********************************** + +V5 use-cases define service endpoints or a collection of endpoints that should be registered in the Services on DID Documents. +The concrete endpoints are usually on the DID Document of the vendor, and then referenced by all DID Documents managed by that vendor. +``did:nuts`` for example, requires the registration of a ``NutsComm`` endpoint to authenticate the connection. +Use-cases built on ``did:nuts`` should keep using the DIDMan API to manage and resolve Services on DID Documents. +Any Service change made using the DIDMan API will only update ``did:nuts`` DID Documents. + +For use-cases built on v6 and later, any endpoint needed for the use-case should instead be listed in the registration on the :ref:`Discovery Service ` for that use-case. +This means that ``did:web`` DID Documents (or non-did:nuts if we look further ahead) will contain very few Services, if any. +If there is a need to add a Service for these use-cases, they should be added using the VDR v2 API, which will then add the Service to _all_ DIDs that are part of the Subject. +Note that resolving Services using the VDR v2 API will return the Service from the document as is. +So, it resolves Services without following any references in the Service to a concrete endpoint, as is done by DIDMan. diff --git a/docs/pages/deployment/configuration.rst b/docs/pages/configuration/options.rst similarity index 93% rename from docs/pages/deployment/configuration.rst rename to docs/pages/configuration/options.rst index dac3951916..99a613f188 100644 --- a/docs/pages/deployment/configuration.rst +++ b/docs/pages/configuration/options.rst @@ -1,7 +1,7 @@ .. _nuts-node-config: -Configuration -############# +Options +####### .. marker-for-readme @@ -50,20 +50,10 @@ The following options can be configured on the server: .. include:: server_options.rst -Options specific for ``did:nuts``/gRPC -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +If your use case still uses ``did:nuts`` DIDs and/or the gRPC network, there's an additional (deprecated) options +table in :ref:`Legacy did:nuts configuration `. -The following table contains additional (deprecated) options that are relevant for use cases that use ``did:nuts`` DIDs and/or the gRPC network. -If your use case does not use these features, you can ignore this table. - -.. include:: server_options_didnuts.rst - -This table is automatically generated using the configuration flags in the core and engines. When they're changed -the options table must be regenerated using the Makefile: - -.. code-block:: shell - - $ make docs +See :ref:`The node's URL ` for guidance on choosing the ``url`` option. Secrets ******* diff --git a/docs/pages/deployment/server_options.rst b/docs/pages/configuration/server_options.rst similarity index 97% rename from docs/pages/deployment/server_options.rst rename to docs/pages/configuration/server_options.rst index 9d297c3503..5689e26f2f 100755 --- a/docs/pages/deployment/server_options.rst +++ b/docs/pages/configuration/server_options.rst @@ -29,7 +29,7 @@ - When set, insecure settings are forbidden. * - url - - - Public facing URL of the server (required). Must be HTTPS when strictmode is set. + - Public facing URL of the server (required). Must be HTTPS when strictmode is set. It's baked into every DID and OAuth identity the node issues, so choose a domain you own, that is stable (avoid TLDs that block re-registration after expiry, and cloud-provider subdomains that don't identify the owner), and that serves security.txt and robots.txt at its root. * - verbosity - info - Log level (trace, debug, info, warn, error) diff --git a/docs/pages/deployment/server_options_didnuts.rst b/docs/pages/configuration/server_options_didnuts.rst similarity index 100% rename from docs/pages/deployment/server_options_didnuts.rst rename to docs/pages/configuration/server_options_didnuts.rst diff --git a/docs/pages/configuration/url.rst b/docs/pages/configuration/url.rst new file mode 100644 index 0000000000..e8425b94cf --- /dev/null +++ b/docs/pages/configuration/url.rst @@ -0,0 +1,35 @@ +.. _nuts-node-url: + +The node's URL +############### + +The ``url`` option is the node's public facing URL: + +.. code-block:: yaml + + url: https://example.com + +It's used in the following ways: + +- It provides some information about the owner of DIDs and is part of the client_id in OAuth flows. + Other parties can use it to identify your node. +- It's part of the DIDs the node creates for you when you create a new subject. + For example: DID web URLs are constructed as ``did:web::iam:``. +- It's listed in OAuth metadata. + For example: the default identity URL is ``https:///oauth2/``. + This URL is then used to lookup .well-known endpoints. +- It's part of the URL for the StatusList2021 revocation mechanism. + +There are no strict requirements for it, but please consider the following: + +- You must own the domain. +- The domain should be stable. It should not change. +- It should use a TLD that allows for retention of the domain name. + For example, a .com domain name can be blocked for a period of time after it's no longer registered. + This will prevent the next owner from using it. +- The domain should be human readable. + Sub-domains from cloud providers are not recommended since they don't provide information about the owner. +- There should be a security.txt and robots.txt file at the root of the domain. + This is a best practice for security and privacy. + +Once chosen, changing it is a disruptive operation; see :ref:`changing-base-url` for the procedure. diff --git a/docs/pages/deployment/policy.rst b/docs/pages/configuration/usecase/access-policies.rst similarity index 100% rename from docs/pages/deployment/policy.rst rename to docs/pages/configuration/usecase/access-policies.rst diff --git a/docs/pages/deployment/discovery.rst b/docs/pages/configuration/usecase/discovery.rst similarity index 100% rename from docs/pages/deployment/discovery.rst rename to docs/pages/configuration/usecase/discovery.rst diff --git a/docs/pages/configuration/usecase/index.rst b/docs/pages/configuration/usecase/index.rst new file mode 100644 index 0000000000..f79fc83086 --- /dev/null +++ b/docs/pages/configuration/usecase/index.rst @@ -0,0 +1,14 @@ +.. _usecase-configuration: + +Use case configuration +######################## + +Beyond the server options, most use cases require additional configuration: which credentials are required for a scope, +which parties are discoverable, and how custom credential types are recognized. This section covers those per-use-case settings. + +.. toctree:: + :maxdepth: 1 + + access-policies.rst + verifiable-credentials.rst + discovery.rst diff --git a/docs/pages/deployment/verifiable-credentials.rst b/docs/pages/configuration/usecase/verifiable-credentials.rst similarity index 100% rename from docs/pages/deployment/verifiable-credentials.rst rename to docs/pages/configuration/usecase/verifiable-credentials.rst diff --git a/docs/pages/contact.rst b/docs/pages/contact.rst deleted file mode 100644 index 7c37b932f2..0000000000 --- a/docs/pages/contact.rst +++ /dev/null @@ -1,14 +0,0 @@ -####### -Contact -####### - -*********** -Information -*********** - -More information about the Nuts foundation can be found at `nuts.nl `_ - -************* -Communication -************* -The main means of communication is via `Slack `_. \ No newline at end of file diff --git a/docs/pages/contribute.rst b/docs/pages/contribute.rst deleted file mode 100644 index 3330d5d582..0000000000 --- a/docs/pages/contribute.rst +++ /dev/null @@ -1,5 +0,0 @@ -########## -Contribute -########## - -See `COLLABORATORS.md `_ for how to contribute to ``nuts-node``. diff --git a/docs/pages/deployment/audit-logging.rst b/docs/pages/deployment/audit-logging.rst deleted file mode 100644 index 66caf6c3ca..0000000000 --- a/docs/pages/deployment/audit-logging.rst +++ /dev/null @@ -1,18 +0,0 @@ -.. _audit-logging: - -Audit Logging -############# - -.. note :: - This feature is under development, not all relevant operations might be included in the audit log. - -Important events are logged as audit events. Examples are creation of a new cryptographic key pair or its usage when signing or decrypting. - -Audit events are logged to application log and can be recognized by the ``audit`` log level. -In addition, the events contain the following fields: - -- ``operation`` which contains name action that was performed -- ``actor`` which contains the name of the user/system that performed the action. - -To redirect the audit log to safe storage, it is advised to use a log processor (e.g. `Fluentbit `_). -You can use the ``audit`` log level to detect audit logs and redirect it to a separate log collector. \ No newline at end of file diff --git a/docs/pages/deployment/certificates.rst b/docs/pages/deployment/certificates.rst deleted file mode 100644 index 048d7a10f6..0000000000 --- a/docs/pages/deployment/certificates.rst +++ /dev/null @@ -1,29 +0,0 @@ -.. _certificates: - -Certificates -############ - -client authentication -********************* - -Nuts-node versions before v6 only use TLS certificates for client authentication on the ``/n2n`` endpoints and in the ``gRPC Nuts network``. -The Nuts-node also validates the client certificates used by its peers on the ``gRPC network`` when a new connection is established, and periodically after that as long as the connection exists. -To do this, all trusted certificate chains must be configured in ``tls.truststorefile``. -The Certificate Revocation List (CRL) of the CAs in the truststore are periodically downloaded to confirm a peer's client certificate is not revoked. -To prevent a CA with downtime on its CRL endpoint from bringing down the network, the Nuts-node uses a soft-fail strategy that does not reject certificates if it cannot download the CRL. -This behavior can be changed to hard-fail (fail if certificate is invalid, expired, of revoked, or if any of the previous cannot be determined) using the ``pki.softfail`` config flag. -The ``gRPC Nuts network`` and ``/n2n`` endpoints are deprecated and will be removed in the future. - -did:x509 -******** - -In ``did:x509`` a certificate is converted to a DID Document (that includes its entire certificate chain) so it can be used in the Verifiable Credentials ecosystem. -This DID Method provides a temporary bridge between the 'old' world of CAs/Certificates and the 'new' Verifiable Credential world. -With other DID Methods, certificates are only used to create an secure channel for communication and optionally for client authentication. -In ``did:x509`` the certificates are also used in the cryptographic proofs to obtain access-tokens. -This means the certificate chain now provides the root of trust and has stricter requirements than connection certificates. - -Trust in specific certificate CAs is configured per use-case in a :ref:`Discovery ` and :ref:`Policy ` definition file. -CRLs from trusted chains (per the above definition files) are consulted when evaluating ``did:x509`` Verifiable Credentials. -For certificate chains used in ``did:x509`` the Nuts-node always uses a hard-fail strategy, i.e., the ``pki.softfail`` config value is ignored during certificate validation for ``did:x509``. -This means that the Nuts-node will not be able to verify a ``did:x509`` DID or Verifiable Credential signed by this DID Method if the CRL cannot be downloaded and the CRL in the cache is older than ``pki.maxupdatefailhours``. diff --git a/docs/pages/deployment/clustering.rst b/docs/pages/deployment/clustering.rst index e7520a28fa..efe094f7f2 100644 --- a/docs/pages/deployment/clustering.rst +++ b/docs/pages/deployment/clustering.rst @@ -3,7 +3,7 @@ Clustering ########## -Clustering is currently limited to nodes that have the ``did:nuts`` method disabled. +Clustering is currently limited to nodes that have the ``did:nuts`` method disabled (see :ref:`Legacy did:nuts configuration `). To enable clustering, you must support the following: - A clustered SQL database (SQLite is not supported) diff --git a/docs/pages/deployment/domain.rst b/docs/pages/deployment/domain.rst deleted file mode 100644 index cf7aa90b53..0000000000 --- a/docs/pages/deployment/domain.rst +++ /dev/null @@ -1,55 +0,0 @@ -.. _domain: - -Choosing a domain name -###################### - -When deploying a Nuts node, you need to choose a domain name for the node. - -.. code-block:: yaml - - url: https://example.com - -The domain name is used in the following ways: - -- It provides some information about the owner of DIDs and is part of the client_id in OAuth flows. - Other parties can use the domain name to identify your node. -- It's part of the DIDs the node creates for you when you create a new subject. - For example: DID web URLs are constructed as ``did:web::iam:``. -- It's listed in OAuth metadata. - For example: the default identity URL is ``https:///oauth2/``. - This URL is then used to lookup .well-known endpoints. -- It's part of the URL for the StatusList2021 revocation mechanism. - -There are no strict requirements for the domain name, but please consider the following: - -- You must own the domain name. -- The domain name should be stable. - It should not change. -- It should use a TLD that allows for retention of the domain name. - For example, a .com domain name can be blocked for a period of time after it's no longer registered. - This will prevent the next owner from using it. -- The domain name should be human readable. - Sub-domains from cloud providers are not recommended since they don't provide information about the owner. -- There should be a security.txt and robots.txt file at the root of the domain. - This is a best practice for security and privacy. - -Changing the domain name -************************ - -Changing the domain name of a Nuts node is basically the same as setting up a new node on a new domain. -All DIDs, VerifiableCredentials, Revocations, and other data is no longer usable. -You will not be able to revoke credentials. This means that all credentials issued by the old domain can no longer be trusted. -Actions should be taken to remove any trust that might have been established in your old identity. - -As an issuer, you'll have to run the new domain side-by-side with the old domain for some time. - -.. note:: - - How long you should run the old domain side-by-side with the new domain depends on the use case and the validity of the credentials. - As a rule of thumb, credentials should be renewed once a year, so you should run the old domain side-by-side with the new domain for at least a year. - -As a holder, you'll have to generate DIDs for all your tenants and have issuers re-issue credentials. -This might involve a lot of manual work from your tenants. - -As a verifier, nothing changes. - diff --git a/docs/pages/deployment/logging.rst b/docs/pages/deployment/logging.rst index 28b823990a..56b79aae88 100644 --- a/docs/pages/deployment/logging.rst +++ b/docs/pages/deployment/logging.rst @@ -28,4 +28,21 @@ The following fields are always available: - ``time``: the timestamp of the log message, e.g. ``2023-01-14T07:34:16+01:00`` - ``msg``: the actual log message -Operations that manipulate or use private keys directly will generate a log with ``level`` set to ``audit``. \ No newline at end of file +Operations that manipulate or use private keys directly will generate a log with ``level`` set to ``audit``. + +Audit logging +************* + +.. note :: + This feature is under development, not all relevant operations might be included in the audit log. + +Important events are logged as audit events. Examples are creation of a new cryptographic key pair or its usage when signing or decrypting. + +Audit events are logged to the application log and can be recognized by the ``audit`` log level. +In addition, the events contain the following fields: + +- ``operation`` which contains the name of the action that was performed +- ``actor`` which contains the name of the user/system that performed the action. + +To redirect the audit log to safe storage, it is advised to use a log processor (e.g. `Fluentbit `_). +You can use the ``audit`` log level to detect audit logs and redirect them to a separate log collector. \ No newline at end of file diff --git a/docs/pages/technology/saas.rst b/docs/pages/deployment/saas.rst similarity index 100% rename from docs/pages/technology/saas.rst rename to docs/pages/deployment/saas.rst diff --git a/docs/pages/deployment/storage.rst b/docs/pages/deployment/storage.rst index 0d3e4c572c..3cbb74fe9e 100644 --- a/docs/pages/deployment/storage.rst +++ b/docs/pages/deployment/storage.rst @@ -189,14 +189,7 @@ Migrating to Hashicorp Vault ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Migrating your private keys from the filesystem to Vault is relatively easy: just upload the keys to Vault under ``kv/nuts-private-keys``. - -Alternatively you can use the ``fs2vault`` crypto command, which takes the directory containing the private keys as argument (the example assumes the container is called *nuts-node* and *NUTS_DATADIR=/opt/nuts/data*): - -.. code-block:: shell - - docker exec nuts-node nuts crypto fs2vault /opt/nuts/data/crypto - -In any case, make sure the key-value secret engine exists before trying to migrate (default engine name is ``kv``). +Make sure the key-value secret engine exists before trying to migrate (default engine name is ``kv``). External Store API ================== diff --git a/docs/pages/development/1-development.rst b/docs/pages/development/1-development.rst deleted file mode 100644 index cd6cd5530e..0000000000 --- a/docs/pages/development/1-development.rst +++ /dev/null @@ -1,86 +0,0 @@ -.. _nuts-node-development: - -Nuts node development -##################### - -Requirements -************ - -.. marker-for-readme - -.. |gover| image:: https://img.shields.io/github/go-mod/go-version/nuts-foundation/nuts-node - :alt: GitHub go.mod Go version - -|gover| or higher is required. - -Building -******** - -Just use ``go build``. - -ES256 Koblitz support -===================== - -To enable ES256K (Koblitz) support, you need to build with the ``jwx_es256k`` tag: - -.. code-block:: shell - - go build -tags jwx_es256k - -Running tests -************* - -Tests can be run by executing - -.. code-block:: shell - - go test ./... - -Code Generation -*************** - -Code generation is used for generating mocks, OpenAPI client- and servers, and gRPC services. -Make sure that ``GOPATH/bin`` is available on ``PATH`` and that the dependencies are installed - -Install ``protoc``: - - | MacOS: ``brew install protobuf`` - | Linux: ``apt install -y protobuf-compiler`` - -Install Go tools: - -.. code-block:: shell - - make install-tools - -Generating code: - -To regenerate all code run the ``run-generators`` target from the makefile or use one of the following for a specific group - -================ ======================= -Group Command -================ ======================= -Mocks ``make gen-mocks`` -OpenApi ``make gen-api`` -Protobuf + gRCP ``make gen-protobuf`` -All ``make run-generators`` -================ ======================= - -Documentation -============= - -The documentation is automatically build on readthedocs based on the config in ``.readthedocs.yaml``. -All files to be included can be generated using: - -.. code-block:: shell - - make cli-docs - -This regenerates files from code, and the ``README.rst`` file which requires python package ``rst-include`` (``pip install rst-include``). - -If needed, you can also build the documentation locally in ``/docs/_build`` using docker: - -.. code-block:: shell - - docker build -t local/nuts-node-docs ./docs - docker run --rm -v ./docs:/docs local/nuts-node-docs \ No newline at end of file diff --git a/docs/pages/development/2-releasing.rst b/docs/pages/development/2-releasing.rst deleted file mode 100644 index f1aca5ef46..0000000000 --- a/docs/pages/development/2-releasing.rst +++ /dev/null @@ -1,94 +0,0 @@ -.. _releasing-nuts-node: - -Releasing Nuts Node -################### - -Semantic versioning -******************* - -Nuts Node and auxiliary tools/applications follow a semantic versioning scheme (``..(-rc.)``): - -| Given a version number MAJOR.MINOR.PATCH, increment the: -| -| 1. MAJOR version when you make incompatible API changes, -| 2. MINOR version when you add functionality in a backwards compatible manner, or -| 3. PATCH version when you make backwards compatible bug fixes. - -(Taken from `semver.org `_) - - Note: "API" is a broad term, it covers every interface interacted with by applications or other nodes (including Nuts network protocols). - -When a new minor or major version is released, always create a release candidate first: ``..0-rc.1``, eg: ``v5.1.0-rc.1``. -This version will symbolize a feature freeze and will be used for the first tests. -All problems will be fixed and the release candidate version is increased on every bugfix release, eg: ``v5.1.0-rc.2``. -When no more problems are found the major/minor version is released without a ``-rc.`` postfix. -This approach prevents the docker ``latest`` tags to be updated to a new version automatically. - -Aside from the Nuts Node itself, all projects that follow the same versions need to be released. -They follow the major version from the Nuts Node, but minor and patch versions may differ. - -Major release -************* - -A major release starts with version number ``.0.0``. Every Nuts Node release has a name (e.g. "Brazil") and a version number. -A release consists of a Git tag and a release in Github with release notes. Releases are created according to the following format: - -- Git tag: ``v..``, e.g. ``v2.0.0`` -- Release name: `` release ()``, e.g.: ``Brazil release (v2.0.0)`` (every release has a designated name) -- Release notes: auto-generated by Github. - -Bugfix release/patches -********************** - -When an issue is fixed in a released version a bugfix/patch version must be released. -The bug must be fixed on a branch named after the major version, e.g. ``v1`` or ``v2``. -The release name follows the release name, but is named "bugfix" instead of "release". E.g.: ``Brazil bugfix (v2.0.1)``. - -Backports -^^^^^^^^^ - -Bugfixes often need to be backported, e.g. it's fixed on the ``master`` branch but also need to be fixed in the last version, -and maybe even in the before last version. Bugfix releases stemming from backports follow the same versioning and naming scheme as regular bugfix releases. - -Building a release -****************** - -Make sure all changes are on the relevant branch. -If it's for an older version (backport), cherry-pick all changes that need to be included and merge them into be the correct branch (e.g., V5.4, V6.1). -Make sure to add the release notes to the branch *before* tagging a release or it will not be visible on read-the-docs. -For good measure, also run ``make cli-docs`` (requires ``rst_include`` python package to be installed) to make sure we didn't forget to update the documentation. - -Go to `releases on github `_ and perform the following steps: - -#. ``Draft new release`` -#. set the target branch -#. ``Choose a tag`` and create a new one according to git tag convention above (e.g. v6.1.0) -#. ``Generate release notes`` -#. (optional) curate release text. For major/minor versions probably replace with the release notes written for read the docs. -#. Set the ``Set as latest release`` checkbox as needed -#. ``Publish release`` - -This will trigger github actions that publish a new release to `Docker Hub `_, and a message will be posted in ``#releases`` channel on Slack. - -Major/Minor version updates -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -**Make a new branch** - -Every new major or minor version has its own branch. -After creating a new release, a new branch should be made based off of the git tag for the release. -The naming convention for branches is ``V.``, e.g. V6.0 or V5.4. (Yes git/github tag/version uses lowercase ``v``, branches use uppercase ``V``, and Docker tags omit the prefix entirely since version 6.0) -Add branch protection to the new branch on Github. - -**read the docs** - -Go to `app.readthedocs.org/projects/nuts-node `_ and click on ``+ Add version`` to add the new branch to the available documentation versions on `nuts-node.readthedocs.io `_. - -**Automated tests** - -Testing is automated using Github workflows. -Some of the tests cannot handle branch patterns and require updating relevant major/minor version branches to the workflow file manually. -The current list of files that need to be updated are: - -- **Scheduled govulncheck** action: ``.github/workflows/govulncheck-cron-schedule.yaml``. Runs every day and sends vulnerability warnings to the ``#nuts-core-team`` slack channel. -- **Scheduled CodeQL** action: ``.github/workflows/codeql-analyisis-cron-schedule.yaml`` diff --git a/docs/pages/development/3-api.rst b/docs/pages/development/3-api.rst deleted file mode 100644 index d69de7778a..0000000000 --- a/docs/pages/development/3-api.rst +++ /dev/null @@ -1,68 +0,0 @@ -.. _api-dev: - -API development -############### - -When developing APIs, please follow these guidelines. - -Contract first -************** - -The Nuts node APIs are specified in `Open API Specification (OAS) `_. -The files are located under ``/docs/_static//.yaml``. -Where ```` is a specific module like ``crypto`` or ``auth`` and ```` defines the version of the API. -We use version ``3.0.y`` of the OAS. - -Versioning -========== - -We use versioning of the APIs. -This is reflected in both the OAS files and the HTTP paths. -Versions must follow the pattern ``v`` and start at ``v1``. -These are major versions, any breaking change results in a new major version of the API. -New additions, bug fixes and changes that are backwards compatible may be done in the current version. - -Code generation -=============== - -The OAS files are used for code generation. The makefile contains the ``gen-api`` target which will generate the code. -The build target only needs to be extended when a new version or new engine is added. -Generated code is always placed in ``//api//generated.go``. - -Return codes -============ - -The error return values are generalized for all API calls. -The return values follow `RFC7807 `_. -The definition is available under ``/docs/_static/common/error_response.yaml``. -The error definition can be used in a OAS file: - -.. code-block:: yaml - - paths: - /some/path: - get: - responses: - default: - $ref: '../common/error_response.yaml' - -The error responses will not be listed as responses in the online generated documentation. -To describe error responses, the specific responses need to be added to the API description: - -.. code-block:: yaml - - paths: - /some/path: - post: - description: | - Some description on the API - - error returns: - * 400 - incorrect input - -Paths -***** - -The API paths are designed so it's clear which APIs are to be blocked for external traffic. - -- ``/internal/**`` These APIs are meant to be behind a firewall and should only be available to the internal infrastructure. diff --git a/docs/pages/development/4-vault.rst b/docs/pages/development/4-vault.rst deleted file mode 100644 index 819f738956..0000000000 --- a/docs/pages/development/4-vault.rst +++ /dev/null @@ -1,37 +0,0 @@ - .. _vault-dev: - -Development with Vault -###################### - -You can start a development Vault server as follows: - -.. code-block:: shell - - docker run --cap-add=IPC_LOCK -d -p 8200:8200 \ - -e 'VAULT_DEV_ROOT_TOKEN_ID=unsafe' -e 'VAULT_ADDRESS=http://localhost:8200' \ - --name=dev-vault \ - vault - -The server will start unsealed, with root token ``unsafe``. - -Now log in and enable a key-value secret engine named ``kv``: - -.. code-block:: shell - - docker exec -e 'VAULT_ADDR=http://0.0.0.0:8200' dev-vault vault login - -Enter the root token ``unsafe``, then enable the ``kv`` engine: - -.. code-block:: shell - - docker exec -e 'VAULT_ADDR=http://0.0.0.0:8200' dev-vault vault secrets enable -path=kv kv - -Then configure the Nuts node to use the Vault server: - -.. code-block:: yaml - - crypto: - storage: vaultkv - vault: - address: http://localhost:8200 - token: unsafe diff --git a/docs/pages/integrating/version-incompatibilities.rst b/docs/pages/integrating/version-incompatibilities.rst deleted file mode 100644 index 23e167f96a..0000000000 --- a/docs/pages/integrating/version-incompatibilities.rst +++ /dev/null @@ -1,33 +0,0 @@ -.. _version-incompatibilities: - -Version Incompatibilities -######################### - -V5/V6, DID methods and API versions -*********************************** - -V6 introduced the support for multiple DID methods. To enable this, a new version of the VDR API has been added. -There's also a config parameter that allows you to limit the DID methods in use. -Not all combinations of API usage and DID methods are supported. -There are basically two options. - -1. Keep using the VDR V1 API (for now) and set ``didmethods`` to ``["nuts"]``. -2. Use the VDR V2 API and set ``didmethods`` to include other methods or leave blank for default setting. - -Do not use the VDR V1 and VDR V2 API at the same time. This will lead to unexpected behavior. -Once you use the VDR V2 API, you cannot go back to the VDR V1 API. The VDR V1 API has also been marked as deprecated. - -Publishing Services for use-cases -********************************* - -V5 use-cases define service endpoints or a collection of endpoints that should be registered in the Services on DID Documents. -The concrete endpoints are usually on the DID Document of the vendor, and then referenced by all DID Documents managed by that vendor. -And ``did:nuts`` for example, requires the registration of a ``NutsComm`` endpoint to authenticate the connection. -Use-cases built on V5 should keep using the DIDMan API to manage and resolve Services on DID Documents. -Any Service change made using the DIDMan API will only update ``did:nuts`` DID Documents. - -For use-cases built on V6, any endpoint needed for the use-case should be listed in the registration on the Discovery Service for that use-case, see :ref:`discovery` Registration. -This means that ``did:web`` DID Documents (or non-did:nuts if we look further ahead) will contain very few Services, if any. -If there is a need to add a Service for V6 use-cases, they should be added using the VDR v2 API, which will then add the Service to _all_ DIDs that are part of the Subject. -Note that resolving Services using the VDR v2 API will return the Service from the document as is. -So, it resolves Services without following any references in the Service to a concrete endpoint as is done by DIDMan. \ No newline at end of file diff --git a/docs/pages/operations/base-url-change.rst b/docs/pages/operations/base-url-change.rst new file mode 100644 index 0000000000..a1ea6281ce --- /dev/null +++ b/docs/pages/operations/base-url-change.rst @@ -0,0 +1,21 @@ +.. _changing-base-url: + +Changing the node's base URL +############################# + +Changing the node's ``url`` (see :ref:`The node's URL `) is basically the same as setting up a new node on a new URL. +All DIDs, VerifiableCredentials, Revocations, and other data is no longer usable. +You will not be able to revoke credentials. This means that all credentials issued under the old URL can no longer be trusted. +Actions should be taken to remove any trust that might have been established in your old identity. + +As an issuer, you'll have to run the new URL side-by-side with the old URL for some time. + +.. note:: + + How long you should run the old URL side-by-side with the new URL depends on the use case and the validity of the credentials. + As a rule of thumb, credentials should be renewed once a year, so you should run the old URL side-by-side with the new URL for at least a year. + +As a holder, you'll have to generate DIDs for all your tenants and have issuers re-issue credentials. +This might involve a lot of manual work from your tenants. + +As a verifier, nothing changes. diff --git a/docs/pages/operations/certificate-revocation.rst b/docs/pages/operations/certificate-revocation.rst new file mode 100644 index 0000000000..7738b65011 --- /dev/null +++ b/docs/pages/operations/certificate-revocation.rst @@ -0,0 +1,23 @@ +.. _certificate-revocation-handling: + +Certificate revocation (CRL) handling +####################################### + +The Nuts-node periodically downloads the Certificate Revocation List (CRL) of configured trusted certificate chains to confirm a certificate has not been revoked. +This applies to the ``tls.truststorefile`` used for :ref:`legacy did:nuts/gRPC client authentication `, and to the certificate chains trusted per use case for :ref:`did:x509 `. + +Soft-fail vs. hard-fail +************************ + +By default the node uses a soft-fail strategy: it does not reject a certificate if the CRL cannot be downloaded. +This prevents a CA with downtime on its CRL endpoint from bringing down the network. +Change this to hard-fail (reject the certificate if its revocation status cannot be established) using the ``pki.softfail`` config flag. + +For ``did:x509`` the node always uses a hard-fail strategy: the ``pki.softfail`` config value is ignored during certificate validation for ``did:x509``. +This means the node will not be able to verify a ``did:x509`` DID or Verifiable Credential if the CRL cannot be downloaded and the cached CRL is older than ``pki.maxupdatefailhours``. + +Operational guidance +********************** + +If you use ``did:x509``, monitor CRL fetch failures and keep ``pki.maxupdatefailhours`` tuned to the actual refresh cadence of your CAs' CRLs — since failures always hard-fail for did:x509, a CRL endpoint outage that outlasts this window will start rejecting otherwise-valid credentials. +For legacy did:nuts/gRPC client authentication, the default soft-fail behavior is usually appropriate; switch to hard-fail only if your use case requires certificate validity to always be provable. diff --git a/docs/pages/deployment/cli-reference.rst b/docs/pages/operations/cli-reference.rst similarity index 58% rename from docs/pages/deployment/cli-reference.rst rename to docs/pages/operations/cli-reference.rst index a489f00f4e..a718b2354d 100755 --- a/docs/pages/deployment/cli-reference.rst +++ b/docs/pages/operations/cli-reference.rst @@ -17,16 +17,6 @@ Prints the current config nuts config [flags] -nuts crypto fs2vault -^^^^^^^^^^^^^^^^^^^^ - -Imports private keys from filesystem based storage into Vault. The given directory must contain the private key files.The Nuts node must be configured to use Vault as crypto storage. Can only be run on the local Nuts node, from the directory where nuts.yaml resides. - -:: - - nuts crypto fs2vault [directory] [flags] - - nuts server ^^^^^^^^^^^ diff --git a/docs/pages/deployment/key-rotation.rst b/docs/pages/operations/key-rotation.rst similarity index 64% rename from docs/pages/deployment/key-rotation.rst rename to docs/pages/operations/key-rotation.rst index 7f61b79235..defd62b42a 100644 --- a/docs/pages/deployment/key-rotation.rst +++ b/docs/pages/operations/key-rotation.rst @@ -5,13 +5,12 @@ Key rotation procedure To minimize the impact of stolen/leaked keys, private keys should be rotated at a regular, scheduled interval. This applies to any private key used for a longer period of time. -The node aids this procedure by supporting operations to add to DID documents. -Removal of keys is currently not supported. Newer keys are automatically used for cryptographic operations. +The node currently only supports the "add" half of rotation: adding a new key to a DID document. +Removing an existing key is not supported — once added, a key remains part of the DID document indefinitely. +Newer keys are automatically used for cryptographic operations. -Procedure -********* - -The procedure to rotate a key is two fold. The two procedures can be performed independently. +Adding a new key +***************** Given a period of time, eg. every month when issuing a lot of credentials or every year when issuing only a few, a new key should be added to the DID document. @@ -20,10 +19,7 @@ Given a period of time, eg. every month when issuing a lot of credentials or eve The current API doesn't support finding VCs based on validity period or specific key. The only possibility is to find all and loop over the results to check the validity period and the key used to sign the VC. -1. Add a new key -================ - -Then, you add a new key which generates a new key pair in your crypto storage and adds it to the DID document: +You add a new key, which generates a new key pair in your crypto storage and adds it to the DID document: .. code-block:: shell diff --git a/docs/pages/deployment/migration.rst b/docs/pages/operations/migration.rst similarity index 92% rename from docs/pages/deployment/migration.rst rename to docs/pages/operations/migration.rst index a9ca00acaf..56a7148278 100644 --- a/docs/pages/deployment/migration.rst +++ b/docs/pages/operations/migration.rst @@ -104,7 +104,12 @@ Creating DIDs with ``selfControl=false`` Mixing VDR v1 and v2 APIs ------------------------- -The v1 and v2 APIs read from different stores. Do not mix usage, or you risk data drift and stale reads. VDR v1 / DIDMan v1 are deprecated and slated for removal in a future major release. +V6 introduced support for multiple DID methods, backed by a new version of the VDR API. Not all combinations of API usage and ``didmethods`` are supported — there are basically two options: + +1. Keep using the VDR v1 API (for now) and set ``didmethods = ["nuts"]``. +2. Use the VDR v2 API and set ``didmethods`` to include other methods, or leave it at its default. + +The v1 and v2 APIs read from different stores. Do not mix usage, or you risk data drift and stale reads. Once you use the VDR v2 API on a subject, you cannot go back to the VDR v1 API for it. VDR v1 / DIDMan v1 are deprecated and slated for removal in a future major release. .. _also-enabling-did-web: diff --git a/docs/pages/release_notes.rst b/docs/pages/release_notes.rst index e70c3064cb..3b73be5948 100644 --- a/docs/pages/release_notes.rst +++ b/docs/pages/release_notes.rst @@ -6,14 +6,20 @@ Release notes Unreleased **************** -## New features +============ +New Features +============ + * #4063: Enable ``storage.debug`` flag to log go-leia performance issues (full table scans, suboptimal index usage) by @reinkrul in https://github.com/nuts-foundation/nuts-node/pull/4064 * #4078: Allow policy profiles to define a ``service_provider`` PresentationDefinition for the OAuth client (RFC 7523 ``jwt-bearer`` flow) by @stevenvegt in https://github.com/nuts-foundation/nuts-node/pull/4226 * #4078: Add the experimental RFC 7523 ``jwt-bearer`` two-VP token request flow, gated behind ``auth.experimental.jwtbearerclient`` (default ``false``, subject to change) by @stevenvegt in https://github.com/nuts-foundation/nuts-node/pull/4227 * #4078: Expose the experimental two-VP flow on ``POST /internal/auth/v2/{subjectID}/request-service-access-token`` via the optional ``service_provider_subject_id`` body field by @stevenvegt in https://github.com/nuts-foundation/nuts-node/pull/4228 * #4233: ``request-credential`` API gains an optional ``credential_request_params`` JSON object overlaid on top of the OpenID4VCI Credential Request body sent to the issuer. Lets the wallet talk to issuers that accept additional fields, or to override the credential request entirely. -## Security +======== +Security +======== + * #4441: Inbound HTTP request bodies are now limited to 1MB on both the public and internal interfaces; larger requests are rejected with HTTP 413 (Request Entity Too Large). Previously no limit was enforced, contrary to what the deployment documentation stated. The heaviest legitimate requests (OAuth POSTs carrying Verifiable Presentations) stay well below this limit, and it matches the ``client_max_body_size 1M`` reverse proxy configuration the documentation recommends. By @stevenvegt in https://github.com/nuts-foundation/nuts-node/pull/4441 * #4439: Helm chart (version 0.0.9): default ``verbosity`` changed from ``debug`` to ``info``, matching the node's own default. Debug verbosity produces far more log output than production needs and increases the impact of any log-hygiene issue. Set ``nuts.config.verbosity: debug`` in your own values to restore the old behavior. By @stevenvegt in https://github.com/nuts-foundation/nuts-node/pull/4439 * #4440: In strictmode, ``http.log: metadata-and-body`` is no longer honored: the node resets it to ``metadata`` at startup and logs a warning. Full body logging wrote OAuth token endpoint request and response bodies (client assertions, VP tokens, authorization codes and issued access tokens) to the log at Info severity. Non-strictmode deployments are unaffected. By @stevenvegt in https://github.com/nuts-foundation/nuts-node/pull/4440 @@ -459,7 +465,7 @@ Breaking changes - Removed legacy API authentication tokens. - Removed ``auth.publicURL`` config param. The requirement for Yivi is now also covered by the ``url`` config param value. - Default port bindings and http config options have changed. See below. -- See caveats in :ref:`version-incompatibilities`. +- See caveats in :ref:`nuts-node-migrations`. ============ New Features