Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions COLLABORATORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ new major version, which the maintainers plan.
What counts as breaking and how it maps to version numbers is already
documented:

- [API development](docs/pages/development/3-api.rst) — API versioning rules.
- [Releasing — Semantic versioning](docs/pages/development/2-releasing.rst) —
- [API development](docs/README.md#api-development-guidelines) — API versioning rules.
- [Releasing — Semantic versioning](RELEASING.md#semantic-versioning) —
how "API" is scoped (it also covers config and network protocols) and
which version part a change bumps.

Expand Down
3 changes: 3 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Contributing to nuts-node

See [COLLABORATORS.md](COLLABORATORS.md) for how to contribute to `nuts-node`, and [DEVELOPMENT.md](DEVELOPMENT.md) for setting up a development environment.
105 changes: 105 additions & 0 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Developing nuts-node

## Requirements

![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/nuts-foundation/nuts-node)
or higher is required.

## Building

Just use `go build`.

### ES256 Koblitz support

To enable ES256K (Koblitz) support, you need to build with the `jwx_es256k` tag:

```shell
go build -tags jwx_es256k
```

## Running tests

Tests can be run by executing

```shell
go test ./...
```

## Code generation

Code generation is used for generating mocks, OpenAPI client- and servers, and gRPC services.
Make sure that `GOPATH/bin` is available on `PATH` and that the dependencies are installed.

Install `protoc`:

- MacOS: `brew install protobuf`
- Linux: `apt install -y protobuf-compiler`

Install Go tools:

```shell
make install-tools
```

Generating code:

To regenerate all code run the `run-generators` target from the makefile, or use one of the following for a specific group:

| Group | Command |
|-----------------|------------------------|
| Mocks | `make gen-mocks` |
| OpenApi | `make gen-api` |
| Protobuf + gRPC | `make gen-protobuf` |
| All | `make run-generators` |

See [docs/README.md](docs/README.md) for API development guidelines (OpenAPI contract-first workflow, versioning, error responses).

### Documentation

The documentation is automatically built on readthedocs based on the config in `.readthedocs.yaml`.
All files to be included can be generated using:

```shell
make cli-docs
```

This regenerates the config-option and CLI reference tables from code, and the root `README.rst` from `README_template.rst`.
Whenever you add, remove, or change a config flag or CLI command, run this and commit the regenerated files —
don't hand-edit `docs/pages/configuration/server_options*.rst` or `docs/pages/operations/cli-reference.rst`.

If needed, you can also build the documentation locally in `/docs/_build` using Docker; see [docs/README.md](docs/README.md).

## Developing with Vault

You can start a development Vault server as follows:

```shell
docker run --cap-add=IPC_LOCK -d -p 8200:8200 \
-e 'VAULT_DEV_ROOT_TOKEN_ID=unsafe' -e 'VAULT_ADDRESS=http://localhost:8200' \
--name=dev-vault \
vault
```

The server will start unsealed, with root token `unsafe`.

Now log in and enable a key-value secret engine named `kv`:

```shell
docker exec -e 'VAULT_ADDR=http://0.0.0.0:8200' dev-vault vault login
```

Enter the root token `unsafe`, then enable the `kv` engine:

```shell
docker exec -e 'VAULT_ADDR=http://0.0.0.0:8200' dev-vault vault secrets enable -path=kv kv
```

Then configure the Nuts node to use the Vault server:

```yaml
crypto:
storage: vaultkv
vault:
address: http://localhost:8200
token: unsafe
```
434 changes: 219 additions & 215 deletions README.rst

Large diffs are not rendered by default.

5 changes: 2 additions & 3 deletions README_template.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,10 @@ Claude Code skill files for common maintenance tasks can be found in the `.claud
Development
^^^^^^^^^^^

.. include:: docs/pages/development/1-development.rst
:start-after: .. marker-for-readme
See `DEVELOPMENT.md <DEVELOPMENT.md>`_ for build, test and code generation instructions.

Configuration
^^^^^^^^^^^^^

.. include:: docs/pages/deployment/configuration.rst
.. include:: docs/pages/configuration/options.rst
:start-after: .. marker-for-readme
85 changes: 85 additions & 0 deletions RELEASING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Releasing Nuts Node

## Semantic versioning

Nuts Node and auxiliary tools/applications follow a semantic versioning scheme (`<major>.<minor>.<patch>(-rc.<rc>)`):

> Given a version number MAJOR.MINOR.PATCH, increment the:
>
> 1. MAJOR version when you make incompatible API changes,
> 2. MINOR version when you add functionality in a backwards compatible manner, or
> 3. PATCH version when you make backwards compatible bug fixes.
>
> (Taken from [semver.org](https://semver.org/))

Note: "API" is a broad term, it covers every interface interacted with by applications or other nodes (including Nuts network protocols).

When a new minor or major version is released, always create a release candidate first: `<major>.<minor>.0-rc.1`, e.g.: `v5.1.0-rc.1`.
This version will symbolize a feature freeze and will be used for the first tests.
All problems will be fixed and the release candidate version is increased on every bugfix release, e.g.: `v5.1.0-rc.2`.
When no more problems are found the major/minor version is released without a `-rc.<rc>` postfix.
This approach prevents the docker `latest` tag from being updated to a new version automatically.

Aside from the Nuts Node itself, all projects that follow the same versions need to be released.
They follow the major version from the Nuts Node, but minor and patch versions may differ.

## Major release

A major release starts with version number `<major>.0.0`. Every Nuts Node release has a name (e.g. "Brazil") and a version number.
A release consists of a Git tag and a release in Github with release notes. Releases are created according to the following format:

- Git tag: `v<major>.<minor>.<patch>`, e.g. `v2.0.0`
- Release name: `<name> release (<version>)`, e.g.: `Brazil release (v2.0.0)` (every release has a designated name)
- Release notes: auto-generated by Github.

## Bugfix release/patches

When an issue is fixed in a released version a bugfix/patch version must be released.
The bug must be fixed on a branch named after the major version, e.g. `v1` or `v2`.
The release name follows the release name, but is named "bugfix" instead of "release". E.g.: `Brazil bugfix (v2.0.1)`.

### Backports

Bugfixes often need to be backported, e.g. it's fixed on the `master` branch but also needs to be fixed in the last version,
and maybe even in the version before that. Bugfix releases stemming from backports follow the same versioning and naming scheme as regular bugfix releases.

## Building a release

Make sure all changes are on the relevant branch.
If it's for an older version (backport), cherry-pick all changes that need to be included and merge them into the correct branch (e.g., V5.4, V6.1).
Make sure to add the release notes to the branch *before* tagging a release or it will not be visible on read-the-docs.
For good measure, also run `make cli-docs` (requires the `rst_include` python package to be installed) to make sure we didn't forget to update the documentation.

Go to [releases on github](https://github.com/nuts-foundation/nuts-node/releases) and perform the following steps:

1. `Draft new release`
2. set the target branch
3. `Choose a tag` and create a new one according to the git tag convention above (e.g. v6.1.0)
4. `Generate release notes`
5. (optional) curate release text. For major/minor versions probably replace with the release notes written for read the docs.
6. Set the `Set as latest release` checkbox as needed
7. `Publish release`

This will trigger github actions that publish a new release to [Docker Hub](https://hub.docker.com/r/nutsfoundation/nuts-node/tags), and a message will be posted in the `#releases` channel on Slack.

### Major/Minor version updates

**Make a new branch**

Every new major or minor version has its own branch.
After creating a new release, a new branch should be made based off of the git tag for the release.
The naming convention for branches is `V<major>.<minor>`, e.g. V6.0 or V5.4. (Yes, git/github tag/version uses lowercase `v`, branches use uppercase `V`, and Docker tags omit the prefix entirely since version 6.0)
Add branch protection to the new branch on Github.

**Read the docs**

Go to [app.readthedocs.org/projects/nuts-node](https://app.readthedocs.org/projects/nuts-node/) and click on `+ Add version` to add the new branch to the available documentation versions on [nuts-node.readthedocs.io](https://nuts-node.readthedocs.io/).

**Automated tests**

Testing is automated using Github workflows.
Some of the tests cannot handle branch patterns and require updating relevant major/minor version branches to the workflow file manually.
The current list of files that need to be updated are:

- **Scheduled govulncheck** action: `.github/workflows/govulncheck-cron-schedule.yaml`. Runs every day and sends vulnerability warnings to the `#nuts-core-team` slack channel.
- **Scheduled CodeQL** action: `.github/workflows/codeql-analysis-cron-schedule.yml`
4 changes: 3 additions & 1 deletion core/server_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,9 @@ func FlagSet() *pflag.FlagSet {
flagSet.Bool("strictmode", defaultCfg.Strictmode, "When set, insecure settings are forbidden.")
flagSet.Bool("internalratelimiter", defaultCfg.InternalRateLimiter, "When set, expensive internal calls are rate-limited to protect the network. Always enabled in strict mode.")
flagSet.String("datadir", defaultCfg.Datadir, "Directory where the node stores its files.")
flagSet.String("url", defaultCfg.URL, "Public facing URL of the server (required). Must be HTTPS when strictmode is set.")
flagSet.String("url", defaultCfg.URL, "Public facing URL of the server (required). Must be HTTPS when strictmode is set. "+
"It's baked into every DID and OAuth identity the node issues, so choose a domain you own, that is stable (avoid TLDs that block re-registration after expiry, "+
"and cloud-provider subdomains that don't identify the owner), and that serves security.txt and robots.txt at its root.")
flagSet.StringSlice("didmethods", defaultCfg.DIDMethods, "Comma-separated list of enabled DID methods (without did: prefix). "+
"It also controls the order in which DIDs are returned by APIs, and which DID is used for signing if the verifying party does not impose restrictions on the DID method used.")
flagSet.Duration("httpclient.timeout", defaultCfg.HTTPClient.Timeout, "Request time-out for HTTP clients, such as '10s'. Refer to Golang's 'time.Duration' syntax for a more elaborate description of the syntax.")
Expand Down
56 changes: 56 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,59 @@ To build the documentation locally, run `make cli-docs` in the parent directory
docker build -t nuts-node-docs .
docker run --rm -v $PWD:/docs nuts-node-docs
```

## API development guidelines

When developing APIs, please follow these guidelines.

### Contract first

The Nuts node APIs are specified in [Open API Specification (OAS)](https://swagger.io/specification/).
The files are located under `/docs/_static/<engine>/<version>.yaml`, where `<engine>` is a specific module like `crypto` or `auth`, and `<version>` defines the version of the API.
We use version `3.0.y` of the OAS.

#### Versioning

We use versioning of the APIs. This is reflected in both the OAS files and the HTTP paths.
Versions must follow the pattern `v` and start at `v1`. These are major versions: any breaking change results in a new major version of the API.
New additions, bug fixes and changes that are backwards compatible may be done in the current version.

#### Code generation

The OAS files are used for code generation. The makefile contains the `gen-api` target which generates the code.
The build target only needs to be extended when a new version or new engine is added.
Generated code is always placed in `/<engine>/api/<version>/generated.go`.

#### Return codes

The error return values are generalized for all API calls. The return values follow [RFC7807](https://tools.ietf.org/html/rfc7807).
The definition is available under `/docs/_static/common/error_response.yaml`. The error definition can be used in an OAS file:

```yaml
paths:
/some/path:
get:
responses:
default:
$ref: '../common/error_response.yaml'
```

The error responses will not be listed as responses in the online generated documentation.
To describe error responses, the specific responses need to be added to the API description:

```yaml
paths:
/some/path:
post:
description: |
Some description on the API

error returns:
* 400 - incorrect input
```

### Paths

The API paths are designed so it's clear which APIs are to be blocked for external traffic.

- `/internal/**` These APIs are meant to be behind a firewall and should only be available to the internal infrastructure.
2 changes: 1 addition & 1 deletion docs/_static/didman/v1.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,7 @@ components:
type: object
description: >
An object describing the found entity, modelled as a concept as specified by VCR's OpenAPI specification.
See https://nuts-node.readthedocs.io/en/latest/pages/development/3-vc.html for examples on which concepts are supported and how they're structured.
See https://nuts-node.readthedocs.io/en/latest/pages/api/verifiable-credentials.html for examples on which concepts are supported and how they're structured.
didDocument:
$ref: '../common/ssi_types.yaml#/components/schemas/DIDDocument'
EndpointResponse:
Expand Down
7 changes: 6 additions & 1 deletion docs/_static/vdr/v1.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
openapi: "3.0.0"
info:
title: Nuts Verifiable Data Registry API spec
description: API specification for the Verifiable Data Registry
description: |
API specification for the Verifiable Data Registry.

Deprecated: this is the v1 API, which only supports `did:nuts`. Use the v2 API for new integrations.
Do not mix v1 and v2 API usage on the same subject: they read from different stores, and switching from v2 back to v1 is not supported.
See the migration guide (Operations > Migration) for details.
version: 1.0.0
license:
name: GPLv3
Expand Down
8 changes: 4 additions & 4 deletions docs/generate_docs.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ func (sl stringSlice) contains(s string) bool {
}

// serverCommands lists the commands that use the server config. The options server commands are only printed once, because the list is quite long.
var serverCommands stringSlice = []string{"nuts config", "nuts server", "nuts crypto fs2vault", "nuts crypto fs2external", "nuts http gen-token"}
var serverCommands stringSlice = []string{"nuts config", "nuts server", "nuts crypto fs2external", "nuts http gen-token"}

func generateDocs() {
system := cmd.CreateSystem(func() {})
Expand All @@ -56,7 +56,7 @@ func generateDocs() {
}

func generateCLICommands(system *core.System) {
const targetFile = "docs/pages/deployment/cli-reference.rst"
const targetFile = "docs/pages/operations/cli-reference.rst"
writer, _ := os.OpenFile(targetFile, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, os.ModePerm)
defer writer.Close()

Expand Down Expand Up @@ -137,8 +137,8 @@ func generateServerOptions(system *core.System) {
},
}

generatePartitionedConfigOptionsDocs("Server Options", "docs/pages/deployment/server_options.rst", filterFlags(flags, v5FlagsPredicates, true))
generatePartitionedConfigOptionsDocs("did:nuts/gRPC Server Options", "docs/pages/deployment/server_options_didnuts.rst", filterFlags(flags, v5FlagsPredicates, false))
generatePartitionedConfigOptionsDocs("Server Options", "docs/pages/configuration/server_options.rst", filterFlags(flags, v5FlagsPredicates, true))
generatePartitionedConfigOptionsDocs("did:nuts/gRPC Server Options", "docs/pages/configuration/server_options_didnuts.rst", filterFlags(flags, v5FlagsPredicates, false))
}

func filterFlags(flags map[string]*pflag.FlagSet, predicates []func(f *pflag.Flag) bool, exclude bool) map[string]*pflag.FlagSet {
Expand Down
Loading
Loading