migrate to uv

.vscode

.env

__pycache__

.env

.pytest_cache

.venv

.vscode

lib

[[source]]

url = "https://pypi.org/simple"

verify_ssl = true

name = "pypi"

[packages]

rdflib = "~=7.0.0"

rdflib-leveldb = {git = "https://github.com/RDFLib/rdflib-leveldb"}

owlready2 = "*"

gitpython = "*"

pyyaml = "*"

pyshacl = "*"

requests = "*"

dotenv = "*"

python-gitlab = "*"

jinja2 = "*"

dominate = "*"

markdown = "*"

pygments = "*"

beautifulsoup4 = "*"

python-docx = "*"

python-dotenv = "*"

[dev-packages]

pytest = "*"

# SAREF Pypeline

This project implements the SAREF development and documentation generation pipeline.

This project provides a number of tools to SAREF ontology developers:

SAREF Pipeline rewritten in Python with more functionalities and improved performances.

SAREF Pypeline implements the Technical Specification:

* check the conformance of a SAREF project against ETSI TS 103 673 [[1]](#[1])

* generate the documentation website https://saref.etsi.org/

* generate ETSI Technical Specification documents from the sources of the ontology

  ETSI TS 103 673 V1.2.1: "SmartM2M; SAREF Development Framework and Workflow, Streamlining the Development of SAREF and its Extensions"

The SAREF pipeline is part of the SAREF Development Framework. The following projects contributed to the SAREF Pipeline:

* [STF 578](https://portal.etsi.org/STF/STFs/STF-HomePages/STF578)

* [STF 653](https://portal.etsi.org/XTFs/#/xTF/653)

* [StandICT grant](https://standict.eu/)

TODO: publish to pypi.

## Installation

 [Download the latest build here](https://labs.etsi.org/rep/saref/saref-pipeline/-/releases/permalink/latest/downloads/saref-pipeline.jar)

NOTE: SAREF Pypeline has been tested on Linux and Windows WSL2. It's not a goal at the moment to ensure it's working in every environemnt, or provide support for installing and using it. 

This SAREF-Pipeline may be run:

**Prerequisites:** 

- by double-clicking the JAR file to starting the graphical user interface

- or using the command line interface, run `java -jar saref-pipeline.jar help` to get started

* [`uv`](https://docs.astral.sh/uv/) (recommended) or `pipx` with Python >= 3.13

* [`leveldb`](https://github.com/google/leveldb) (package `libleveldb-dev` on ubuntu)

## Usage

**With uv:** 

Requires Python >3.13 ;  libleveldb-dev

`uv tool install saref-pypeline`

Install from pypi:

**With pipx:**

```

pip install saref-pypeline

```

`pipx install saref-pypeline`

TODO: publish to pypi.

## Basic usage

Run as follows:

```

usage: python saref-pypeline <mode> [<options>] [<target>]

Runs the SAREF pipeline as specified in Technical Specification:

  ETSI TS 103 673 V1.2.1: "SmartM2M; SAREF Development Framework and

            Workflow, Streamlining the Development of SAREF and its

            Extensions"

<mode> can take the following values:

  develop   Run the SAREF pipeline in relax mode in the target SAREF

            project

  release   Run the SAREF pipeline in strict mode in the target SAREF

            project

  prerelease-portal    Operate a strict check and generate the portal for

            pre-release or release branches of each source in the

            configuration file `.saref-repositories.yml`

  release-portal    Operate a strict check generate the portal for release

            branches of each source in the configuration file

            `.saref-repositories.yml`

  clean     Remove all files generated by the previous executions

  help      Displays this message

<target> points to the target directory. By default, target is the current

            directory.

<options> can take the following values:

 -e,--no-examples   Do not check examples

 -s,--no-site       Do not generate the static portal

 -t,--no-terms      Do not generate the static portal for terms

$ saref-dev --help

usage: saref-dev [-h] {clean,check,website,ts,ts2md,help} ...

Check or document SAREF projects.

positional arguments:

  {clean,check,website,ts,ts2md,help}

    clean               Delete the target folder.

    check               Check a SAREF project.

    website             Generate the documentation website for a SAREF project.

    ts                  Generate the Technical Specification (DOCX) for a SAREF project.

    ts2md               Experimental - Extract documentation files from the Technical Specification (DOCX) of a SAREF project.

    help                Show the help message and exit.

options:

  -h, --help            show this help message and exit

```

TODO: replace with new help

## Installation from the sources

Requires Python >3.13

Ensure you have python and pipenv installed:

Commands `check`, `website`, `ts`, take a set of common options:

```

python3 -m pip install pipenv

$ saref-dev check --help

usage: saref-dev check [-h] [--verbose] [--strict] [-s DIR] [-p REGEX] [-v REGEX] [-d REGEX] [--skip-fetch] [-c REGEX] [directory]

Check a SAREF project.

positional arguments:

  directory             Project directory (default: current directory).

options:

  -h, --help            show this help message and exit

  --verbose, -V         Increase verbosity level (default: WARNING. -V: INFO. -VV: DEBUG. -VVV TRACE).

  --strict              Only consider release branches.

  -s, --sources DIR     Path to the folder where projects are stored/cloned (default: target/sources relative to the project directory).

  --skip-fetch          Skip git fetch for source folders.

  -p, --include-projects REGEX

                        Regex to select which projects to process. Regexes are evaluated against the project name (e.g., "SAREF", "^SAREF4[A-E]"). Can be given multiple times. Shall only be used on the saref-portal project.

  -v, --include-versions REGEX

                        Regex to select which project versions to process. Regexes are evaluated against the project name and version (e.g., "SAREF V4.1.1", "^SAREF4[A-E] V2.1.1"). Shall be compatible with --include-projects. Can be given multiple times.

                        Shall only be used on the saref-portal project.

  -d, --include-documents REGEX

                        Regex to select which documents to check (ontologies, examples, vocabularies, patterns). Evaluated against the path to the documents. Can be given multiple times.

  -c, --include-clauses REGEX

                        Regex to select which clauses to check. Regexes Evaluated against the TS and clause name (e.g., "TS103673 Clause 9.7.1"). Can be given multiple times.

```

Install the dependencies:

Only option `-c` is not available to `ts` and `ts2md`.

```

pipenv install

```

## About

## Configuration and execution

The SAREF pypeline is part of the SAREF Development Framework. The following projects contributed to the SAREF Pipeline:

Set up your credentials to the ETSI Forge in a file `.env` file. For example with:

* [ETSI STF 578](https://portal.etsi.org/STF/STFs/STF-HomePages/STF578): "Specification of the SAREF development framework and workflow, and development of the Community SAREF Portal for user engagement"

* [ETSI STF 653](https://portal.etsi.org/XTFs/#/xTF/653): "SAREF Patterns"

* [StandICT.eu 2026 grant 07-968](https://standict.eu/): "Improving the ETSI TC SmartM2M SAREF publication framework and workflow"

```

user=lefranois

password=mypasswordisverylong

```

The SAREF pypeline is a port to python of the [SAREF pipeline](https://labs.etsi.org/rep/saref/saref-pipeline), with more features and improved performances.

Run the script: 

```

python3 main.py

```

## References

* <a id="[1]"></a> [ETSI TS 103 673 V1.2.1](https://www.etsi.org/deliver/etsi_ts/103600_103699/103673/01.02.01_60/ts_103673v010201p.pdf): "SmartM2M; SAREF Development Framework and Workflow, Streamlining the Development of SAREF and its Extensions"

## License

[build-system]

requires = ["setuptools>=61.0"]

build-backend = "setuptools.build_meta"

[project]

name = "saref-pypeline"

version = "0.1"

description = "SAREF pipeline tools"

authors = [

  { name = "Maxime Lefrançois", email = "maxime.lefrancois@emse.fr" }

]

requires-python = ">= 3.13"

dependencies = [

  "rdflib ~=7.0.0",

  "rdflib-leveldb @ git+https://github.com/RDFLib/rdflib-leveldb",

  "owlready2",

  "gitpython",

  "pyyaml",

  "pyshacl",

  "requests",

  "python-gitlab",

  "jinja2",

  "dominate",

  "markdown",

  "pygments",

  "beautifulsoup4",

  "python-docx",

  "python-dotenv",

]

[tool.setuptools.packages.find]

where = ["src"]

[project.scripts]

saref-dev = "saref_pypeline.__main__:main"