# TOP: TDL Open source Project

Welcome to the TOP TDL repository. We are transitioning to GitLab as the platform for collaboration under the ETSI Labs umbrella. We are still updating all links and documentation so please bear with us until the transition is completed. Let us know if there are some issues so we can address them promptly.

More info at [TDL website](https://tdl.etsi.org)

## Installing the plug-ins

The TDL tools are available as a set of Eclipse plug-ins. You need to first be able to run Eclipse. The [Eclipse Modeling Tools (2021-06)](https://www.eclipse.org/downloads/packages/release/2021-06/r/eclipse-modeling-tools) distribution contains most of the necessary prerequisites for the TDL tools. Newer versions should be compatible but have not been extensively tested yet.

Once Eclipse is up and running, the easiest way to install the TOP tools is from the Eclipse Marketplace - simply search for TDL in the Eclipse Marketplace client within Eclipse, or go to the [TOP Marketplace Listing](https://marketplace.eclipse.org/content/top-tdl-open-source-project) and drag the install button into the Eclipse window.

Alternatively, you can install the plug-ins manually from an update site. In this case, you need to add the [update site](https://tdl.etsi.org/eclipse/latest/) for the TDL tools: 

    https://tdl.etsi.org/eclipse/latest/

After the update site is added, the list of plug-ins and features shall be shown in the installation dialog (menu item Help->Install New Software..). Select the desired features and proceed with the installation, which will require Eclipse to be restarted once the installation is finished.

In Eclipse 2021-06 some components may need to be updated first to ensure compatibility with the graphical editor. During the installation, this may be proposed as a solution. 

Find more detailed information at [Installation wiki](https://labs.etsi.org/rep/top/ide/-/wikis/Installation).

## Using the plug-ins

Before any models can be created, you need to create a TDL project. In file menu select New and Project...
In the following page, locate the TDL Project and continue to complete the wizard. A file `model.tdl` is created which contains the basic structure of TDL model.

To create a new Generic **TDL diagram**:

* right click on the project and select Create Representation
* select the Generic TDL and on the next page select the root package in the model.tdl file

To create a new TDL model by means of the **TDLan2 textual representation**: 

* right click on a project or a folder in the package explorer and select New -> File,
* type the filename ending with `.tdlan2` and 
* the corresponding editor shall be automatically opened.

To create a new TDL model by means of the TPLan2 textual representation for **structured test objectives**:

* right click on a project or a folder in the package explorer and select New -> File, 
* type the filename ending with `.tplan2` and 
* the corresponding editor shall be automatically opened.

To translate a TDL model from the TDLan2 textual representation to the **XF format** (or the other way around) open the source model and press the <code>T</code> icon on the toolbar. The resulting model shall be stored in the same folder as the source model with the type of the model (`.tdlan2` or `.tdl`) appended to the name of the source model.

The same steps can be applied to create a new TDL model by means of the brace-based and indentation-based **TDL TX textual representations** (using the `.tdltx` and `.tdltxi` endings for the filenames). The translation functionality provides a dialog to select the target format, including the **TDL TX** representations.

More information on using the plugins is available at [Usage wiki](https://labs.etsi.org/rep/top/ide/-/wikis/Usage).

## Textual Syntax Specifications

The complete specification for the standardised textual syntax ([ES 203 119-8 v1.1.1 (to be published in March 2022)](https://tdl.etsi.org/index.php/downloads))  as well as the corresponding extensions for Structured Testo Objective Specifications ([ES 203 119-4 v1.5.1, clause 8 (to be published in March 2022)](https://tdl.etsi.org/index.php/downloads)) Extended Test Configurations ([ES 203 119-7 v1.3.1, clause 8 (to be published in March 2022)](https://tdl.etsi.org/index.php/downloads)) is available in [TDLtx.xtext](plugins/org.etsi.mts.tdl.tx/src/org/etsi/mts/tdl/TDLtx.xtext). It uses the [Xtext Grammar Language](https://www.eclipse.org/Xtext/documentation/301_grammarlanguage.html) which also allows the specification of the bindings between the textual syntax elements and the meta-model elements. By default, brace-based delimiters are used (i.e. the "brace-based variant"). Instead, the [TDLtxi.xtext](plugins/org.etsi.mts.tdl.txi/src/org/etsi/mts/tdl/TDLtxi.xtext) specifies the necessary overrides for the indentation-based delimiters (i.e. the "indentation-based variant"). 

From end-user's perspective this translates to two file extensions - `.tdltx` for the brace-based variant and `.tdltxi` for the indentation-based variant. The TOP plug-ins provide a function (from the menu "TDL->Translate TDL modeling") to convert between the available formats, including the standardised exchange format ([ES 203 119-3 v1.5.1 (to be published in March 2022)](https://tdl.etsi.org/index.php/downloads)). The available formats depend on the installed plug-ins.

## Setting up a local environment for development and testing

 * Download and install Eclipse Modeling Tools package
 * Install following additional components from Eclipse Marketplace
     * Sirius (v7)
     * Eclipse Xtext (v2.26 or newer)
     * Epsilon (v2.x)
     * Eclipse OCL (v6.x)
 * Check out the repository
 * Import the plug-ins in the Eclipse workspace
 * Generate the meta-model code
     * Open the `tdl.genmodel` 
     * Make sure that the "Realisation of OCL embedded within Ecore models" is set to "Delegate for interpretation at run-time" in the Eclipse preferences
     * Right click on the top-node and generate model code 
 * Generate textual editor code
     * Open the `GenerateTDLan2.mwe`
     * Right click and select Run As -> MWE2 Workflow
     * Additionally (or alternatively) repeat the same steps for the `GenerateTPLan2.mwe` (for editing structured test objectives)
     * Additionally (or alternatively) repeat the same steps for the `GenerateTDLtx.mwe` (for brace-based TDL TX representation)
     * Additionally (or alternatively) repeat the same steps for the `GenerateTDLtxi.mwe` (for indentation-based TDL TX representation)
     * Additionally (or alternatively) repeat the same steps for the `GenerateData.mwe` (for the graphical viewer)

## Converters

The converters component can be optionally installed with the TDL core and editor components. It includes implementaions for importing data from OpenAPI and ASN.1 specifications, as well as exporting structured test objectives into Word documents and converting structured test objectives to test descriptions. The functionality is depends on the currently selected resource. To import data from OpenAPI or ASN.1 specifications, use the corresponding menu item or press the `>T>` icon on the toolbar. Note that the converters functionality is still in active development and some parts may be incomplete, while others may change.

### Implementation of the OpenAPI import into TDL

The generation of data types and other elements based on OpenAPI yaml specifications is based on the [KaiZen OpenAPI Parser](https://github.com/RepreZen/KaiZen-OpenApi-Parser). The corresponding [Eclipse plug-in](https://marketplace.eclipse.org/content/kaizen-openapi-editor) needs to be installed before installing the importer. While the transformation components can be installed individually, the user-interface currently depends on all the transformation plug-ins.

The implementation currently supports: 
 * Importing of data definitions under `components/schemas`.
 * Data mappings to the base OpenAPI definition for traceability.
 * Annotations with warnings regarding conventions.

Part of the implementation was developed as a lab project with contributions from:
 * Lennart Thiesen
 * Marlin Preibisch
 * Niklas Mertsch
 * René Lars Wetzelt
 * Vincent Florens Hasse

### Implementation of the ASN.1 import into TDL
The generation of data types based on ASN.1 specifications is based on the [BeanIt ASN1Bean](https://github.com/beanit/asn1bean) (formerly known as jASN).

The implementation currently supports: 
 * Importing of data definitions within the module (imports from other modules) are currently not resolved.
 * Application of annotations or data constraints (e.g. for `CHOICE` data types or other modifiers).

Note that the upstream components provided by the ASN1Bean may not be able to process all ASN.1 specifications as they do not provide 100% coverage of the ASN.1 syntax. In this case, the outcome may be incomplete.

### Implementation of the transformation of Structured Test Objectives to TDL Test Descriptions
The transformation of Structured Test Objectives into TDL Test Descriptions currently supports:
 * Transforming of inline data descriptions within `EventOccurrences` into corresponding data types.
 * Generation of `TestConfigurations` based on the `EventSequences` within the Structured Test Objective.
 * Generation of `TestDescription` skeletons based on the `EventSequences` within the Structured Test Objective with references to it.
 * Importing of the base package and imported packages.

### Implementation of the TDL mapping to TTCN-3

The TDL mapping to TTCN-3 relies on some functionalities from [TRex2 / T3Tools](https://t3tools.informatik.uni-goettingen.de/trac). Before installing or developing the implementation of the TDL mapping to TTCN-3, the corresponding components need to be installed from the following update site:

    https://t3tools.informatik.uni-goettingen.de/downloads/trex2/

After installation, to apply the transformation of a TDL model into TTCN-3 code, open the TDL model (making sure it is selected) and press the `T3` icon on the toolbar. This shall result in the generation of a `.ttcn3` file with the same name in the same folder as the model.

## Known Issues

* The implementation of the OCL constraints validation may occasionally fail to identify all constraint violations or even display an error message. This is likely due to caching issues and should be resolved by restarting Eclipse. Possible solutions to this issue are being investigated.
* The implementation of the TTCN-3 transformation may occasionally fail, likely due to caching issues, and should be resolved by restarting Eclipse. Possible solutions to this issue are being investigated.
* The implementation of the TTCN-3 transformation is not 100% complete. While it provides a sufficient framework for constructing the main elements of a TTCN-3 specification, some elements are not yet fully supported (ProcedureCalls, some DataUses, some Behaviours). Please report any parts than are urgently needed so that future releases could provide support for them with a higher priority.
* The translation between the different representations (textual formats, XF) as well as using the different repersentations during the importing of data definitions in TDL may have some limitations. While using XF as a target format should work in practically all cases, using the legacy textual representations (`.tdlan2` and `.tplan2`) as targets in particular may fail due to some compatibility issues. The standardised TX textual representations should work in most cases and are the recommended target textual representation formats. However, the formatting may not be ideal in some cases. This may cause syntax errors in the indentation-based format in particular as it depends on correct formatting to determine the corresponding syntax structures. Please report any issues with corresponding examples.
* The importing of data definitions from OpenAPI and ASN.1 relies on third-party components, thus problems with processing the OpenAPI / ASN.1 specifications due to the third-party components will likely result in incomplete importing of the data definitions.
