Constraint test results.

# Constraint suite: what it reports on the current implementation (2026-09-15)

The fixtures state what the standard's OCL requires. They are not adjusted to what the IDE currently

does, so a red test is a finding rather than a broken test. This file records what is red today,

measured over repeated runs of

    mvn verify -Dtest=ConstraintSuiteTest -DfailIfNoTests=false

on branch `java_validation` (dev plus this suite), against the interpreted Complete OCL validator.

> **A correction to a constraint is a change to the standard.** The `.ocl` documents in

> `org.etsi.mts.tdl.model/model` are generated from the specification document by `ConstraintExporter`,

> and so are the operation bodies in `tdl.ecore`. A fix that lives only in the repository is reverted

> by the next export. Every correction made here is listed in

> `org.etsi.mts.tdl.model/model/STANDARD-CHANGES.md` for carrying into the document.

## Coverage

| | |

|---|---|

| Constraints in the standard's three OCL documents | 172 (both reduction pairs merged into one constraint each, MemberReference::CollectionIndex removed as redundant) |

| Covered by at least one violating fixture | 137 |

| Not expressible in the textual syntax, or unable to fail by construction | 35 (see `EXCEPTIONS.md`) |

| Open | 0 |

| Fixture files | 285 (violating cases plus conformant variants for branch coverage) |

## Only one of the three OCL documents was registered headlessly

`TDLtxAbstractTest` registers `tdl-constraints.ocl` and nothing else, while the product registers all

three from the model plugin's activator. Every constraint of the structured objectives and extended

configurations documents therefore passed silently in JUnit while the same file was correctly flagged

in the editor. `ConstraintSuiteTest` now registers the other two documents itself. Anyone writing

headless tests against `dev` should know this: without those two registrations, two thirds of the

constraint documents are simply not evaluated, and a green test means nothing was checked.

## Fixed: permitted components, and two constraints the fix exposed

`MultipleCombinedBehaviour::getPermittedComponents()` read

    self.block.behaviour->first()

which flattens the behaviours of every block into one collection and then takes a single element, so

only the first block's first behaviour was ever considered. Every block contributes its own first

behaviour, so it is now

    self.block->collect(b | b.behaviour->first())

in `org.etsi.mts.tdl.model/model/tdl.ecore`, three times in the same body. With that,

`AlternativeBehaviour::AlternativeBlocksComponent` reports the alternative whose blocks start on two

different testers, and the constraints that share the operation (alternative, optional and

exceptional participation) keep reporting as before.

`ExceptionalBehaviour` did not define the operation at all and inherited `Behaviour`'s empty set, so

`includesAll(empty)` was trivially true and `ExceptionalGuardedandTargetComponent` could never fail.

It now has its own body, the single-block form of the one above. The constraint left the exceptions

list and has a violating fixture and two conformant variants.

Giving that operation a real value exposed two constraints that had been passing only because it was

empty, both corrected in `model/tdl-constraints.ocl`:

- **`ExceptionalBehaviour::ExceptionalGuardedandTargetComponent`** fired on every exceptional behaviour

  that has no guarded component, although its text begins "if ... guardedComponent is specified". It

  now starts with `self.guardedComponent.oclIsUndefined() or`.

- **`AlternativeBehaviour::AlternativeBehaviourParticipation`** and

  **`ExceptionalBehaviour::ExceptionalBehaviourParticipation`** counted the components participating

  inside contained optional behaviours, although both messages end "and 'ComponentInstance's

  participating in blocks of contained 'OptionalBehaviour's". Both now collect participants from

  `nonOptionalBlocks.behaviour->reject(oclIsKindOf(OptionalBehaviour))`.

Coverage rose to 137 constraints with a violating fixture and 38 exclusions.

## Reproducible failures: none

The three that were reproducible have been corrected in the constraints and in the metamodel

operations, each recorded in `STANDARD-CHANGES.md`: permitted components of a combined behaviour, the

guard and the participation rules that depended on them, the parameter-type operation, and the

applicability guard on the collection item-type constraint. What each of them had in common is worth

keeping in mind while writing the Java implementation:

**When a constraint cannot be evaluated on an element, the constraints evaluated after it on that

same element are not evaluated either, and the reports already made for that element are sometimes

discarded as well.** One expression that casts without testing therefore silences its neighbours, and

the file shows nothing rather than the violation the user could act on. Every case found so far was

the same shape, a downcast to `CollectionDataType` or `StructuredDataType` on a value whose type a

companion constraint is meant to establish.

## Nondeterministic failures: the two structured visibility constraints

`LiteralValueReference::VisibleValue` and `ContentReference::VisibleContent`, each with a violating

fixture and a conformant variant, give different verdicts from run to run. In one run the violating

fixture is not reported; in the next the conformant variant is reported instead; in a third both

behave correctly. Nothing in the fixtures changes between runs.

`ParameterBinding::ReductionMembers` was intermittent until two things changed. The merged expression

used a null marker for an inapplicable step and now compares chain lengths instead; and its fixture

bound `?` in place of a literal, so that `ParameterBindingTypes` on the same element no longer has a

type to resolve and can no longer yield invalid and stop the constraint under test. It has been stable

since, and it no longer needs an `allow` for its neighbour. Two lessons from that episode are recorded

in `STANDARD-CHANGES.md`: Eclipse OCL does not treat a null inside a typed collection consistently

across `excludes`, `forAll` and `select`, and an `iterate` accumulator whose type is not declared makes

the constraint silently never report. The third is now rule 7 of `AUTHORING.md`: keep everything around

the constraint under test undemanding, or a neighbour will decide the verdict.

Two attempts to remove the variation did not:

- giving every fixture its own resource set, which the suite now does anyway, and

- resetting the OCL executor before each fixture, which made a run twelve times slower (166 s instead

  of 13 s) and still produced different results from run to run.

What remains is the interpreted validator's own state: one validator instance shared process-wide, an

OCL environment captured from the first object it ever validates, and a thread-local executor whose

extent depends on what has been loaded. A verdict therefore depends on which files were validated

before it, in the same process. This is what the generated Java constraints remove by construction,

and the remaining fixtures are the acceptance test for that: they must become stable.

## What the suite is for from here

Two things. It is the regression baseline for the transition to generated Java constraints: the same

fixtures, run through `IResourceValidator`, must give the same verdicts once the implementation

changes, and the nondeterministic rows below must become stable. And it is the evidence that the

current solution is fragile, in a form that can be re-run rather than argued: a constraint that

cannot be evaluated silences its neighbours on the same element and sometimes discards their reports,

and the same fixture can come out violating in one run and conformant in the next.

Phase status: the suite covers every constraint of the standard, has no reproducible failures left,

and the corrections it produced are recorded in `../../../org.etsi.mts.tdl.model/model/STANDARD-CHANGES.md`

with their state in the specification document.

## How to use this file

Fix what is functional, leave what is framework-induced. When a reproducible failure is corrected in

the standard's OCL or in a model operation, the corresponding fixture turns green on its own and its

row here should go. The nondeterministic rows should disappear together, when the constraints stop

being interpreted.

## Reading a failure message

    Fixture.tdltx: expected Ctx::Name, reported 0 time(s)

      all reported: [] ==> expected: <true> but was: <false>

The head names the fixture and what went wrong; `all reported` lists every issue the validator

produced for it, empty meaning none at all; the trailing `expected: <true> but was: <false>` is JUnit

rendering the assertion and carries no information. Four problems can appear in the head:

- `expected X::Y, reported 0 time(s)` — the declared violation was not reported, or not the declared

  number of times.

- `unexpected X::Y xN` — a constraint fired that the header does not declare. On an `expect none`

  fixture that is a false positive; elsewhere the fixture probably violates more than its target.

- `X::Y could not be evaluated on this model` — its OCL yielded invalid, and the constraints after it

  on the same element were skipped.

- `issues that are not constraints (broken fixture?)` — syntax or linking errors: the fixture itself

  is wrong, fix it before reading anything else into the result.