ETSI's Bug Tracker - Part 01: TTCN-3 Core Language
View Issue Details

ID | Project | Category | View Status | Date Submitted | Last Update
0007999 | Part 01: TTCN-3 Core Language | Technical | public | 15-01-2021 10:20 | 01-12-2021 08:25

Reporter | Wolfgang Seka |
Assigned To | Jens Grabowski |
Priority | normal | Severity | major | Reproducibility | have not tried
Status | closed | Resolution | fixed |
Platform |  | OS |  | OS Version |
Product Version |  |
Target Version |  | Fixed in Version | 4.13.1 (ongoing) |
Clause Reference(s) | 15.3
Source (company - Author) | MCC TF160 - Wolfgang

Summary | 0007999: Evaluation of function calls in templates

Description

ES 201 873-1 clause 15.3 distinguishes template fields affected by parameterisation from (static) template fields not affected by parameterisation: "Both global and local templates are initialized at the place of their declaration. This means, all template fields which are not affected by parameterization shall receive a value or matching mechanism. Template fields affected by parameterization are initialized at the time of template use."
Use of pre-defined (built-in) functions (e.g. testcasename) or external functions does not seem to have been considered (yet).
Strictly applying clause 15.3 may cause non-deterministic behaviour as a function call from a template, not using any parameter of this template, may be treated as static and therefore done only once at the point in time when the static fields are evaluated. This point in time depends on the compiler implementation and on the test environment.
=> When e.g. using, in a send-template, an external function retrieving the system time, the used time value may not be the time when sending the message but a non-deterministic time somewhere in the past. This is surely not what a user wants or expects.
=> We would like to propose to clarify ES 201 873-1 clause 15.3 to explicitly state that template fields using function calls shall be treated like template fields which are affected by parameterization. Hereafter is an example text update to exemplify what we have in mind:
"Both global and local templates are initialized at the place of their declaration. This means, all template fields which are not affected by parameterization shall receive a value or matching mechanism. Template fields affected by parameterization or using function calls are initialized at the time of template use."

Steps To Reproduce |
Additional Information |
Tags | No tags attached.

Relationships

related to | 0008032 | closed | Jens Grabowski | BNF links and formatting

Attached Files

CR7999.docx (173,425) 05-02-2021 12:39
http://oldforge.etsi.org/mantis/file_download.php?file_id=3995&type=bug
CR7999-v2.docx (198,269) 07-02-2021 09:33
http://oldforge.etsi.org/mantis/file_download.php?file_id=3996&type=bug
CR7999-v3.docx (253,255) 09-09-2021 11:30
http://oldforge.etsi.org/mantis/file_download.php?file_id=4010&type=bug
CR7999-v4.docx (332,918) 08-11-2021 10:36
http://oldforge.etsi.org/mantis/file_download.php?file_id=4017&type=bug
CR7999-v5.docx (279,176) 08-11-2021 14:35
http://oldforge.etsi.org/mantis/file_download.php?file_id=4027&type=bug

Issue History
Date Modified | Username | Field | Change
15-01-2021 10:20 | Wolfgang Seka | New Issue |
20-01-2021 18:01 | Jacob Wieland - Spirent | Note Added: 0015901 |
21-01-2021 09:43 | Thilo Lauer | Note Added: 0015902 |
25-01-2021 11:01 | Wolfgang Seka | Note Added: 0015903 |
04-02-2021 09:51 | Jens Grabowski | Note Added: 0015904 |
05-02-2021 12:39 | Jacob Wieland - Spirent | File Added: CR7999.docx |
05-02-2021 12:40 | Jacob Wieland - Spirent | Note Added: 0015905 |
05-02-2021 12:40 | Jacob Wieland - Spirent | Assigned To | => Tomas Urban
05-02-2021 12:40 | Jacob Wieland - Spirent | Status | new => confirmed
07-02-2021 09:33 | Tomas Urban | File Added: CR7999-v2.docx |
07-02-2021 09:50 | Tomas Urban | Note Added: 0015906 |
07-02-2021 09:50 | Tomas Urban | Assigned To | Tomas Urban => Jacob Wieland - Spirent
09-02-2021 14:36 | Jacob Wieland - Spirent | Note Added: 0015907 |
10-02-2021 11:35 | Tomas Urban | Note Added: 0015908 |
11-02-2021 11:10 | Gyorgy Rethy | Note Added: 0015909 |
11-02-2021 11:10 | Gyorgy Rethy | Note Edited: 0015909 | bug_revision_view_page.php?bugnote_id=15909#r544
15-02-2021 10:28 | Jacob Wieland - Spirent | Note Added: 0015910 |
24-02-2021 13:54 | Thilo Lauer | Note Added: 0015911 |
25-02-2021 10:30 | Jacob Wieland - Spirent | Note Added: 0015912 |
23-03-2021 15:59 | Jens Grabowski | Project | TTCN-3 Change Requests => Part 01: TTCN-3 Core Language
09-09-2021 11:26 | Jacob Wieland - Spirent | File Added: CR7999-v3.docx |
09-09-2021 11:26 | Jacob Wieland - Spirent | File Deleted: CR7999-v3.docx |
09-09-2021 11:30 | Jacob Wieland - Spirent | File Added: CR7999-v3.docx |
09-09-2021 11:47 | Jacob Wieland - Spirent | Assigned To | Jacob Wieland - Spirent =>
09-09-2021 11:47 | Jacob Wieland - Spirent | Assigned To | => Jacob Wieland - Spirent
09-09-2021 11:47 | Jacob Wieland - Spirent | Status | confirmed => assigned
09-09-2021 11:48 | Jacob Wieland - Spirent | Note Added: 0015966 |
09-09-2021 11:48 | Jacob Wieland - Spirent | Assigned To | Jacob Wieland - Spirent => Tomas Urban
09-09-2021 11:48 | Jacob Wieland - Spirent | Status | assigned => confirmed
14-09-2021 11:39 | sekaw | Note Added: 0015992 |
08-11-2021 10:36 | Tomas Urban | File Added: CR7999-v4.docx |
08-11-2021 10:54 | Tomas Urban | Note Added: 0015994 |
08-11-2021 10:56 | Tomas Urban | Note Added: 0015995 |
08-11-2021 10:56 | Tomas Urban | Assigned To | Tomas Urban => Jacob Wieland - Spirent
08-11-2021 14:35 | Jacob Wieland - Spirent | File Added: CR7999-v5.docx |
08-11-2021 14:36 | Jacob Wieland - Spirent | Note Added: 0016005 |
08-11-2021 14:36 | Jacob Wieland - Spirent | Status | confirmed => resolved
08-11-2021 14:36 | Jacob Wieland - Spirent | Fixed in Version | => 4.13.1 (ongoing)
08-11-2021 14:36 | Jacob Wieland - Spirent | Resolution | open => fixed
08-11-2021 14:36 | Jacob Wieland - Spirent | Assigned To | Jacob Wieland - Spirent => Jens Grabowski
23-11-2021 11:17 | Jens Grabowski | Note Added: 0016092 |
23-11-2021 11:17 | Jens Grabowski | Status | resolved => closed
23-11-2021 15:28 | Jens Grabowski | Note Added: 0016099 |
23-11-2021 15:28 | Jens Grabowski | Status | closed => confirmed
30-11-2021 12:14 | Jens Grabowski | Assigned To | Jens Grabowski => Axel Rennoch
30-11-2021 12:14 | Jens Grabowski | Status | confirmed => assigned
01-12-2021 07:18 | Axel Rennoch | Relationship added | related to 0008032
01-12-2021 07:25 | Axel Rennoch | Note Added: 0016135 |
01-12-2021 07:25 | Axel Rennoch | Assigned To | Axel Rennoch => Jens Grabowski
01-12-2021 07:25 | Axel Rennoch | Status | assigned => confirmed
01-12-2021 08:25 | Jens Grabowski | Note Added: 0016140 |
01-12-2021 08:25 | Jens Grabowski | Status | confirmed => closed

Notes

(0015901)
Jacob Wieland - Spirent
20-01-2021 18:01

The intention of the standard so far is to say that it basically should not matter where a template body is evaluated, it should have the same value in the same context. This implies that all the fields whose right-hand-side is not dependent of a parameter should be initializeable at the point of declaration of the template, while, of course, the fields that are dependent on a template parameter can only be evaluated once the formal parameter is instantiated with an actual parameter at the point of usage of the template:
To illustrate this, consider the following example:
type record R { integer a, integer b }
function f(integer x) return integer { ... }
function g() {
var integer a := 3;
template R t := { a, f(a) };
a := 5;
if (match({3,f(3)}, t)) { ... // true with the standard until now
}
if (match({3,f(5)}, t)) { ... // true with the proposal, but unexpected/confusing
}
if (match({5,f(5)}, t)) { ... // expectation if templates are fully evaluated lazy
}
}
For this reason, this would be backward incompatible change.
If you want the template to consider the change of a, you should make a a parameter of the template:
template R t(integer x) := { x, f(x) }
and then use t(a) whereever you want { a, f(a) }

(0015902)
Thilo Lauer
21-01-2021 09:43

Additionally we would like to get a question clarified, that is related this discussion, but not covered by CR7999 Subject yet:
Question, regarding the direct use of variable a:
Using the modified function g() below, what is the expected behaviour?
function g() {
var integer a;
template R t := { a, f(a) }; // local template declaration: here a is undefined; CR7999 covers f(a), but not yet the direct use of variable a
a := 3;
...
a := 5;
p.send(t); // Question: is it a test case error (as a is not initialized at template declaration), is it { 3, f(3) } (first bound value to a is 3) or is it { 5, f(5) } (value bound to a when t is used for the first time) ?
...
}

(0015903)
Wolfgang Seka
25-01-2021 11:01

Some remarks:
a) First of all I'm confused about Jacob's note: In your first mail you've stated that the Spirent compiler behaves as we expect i.e. evaluates function when a template is used. Or is it different for global and local templates ??
b) Generally it seems that there are differences for global and local templates:
- It is pretty clear and visible for users when a local template gets initialised; from a user's point of view it is similar to a template variable.
- With literal interpretation of 15.3 the initialisation of a global template is unpredictable when it uses a function call like retrieving the system time. On the other hand the common expectation by users might be, that global templates behave like function calls (predictable and easy to understand).
c) Different compilers seem to use different interpretations regarding global templates => the TTCN-3 standard may not be clear enough and independent of outcome of this discussion there might always be a compatibility issue:
For MCC160 test suites it would be a severe issue when we would need to change our code to align it with changes of the compiler(s) (please note that we are using testcasename() in templates since at least ten years). And if so, the criteria for users are still not clear (as templates may use other templates using functions; in this case all templates are affected)
Conclusion:
We (MCC160) need a solution with minimum impact on our code. This would be served with the proposed clarification of 15.3. In addition global and local templates may need to be distinguished. As alternative some global directive may be introduced to specify whether or not global and/or local templates are treated as "fuzzy".

(0015904)
Jens Grabowski
04-02-2021 09:51

TTF discussion: TTF agrees that a solution is required. The TTF proposes to follow the idea of a global "fuzzy" directive (maybe as an attribute on groups or modules).

(0015905)
Jacob Wieland - Spirent
05-02-2021 12:40

uploaded draft proposal, please review

(0015906)
Tomas Urban
07-02-2021 09:50

I removed the reference to explicit lazy templates, because there's no such option in TTCN-3 at the moment. Static templates can be either fuzzy or without a modifier.
However, I think we still have to discuss two aspects of the added feature:
1. Is it necessary to change how local templates behave? Their lifetime is limited to the lifetime of their parent block and their place in the code determines quite precisely when they are initialized. I would prefer that only global templates would be affected.
2. In case we decide in favour of local templates, there's a question what to do with template variables and parameters (which are always local). Technically they are not affected by the rule as they are quite different entities. But it might not be so obvious without detailed knowledge of the specification, so I would prefer to add a note saying they are not affected by the rule.

(0015907)
Jacob Wieland - Spirent
09-02-2021 14:36

The initializer in a template variable declaration is a place of usage, so, of course, the value (unless lazy/fuzzy) is evaluated before the assignment to the variable, at the place of the declaration of the variable.
I don't see why it should be different for local and global templates. That will just add another source of confusion in my opinion. The semantics should not change if I move a local template into the global scope, if it was already independent from its context. With your proposal, suddenly the semantics could change from non-fuzzy to fuzzy.

(0015908)
Tomas Urban
10-02-2021 11:35

I admit that for the sake of consistency, it is better to have a rule that affects all static templates.
However, I would still like to have the note about template variables and paremeters.
We should also consider what happens, if how nesting works in case of groups. Does a group with a missing @fuzzy modifier turn off a @fuzzy setting of the surrounding module?
In my opinion, it would be better if a missing modifier on a group level meant that the setting is inherited. In order to allow overwriting, I propose to introduce another modifier (e.g. @static) that would mean the old "resolve as early as possible" mode and could be used for overwriting the mode set on a higher level (both on group level and in template declarations).
We should also consider adding @lazy modifier support to static templates, modules and groups. I dont'n see a reason, why it should be missing.

(0015909)
Gyorgy Rethy
11-02-2021 11:10

Hi,
In general I disagree with the module and group @fuzzy solution for two reasons:
- that would complicate user's life even more (and what is worth, decrease their productivity, that managers are looking at when choosing a tool/language for a new project...); a typical workflow is: user is writing/debugging a function, which uses a template. In Eclipse ctrl+F3 -> jump to definition, looks at the template; it will see the template ALONE but not it's context in term of group/module; this 'feature' could just increase users' frustration, while not really giving too much to him/her.
- the second is Jacob's point: a definition's behavior shall not change just by moving it out of a group, into another group, other module etc. In the course of normal development re-structuring the code happens quite open in the way from a prototype to a more and more feature-rich 'product'. This could easily lead to chaos in case of such restructurings.
On Thilo's question: TTCN-3 code is executed statement-by-statement. As template t is declared completely, i.e. not parameterized, it is evaluated at the place of its declaration, and as 'a' is unbound, and used as a value of a field, the send operation shall cause an error as trying to send a not completely initialized template.
In summary: as a general rule, the standard should state, that the template shall be evaluated at the place where it gets completely defined. This would be a simple, easy to remember rule and remove ambiguities.

(0015910)
Jacob Wieland - Spirent
15-02-2021 10:28

I agree with Georgys assessment completely and I always thought that this was the intended way of the standard. Unfortunately, this clarification seems to cause trouble for people because some tools misinterpreted the standard (or did not read it as it was intended) and this led to problems with certain code when used by different tools that interepreted it more like the standard intended.
This will then lead to the old code having to be rewritten. In general, I am confused about the whole situation as we informed people that were using our tools some years ago what the problem with their code was (when the problem first cropped up after testcasename was introduced) and they obviously did not heed our advice and instead chose to use other tools which now cause this problem.

(0015911)
Thilo Lauer
24-02-2021 13:54

In preparation of tomorrow's GoToMeeting:
-----------------------------------------
To get back to FACTS and towards a COMMON understanding of the TTCN-3 standard
let's have a look at this enhanced example and questions Q1 .. Q4:
type record R { integer a, integer b, integer c }
function f(integer x) return integer { ... }
function g() {
var integer a, c:=2;
// local template declarations:
template R t := { a, f(a), c}; // field 'a' is undefined; field 'c' is NOT parameterized
template R u(integer k) := {k * a, k* f(a), c}; // field 'a' is undefined, param 'k' has NO default value, field 'c' is NOT parameterized
template R v := {8, f(9), c}; // all fields are defined at template declaration time
template R w(integer m := 1) := {m * 8, m * f(9), m * c}; // param 'm' has default value, all fields are parameterized
a := 3;
...
a := 5;
c := 10;
// Question 1:
p.send(t);
// Q1a: ==> ERROR as 'a' is not initialized at template declaration or
// Q1b: ==> { 3, f(3), 2} (first bound value to 'a' is 3 and to 'c' is 2)
// Q1c: ==> { 5, f(5), 2} (value bound to 'a' when 't' is used for the first time)
// Q1d: ==> { 5, f(5), 10} (completely evaluated at time of use)
// Question 2:
p.send(u(1)); // note: p should be a LOOPBACK port (!)
p.receive(t); // should it match?, see Question 1 above
// Q2a: p.send(u(1)); ==> { 1*5, 1*f(5), 2} ==> { 5, f(5), 2} (only parameterized fields are evaluated, 'c' is 2 at decl. time)
// Q2b: p.send(u(1)); ==> { 1*5, 1*f(5), 10} ==> { 5, f(5), 10} (completely evaluated at time of use)
// Question 3: How to explain possible differences to TTCN-3 users / test suite maintainers?
p.send(v()); // Q3a: ==> { 8, f(9), 2} (completely assigned at decl. time; although c==10 when template is used)
p.send(w); // Q3b: ==> { 1*8, 1*f(9), 1*2} ==> { 8, f(9), 2 } (completely assigned at decl. time; although DECLARATION is parameterized)
// Q3c: ==> { 1*8, 1*f(9), 1*10} ==> { 8, f(9), 10 } (completely evaluated at time of use)
p.send(w(1)); // Q3d: ==> { 1*8, 1*f(9), 1*10} ==> { 8, f(9), 10 } (all fields are parameterized and therefore evaluated at time of use)
// Q4: Generally asking from a TTCN-3 user's perspective:
// ======================================================
// How to remember easily the differences between u(1) and t AND between v, w and w(1) ???
// Why do we need different handling rules for parameterized and non-parameterized templates?
}

(0015912)
Jacob Wieland - Spirent
25-02-2021 10:30

Discussion with STF 160: We propose to introduce a template-field modifier @fuzzy into the standard that signifies that the evaluation of the right-hand-side of the field is deferred until the place of usage of the field (where modification of the template or instantiation of the template inside another template does not count as 'place of usage').
Such a fuzzy field shall only be evaluated inside a template-structure when the whole template-structure is evaluated for use (i.e. for sending, for isvalue, for matching or other places that require templates to be evaluated like actual parameters).
Here a few examples to my current understanding of the new feature:
template R base := { @fuzzy name := testcasename() }
template R modifiedR modifies base := { a := 5 } // name-field is still fuzzy
template R modifiedR2 modifies base := { name := "foobar" } // name-field not fuzzy
When a fuzzy field is used in some non-fuzzy evaluation, it needs to be evaluated.
template R modifiedR3 modifies base := { name := base.name & "_used" } // base.name is evaluated, so this might lead to global evaluation of the name field
template R modifiedR4 modifies base := { @fuzzy name := base.name & "_used" } // here, base.name would only be evaluated when modifiedR4.name is evaluated (fuzzy field assignment overridden by new fuzzy assignment)

(0015966)
Jacob Wieland - Spirent
09-09-2021 11:48

Please review

(0015992)
sekaw
14-09-2021 11:39

A minor issue:
The new text says "... shall only be evaluated when ...".
So, I wonder in which cases the assignment shall not be evaluated when it would be evaluated without the "@fuzzy" modifier.
According to my understanding the "@fuzzy" modifier should force the evaluation to be done whenever the template is used - or do I miss anything ??
Furthermore the examples modifiedR3 and modifiedR4 are hard to understand (at least for me):
Assuming that testcasename() returns "" at the time the templates are created (as no testcase is running yet), in modifiedR3 base.n is still "" and modifiedR3.n will be " used" where ever used, but modifiedR4.n is "<test case> used" with the name of the test case in which modifiedR4 is used.
Is this correct ??
And another question:
Even though in general we are not using local templates (e.g. declared in a function) - is there any difference ??
Is the local template of a function without any parameters and without "@fuzzy" modifier instantiated (and therefore evaluated) whenever the function is called or only once at the very beginning of a test campaign ??.
(in the first case modifiedR3 and modifiedR4 would be effectively the same).

(0015994)
Tomas Urban
08-11-2021 10:54

Regarding Wolfgang's questions:
1. Basically you are correct. However, templates with the @fuzzy modifier can be used in lazy/fuzzy context which doesn't trigger evaluation. For that reason we cannot just write "whenever the template is used", but list either the cases that trigger evaluation or the exception. We chose the fist option.
2. Correct. However, there are diffirences between various tools, because the original version of the TTCN-3 core language specification didn't precisely when this evaluation should take place. Unfortunately it is impossible to overcome this issue at this point as setting a hard limit would make some of the tools incompatible with the standard.
3. There are no major functional differences, both fuzzy and local templates will be evaluated inside test cases (when called). You could also use templates decladed inside component types.

(0015995)
Tomas Urban
08-11-2021 10:56

I moved the rules concerning modifications to 15.5 and made a couple of minor adjustments. Please check.

(0016005)
Jacob Wieland - Spirent
08-11-2021 14:36

Fixed one typo (modifier -> modified) but otherwise ok.

(0016092)
Jens Grabowski
23-11-2021 11:17

implemented

(0016099)
Jens Grabowski
23-11-2021 15:28

BNF needs to be checked

(0016135)
Axel Rennoch
01-12-2021 07:25

BNF changes in CR 7999 have been implemented in CR0008031+32_Core-Language-without-CR7910-r1.docx

(0016140)
Jens Grabowski
01-12-2021 08:25

Implemented