ETSI's Bug Tracker - Part 01: TTCN-3 Core Language
View Issue Details

ID | Project | Category | View Status | Date Submitted | Last Update
0005513 | Part 01: TTCN-3 Core Language | Clarification | public | 14-04-2010 15:24 | 15-12-2010 00:41

Reporter | Jacob Wieland - Spirent |
Assigned To | Ina Schieferdecker |
Priority | high | Severity | major | Reproducibility | have not tried
Status | closed | Resolution | fixed |
Platform |  | OS |  | OS Version |
Product Version |  |
Target Version | v4.3.1 (published 2011-06) | Fixed in Version | v4.3.1 (published 2011-06) |
Clause Reference(s) | 7.1.2
Source (company - Author) | Testing Technologies - Jacob Wieland

Summary | 0005513: resolution of CR5092 contains bogus examples for charstring

Description

In the CR 5092, there are several misleading (or WRONG?) examples concerning concatenation of charstring templates. While mapping * length(N) to a repetition of N ? makes sense in the context of octetstring, hexstring and bitstring types (because there ? has a special meaning already), it does NOT make sense in the context of charstring, as there the character ? is just another character with no special meaning. The only context where that is the case are charstring patterns.
Thus, either the examples are wrong for charstring or it should be explained that concatenating a normal charstring value (template) with a length-template should result in a charstring pattern template (if that is the intention), i.e. all special characters in the charstring value template need to be quoted.
I.e. '"a*?" & * length(2)' would result in a template equivalent to
the charstring template expression 'pattern "a\*\???"'
Interpreting this as the template "a*???" (without pattern) would not make much sense as that would only match the value "a*???" which is totally different from the meaning in the other string types and thus would be totally confusing for the user.

Steps To Reproduce |
Additional Information | This should actually be a note added to CR5092, but since I was told not to reopen closed CRs, I'm submitting a new one.
Tags | No tags attached.

Relationships

related to | 0005092 | closed | Gyorgy Rethy | String concatination is not possible for string templates
related to | 0005809 | closed | Ina Schieferdecker | CL chapter 15.11 Concatenating templates of string and list types is not supported by BNF
related to | 0005828 | closed | Ina Schieferdecker | STF409 comment on [Part 1: TTCN-3 Core Language /Annex B.1.5.2 ]: syntactical ambiguity

Attached Files

CR5513_resolution v2.doc (86,528) 30-06-2010 15:52
http://oldforge.etsi.org/mantis/file_download.php?file_id=2388&type=bug
CR5513_resolution v3.doc (93,184) 01-07-2010 18:01
http://oldforge.etsi.org/mantis/file_download.php?file_id=2399&type=bug
CR5513_resolution v4.doc (91,648) 02-07-2010 13:44
http://oldforge.etsi.org/mantis/file_download.php?file_id=2402&type=bug
CR5513_resolution v5.doc (90,112) 13-07-2010 11:58
http://oldforge.etsi.org/mantis/file_download.php?file_id=2409&type=bug
CR5513_resolution v6.doc (90,112) 14-07-2010 11:47
http://oldforge.etsi.org/mantis/file_download.php?file_id=2410&type=bug
CR5513_resolution v7.doc (91,648) 22-07-2010 12:58
http://oldforge.etsi.org/mantis/file_download.php?file_id=2412&type=bug
CR5513_resolution v8.doc (91,136) 01-12-2010 10:10
http://oldforge.etsi.org/mantis/file_download.php?file_id=2448&type=bug
CR5513_resolution_v9.doc (91,648) 01-12-2010 10:10
http://oldforge.etsi.org/mantis/file_download.php?file_id=2449&type=bug

Issue History
Date Modified | Username | Field | Change
14-04-2010 15:24 | Jacob Wieland - Spirent | New Issue |
14-04-2010 15:24 | Jacob Wieland - Spirent | Clause Reference(s) | => 7.1.2
14-04-2010 15:24 | Jacob Wieland - Spirent | Source (company - Author) | => Testing Technologies - Jacob Wieland
14-04-2010 15:26 | Jacob Wieland - Spirent | Relationship added | related to 0005092
28-06-2010 16:46 | Gyorgy Rethy | Status | new => assigned
28-06-2010 16:46 | Gyorgy Rethy | Assigned To | => Gyorgy Rethy
29-06-2010 10:52 | Gyorgy Rethy | Project | TTCN-3 Change Requests => Part 01: TTCN-3 Core Language
29-06-2010 10:54 | Gyorgy Rethy | Note Added: 0009399 |
29-06-2010 10:54 | Gyorgy Rethy | Target Version | => Edition 4.3.1 (not yet published)
29-06-2010 10:55 | Gyorgy Rethy | Note Edited: 0009399 |
29-06-2010 12:05 | Jacob Wieland - Spirent | Note Added: 0009405 |
30-06-2010 08:33 | Gyorgy Rethy | File Added: CR5553_resolution.doc |
30-06-2010 08:34 | Gyorgy Rethy | Note Added: 0009420 |
30-06-2010 10:27 | Jacob Wieland - Spirent | Note Added: 0009423 |
30-06-2010 13:11 | Jacob Wieland - Spirent | Note Added: 0009430 |
30-06-2010 15:50 | Gyorgy Rethy | Note Added: 0009434 |
30-06-2010 15:51 | Gyorgy Rethy | File Deleted: CR5553_resolution.doc |
30-06-2010 15:52 | Gyorgy Rethy | File Added: CR5513_resolution v2.doc |
30-06-2010 15:52 | Gyorgy Rethy | Assigned To | Gyorgy Rethy => Jacob Wieland - Spirent
30-06-2010 15:53 | Gyorgy Rethy | Note Edited: 0009434 |
30-06-2010 17:07 | Jacob Wieland - Spirent | Note Added: 0009436 |
01-07-2010 16:17 | Gyorgy Rethy | Note Added: 0009456 |
01-07-2010 16:18 | Gyorgy Rethy | Note Edited: 0009456 |
01-07-2010 18:01 | Gyorgy Rethy | File Added: CR5513_resolution v3.doc |
01-07-2010 18:03 | Gyorgy Rethy | Note Added: 0009459 |
02-07-2010 09:45 | Gyorgy Rethy | Assigned To | Jacob Wieland - Spirent => Ina Schieferdecker
02-07-2010 09:46 | Gyorgy Rethy | Note Added: 0009460 |
02-07-2010 11:01 | Ina Schieferdecker | Note Added: 0009466 |
02-07-2010 11:02 | Ina Schieferdecker | Assigned To | Ina Schieferdecker => Benjamin Zeiss
02-07-2010 11:55 | Jacob Wieland - Spirent | Note Added: 0009470 |
02-07-2010 12:40 | Gyorgy Rethy | Note Added: 0009472 |
02-07-2010 12:44 | Jacob Wieland - Spirent | Note Added: 0009473 |
02-07-2010 13:22 | Gyorgy Rethy | Note Edited: 0009472 |
02-07-2010 13:37 | Gyorgy Rethy | Note Edited: 0009472 |
02-07-2010 13:38 | Gyorgy Rethy | Note Edited: 0009472 |
02-07-2010 13:44 | Gyorgy Rethy | File Added: CR5513_resolution v4.doc |
02-07-2010 13:48 | Gyorgy Rethy | Note Added: 0009474 |
02-07-2010 14:03 | Jacob Wieland - Spirent | Note Added: 0009477 |
08-07-2010 15:14 | Benjamin Zeiss | Note Added: 0009483 |
08-07-2010 15:14 | Benjamin Zeiss | Assigned To | Benjamin Zeiss => Ina Schieferdecker
13-07-2010 11:58 | Gyorgy Rethy | Note Added: 0009520 |
13-07-2010 11:58 | Gyorgy Rethy | File Added: CR5513_resolution v5.doc |
13-07-2010 17:03 | Gyorgy Rethy | Note Edited: 0009520 |
14-07-2010 10:30 | Jacob Wieland - Spirent | Note Added: 0009522 |
14-07-2010 11:47 | Gyorgy Rethy | File Added: CR5513_resolution v6.doc |
14-07-2010 11:50 | Gyorgy Rethy | Note Added: 0009523 |
14-07-2010 11:50 | Gyorgy Rethy | Status | assigned => resolved
14-07-2010 11:50 | Gyorgy Rethy | Resolution | open => fixed
22-07-2010 12:58 | Ina Schieferdecker | File Added: CR5513_resolution v7.doc |
22-07-2010 13:04 | Ina Schieferdecker | Note Added: 0009560 |
22-07-2010 13:05 | Ina Schieferdecker | Status | resolved => closed
22-07-2010 13:05 | Ina Schieferdecker | Fixed in Version | => Edition 4.2.2 (not yet published)
29-11-2010 12:57 | Jacob Wieland - Spirent | Status | closed => feedback
29-11-2010 12:57 | Jacob Wieland - Spirent | Resolution | fixed => reopened
29-11-2010 12:57 | Jacob Wieland - Spirent | Note Added: 0009826 |
29-11-2010 13:04 | Jacob Wieland - Spirent | Relationship added | related to 0005808
29-11-2010 13:05 | Jacob Wieland - Spirent | Relationship deleted | related to 0005808
29-11-2010 13:08 | Jacob Wieland - Spirent | Relationship added | related to 0005809
29-11-2010 13:49 | Jacob Wieland - Spirent | Note Added: 0009829 |
29-11-2010 15:54 | Jacob Wieland - Spirent | Relationship added | related to 0005828
29-11-2010 16:04 | Jacob Wieland - Spirent | Note Edited: 0009829 |
30-11-2010 11:20 | Ina Schieferdecker | File Added: CR5513_resolution v8.doc |
30-11-2010 11:21 | Ina Schieferdecker | Note Added: 0009840 |
30-11-2010 11:22 | Ina Schieferdecker | Status | feedback => assigned
30-11-2010 11:22 | Ina Schieferdecker | Assigned To | Ina Schieferdecker => Jacob Wieland - Spirent
30-11-2010 11:22 | Ina Schieferdecker | Assigned To | Jacob Wieland - Spirent => Gyorgy Rethy
01-12-2010 09:16 | Gyorgy Rethy | Note Added: 0009872 |
01-12-2010 09:16 | Gyorgy Rethy | File Added: CR5513_resolution_v9.doc |
01-12-2010 09:17 | Gyorgy Rethy | Assigned To | Gyorgy Rethy => Ina Schieferdecker
01-12-2010 09:17 | Gyorgy Rethy | Status | assigned => resolved
01-12-2010 09:17 | Gyorgy Rethy | Resolution | reopened => fixed
01-12-2010 09:17 | Gyorgy Rethy | Fixed in Version | Edition 4.2.2 (not yet published) => Edition 4.3.1 (not yet published)
01-12-2010 10:09 | Ina Schieferdecker | File Deleted: CR5513_resolution v8.doc |
01-12-2010 10:10 | Ina Schieferdecker | File Deleted: CR5513_resolution_v9.doc |
01-12-2010 10:10 | Ina Schieferdecker | File Added: CR5513_resolution v8.doc |
01-12-2010 10:10 | Ina Schieferdecker | File Added: CR5513_resolution_v9.doc |
01-12-2010 10:11 | Ina Schieferdecker | Note Added: 0009878 |
01-12-2010 15:45 | Ina Schieferdecker | Status | resolved => closed
15-12-2010 00:41 | Ina Schieferdecker | Note Added: 0009978 |

Notes

(0009399)
Gyorgy Rethy
29-06-2010 10:54 (edited on: 29-06-2010 10:55)

Agreed in principle, examples are incorrect, concatenating * length(2) to specific charstring values shall be disallowed; create proposed text. Target: interim version!

(0009405)
Jacob Wieland - Spirent
29-06-2010 12:05

In principle, I don't think it should be disallowed to concatenate different charstring templates to get a new template in general.
I.e. if t1 and t2 are valid charstring template expressions, then t1 & t2 should match a charstring value v, if there exists a concatenation v1 & v2 == v so that t1 matches v1 and t2 matches v2. The same extension can be added to the other string types, as well.
But, for the interim version, I think that forbidding concatenation on charstring templates is the right way to go.

(0009420)
Gyorgy Rethy
30-06-2010 08:34

Proposed resolution is uploaded in CR5513_resolution.doc.

(0009423)
Jacob Wieland - Spirent
30-06-2010 10:27

I have a few other (major) criticisms to this proposal:
1) The BNF does not allow expressions like A & * length(N) because length(N) is only derivable from ExtraMatchingAttributes, which is at the end of TemplateBody which cannot be derived from MulExpression (which is both the left and the right side of AddExpression where & is the operator.
=> Thus, the Examples have to be changed putting () around the * length(N) part.
(I don' think that the grammar should be changed to accommodate the present examples as that would open up a major can of worms)
2) I think that * length(N) in concatenations should rather be ? length(N)
This is because ? and ? length(N) are already valid string matching mechanisms while * is only a valid matching mechanism for an optional field, also matching omit. Since in a concatenation, we are not dealing with fields and we are not really inside values (where * has a different semantics), using * (while not being totally wrong no omit can occur anyway) is misleading. Also, the ? length(N) is forbidden explicitly by the text which I think is totally wrong.

(0009430)
Jacob Wieland - Spirent
30-06-2010 13:11

I think the attached file is supposed to be in a different CR

(0009434)
Gyorgy Rethy
30-06-2010 15:50 (edited on: 30-06-2010 15:53)

Good points. See updated proposal in CR5513_resolution v2.doc.
I have added the BNF and allowed both ? length (N) and * length (N). I see no reason to restrict the syntax only for one of them, though they mean the same. Clause B.1.4.1 allows length for both, to be more restrictive here would be seen by the user as further exception rule. Also more convenient for the user when e.g. changing already existing * to something concatenated.
Pls. review.

(0009436)
Jacob Wieland - Spirent
30-06-2010 17:07

Sorry, but the changes to the BNF do not work. As far as I can see, the concatenated template cannot begin with ? length(N), i.e. there is no derivation for ? length(N) & 'aa'O Why should this be?

(0009456)
Gyorgy Rethy
01-07-2010 16:17 (edited on: 01-07-2010 16:18)

Need not, just the BNF proposal is not complete enough. I will update.

(0009459)
Gyorgy Rethy
01-07-2010 18:03

Pls. find the updated BNF proposal in CR5513_resolution v3.doc. To identify the additions, they are marked with yellow background as the BNF itself contains blue underlined font.

(0009460)
Gyorgy Rethy
02-07-2010 09:46

Ina, pls. cross-check!

(0009466)
Ina Schieferdecker
02-07-2010 11:01

Contentwise it looks ok - Benjamin, could you however please check that there is no reduce conflict.

(0009470)
Jacob Wieland - Spirent
02-07-2010 11:55

This proposal is better, but it has a different problem.
The grammar now is ambiguous because SimpleSpec can be followed by ExtraMatchingAttributes which also can derive to LengthMatch,
so, in TemplateBody expressions like
'aa'O & ? length(10)
it needs to be clarified if the length(10) is the LengthMatch of MatchingSymbol or TemplateBody.
I think it should be the former, but this needs to be clarified in the static semantics, i.e. by enforcing the LengthMatch for AnyValue and AnyValueOrOmit in concatenations. Thereby, it is clear that the length directly after ? must be part of the MatchingSymbol.
Thus, I propose the following:
/* STATIC SEMANTIC â€“ LengthMatch must be used when MatchingSymbol is used in fractions of a concatenated string, otherwise it shall only be used inside list templates (see clause 15.11). In both cases, the Complement, ValueOrAttribList, Range, BitStringMatch, HexStringMatch, OctetStringMatch, CharStringMatch, SubsetMatch and SupersetMatch productions shall not be used.
*/

(0009472)
Gyorgy Rethy
02-07-2010 12:40 (edited on: 02-07-2010 13:38)

Within simple list templates the length from the path TemplateBody -> ArrayValueOrAttrib -> ArrayElementSpecList -> ArrayElementSpec -> TemplateBody is used, thus this newly added length shall be used with concatenation ONLY. But I agree that in case of concatenation length shall be mandatory.

(0009473)
Jacob Wieland - Spirent
02-07-2010 12:44

All the better, then change the text to: /* STATIC SEMANTIC â€“ LengthMatch must be used when MatchingSymbol is used in fractions of a concatenated string. In that case, the Complement, ValueOrAttribList, Range, BitStringMatch, HexStringMatch, OctetStringMatch, CharStringMatch, SubsetMatch and SupersetMatch productions shall not be used. */

(0009474)
Gyorgy Rethy
02-07-2010 13:48

CR5513_resolution v4.doc: only updates in BNF. One more static rule is added to TemplateBody to exclude e.g. '01'B & ? length(2) length (3).

(0009477)
Jacob Wieland - Spirent
02-07-2010 14:03

Why, why, why???
The overall length restriction also makes sense in situations where there are * wildcards inside the string-templates:
'aa*bb'O & ? length(1 .. 10) length(3 .. 40)
So, I don't think that the restriction is necessary, move to strike.
With the static semantic at the MatchingSymbol, the grammar is unambiguous in case that only one length() is provided (if it ends with ?/* length, then it is the inner length, if it ends with a different expression, it is the outer length) and it is also unambiguous if it ends with ?/* length() length()

(0009483)
Benjamin Zeiss
08-07-2010 15:14

Looking for shift-reduce and reduce-reduce conflicts by hand is obviously not an easy task ;-) Anyhow, here is what I tried in my limited time since last week:
I took the 4.2.1 grammar and converted the EBNF model to a BNF model. From that, I wrote a template that converted the grammar to an input grammar of the LALR GOLD parser which is a pretty nice tool to debug and browse LALR states. Due to the way I converted the grammar, there were already numerous conflicts (but that exist mostly due to the transformation). When ignoring them and incorporating the changes from this CR, I was able to rewrite both EBNF changes to BNF so that they did not produce any new conflicts. So from that, I more or less conclude that the changes should be reasonably safe. I will definately fiddle around with this a little more to see whether I can resolve the problems with the transformation conflicts or at least a number of them. Unfortunately, my time to work on this in this week has run out and Gyorgy asked me to reassign the CR to Ina in this week when I'm done. So this is what I'm doing now.
It would probably be worthwhile to go through the sr/rr conflicts again when I have the grammar conversion in a state that doesn't produce as many initial conflicts.

(0009520)
Gyorgy Rethy
13-07-2010 11:58 (edited on: 13-07-2010 17:03)

"Why, why, why??? The overall length restriction also makes sense in situations where there are * wildcards inside the string-templates: 'aa*bb'O & ? length(1 .. 10) length(3 .. 40)" It may look a little bit confusing for the user but I admit, this does not seem to be sufficient to forbid this. So, it is OK to remove the new static rule from Templatebody. Done in CR5513_resolution v5.doc.

(0009522)
Jacob Wieland - Spirent
14-07-2010 10:30

The only thing wrong in v5 is the following example: template RecofInt t_MyRecofInt := { 1, 2 } & { ? length(2), 3, 4 }; // results the template {1, 2, ?, ?, 3, 4 } This should be: template RecofInt t_MyRecofInt := { 1, 2 } & ? length(2) & { 3, 4 }; // results the template {1, 2, ?, ?, 3, 4 } Otherwise, it is ok.

(0009523)
Gyorgy Rethy
14-07-2010 11:50

This is a typo in the example, it was changed unintentionally, the original intention was: template RecofInt t_MyRecofInt := { 1, 2 } & { * length(2), 3, 4 }; // results the template {1, 2, ?, ?, 3, 4 } It is corrected in CR5513_resolution v6.doc.

(0009560)
Ina Schieferdecker
22-07-2010 13:04

small editorials, also in examples

(0009826)
Jacob Wieland - Spirent
29-11-2010 12:57

The LengthMatch for Any and AnyOrOmit in the rule for MatchingSymbol in the BNF should be restricted to contain no optional upper bound (to be consistent with the text).

(0009829)
Jacob Wieland - Spirent
29-11-2010 13:49 (edited on: 29-11-2010 16:04)

There is an ambiguity problem with the proposal:
If SingleExpression derives "X & Y", but also only "X" or "Y" and
SingleValueOrAttrib derives SingleExpression, we have two ways
of deriving "X & Y" from SimpleSpec.
Also, there is another ambiguity with CharStringMatch which can also contain "&".
Therefore, we should change the grammar in the following manner:
SimpleSpec ::= SingleExpression ["&" SimpleTemplateSpec]
| SimpleTemplateSpec
SimpleTemplateSpec ::= CharStringMatch
| SingleTemplateExpression ["&" SimpleSpec]
SingleTemplateExpression ::= MatchingSymbol
| ( TemplateRefWithParList [ExtendedFieldReference] )
MatchingSymbol ::=
Complement |
AnyValue |
AnyOrOmit |
ValueOrAttribList |
Range |
BitStringMatch |
HexStringMatch |
OctetStringMatch |
SubsetMatch |
SupersetMatch
This basically resolves the ambiguity in favor of INNER & operators in case there is a shift/reduce conflict.

(0009840)
Ina Schieferdecker
30-11-2010 11:21

Updated the resolution (v8) acc. to Jacobs feedback - please check

(0009872)
Gyorgy Rethy
01-12-2010 09:16

First example corrected ("*" was missing in the middle template to be concatenated). See in CR5513_resolution v9.doc

(0009878)
Ina Schieferdecker
01-12-2010 10:11

Fixed BNF issue on WildcardLengthMatch

(0009978)
Ina Schieferdecker
15-12-2010 00:41

One last correction for WildcardLengthMatch: 128. WildcardLengthMatch ::= LengthKeyword "(" ConstantExpression ")"