ETSI's Bug Tracker - Ext Pack: Advanced Matching (ES 203 022)
View Issue Details

ID | Project | Category | View Status | Date Submitted | Last Update
0007763 | Ext Pack: Advanced Matching (ES 203 022) | New Feature | public | 15-05-2018 09:31 | 05-01-2019 14:06

Reporter | KristÃ³f Szabados |
Assigned To | Gyorgy Rethy |
Priority | normal | Severity | major | Reproducibility | have not tried
Status | closed | Resolution | fixed |
Platform |  | OS |  | OS Version |
Product Version |  |
Target Version |  | Fixed in Version | v1.3.1 (ongoing) |
Clause Reference(s) | 27
Source (company - Author) | L.M. Ericsson
TS version |

Summary | 0007763: add support for sending erroneous messages

Description

Negative testing is a part of testing when the tester send incorrect/malformed messages to the SUT, to see how it will react.
TTCN-3 currently is not able to support this kind of testing, as it is only able to send correctly encoded messages. But supporting this kind of operation is needed, to ensure the high quality of the tested systems.
To support this behaviour 2 (a static and a dynamic) support for negativ testing would be needed.
Please note, that:
- this would only need to have an effect on the transfer syntax of the send messages, the abstract values does not need to be changed.
- As these incorrect messages are used to test the SUT, it is not needed for the codecs to support decoding them.
- Also as this is special way of coding data, while it should be handled similar to alrady existing encoding/variant attributes ... it would be better have a new attribute kind for it.
Support for static negativ testing:
The request here is to be able to tell the system, that during encoding the message needs to be altered in some user specified way.
This only needs to apply values and templates of record, set, record of, set of and union types.
Currently we have identified the following needs:
- inserting some data before a field in the encoded message.
- overwriting the encoded value of a field (also setting mandatory fields to omit)
- inserting some extra data after the encoded value of a field.
For example:
template CX_Frame t_CX_Frame:=
{
data_length:=0,
data_stream:='AAABBCCDDEEFF1122‘O
} with
{
erroneous (data_length) “after(raw) := 'FF'O “;
erroneous (data_length) “value(raw) := 'AABBCC'O “;
erroneous (data_length) “before(raw) := 'FF'O “;
}
Would like this encoded: ‘FFAABBCCFFAAABBCCDDEEFF112’O
/* the data_length fields encoded value is replaced by 'AABBCC' and also prefixed and postfixed with 'FF'O */
This static encoding has the benefits of being static (efficiently optimizable before execution) and for its attributes the mechanisms existing for encode/variant attributes can be used.
Support for dynamic negativ testing:
Extending the static support for negativ testing, it would also be needed to have a machanism for dynamically altering the erroneous attributes of values and templates.
for example having a value like:
"const MyRec c_myrec := { i:=1, b:=true }
with {
erroneous (i) “before := 123”
erroneous (b) “value := omit”
}
"
Inside the body of functions, altsteps, testcases these attributes could be updated like:
- @update(c_myrec) with { erroneous(i) “value := 3.5” } // the field i's value shall be overwritten, but nothing is inserted before field i and field b is also not omitted.
- @update(c_myrec); // no longer erroneous, all arreneous attributes are removed

Steps To Reproduce |
Additional Information |
Tags | No tags attached.

Relationships

related to | 0007785 | closed | Jens Grabowski | Add Mutation annotations to the Value data type

Attached Files | CR7763.docx (123,228) 17-07-2018 16:45 http://oldforge.etsi.org/mantis/file_download.php?file_id=3767&type=bug CR7763-2.docx (133,303) 20-07-2018 09:01 http://oldforge.etsi.org/mantis/file_download.php?file_id=3783&type=bug

Issue History
Date Modified | Username | Field | Change
15-05-2018 09:31 | KristÃ³f Szabados | New Issue |
15-05-2018 09:34 | KristÃ³f Szabados | Note Added: 0015102 |
16-07-2018 14:06 | Jens Grabowski | Note Added: 0015131 |
16-07-2018 14:07 | Jens Grabowski | Assigned To | => Tomas Urban
16-07-2018 14:07 | Jens Grabowski | Status | new => assigned
16-07-2018 14:09 | KristÃ³f Szabados | Project | TTCN-3 Change Requests => Ext Pack: Advanced Matching (ES 203 022)
17-07-2018 13:50 | Tomas Urban | Assigned To | Tomas Urban => KristÃ³f Szabados
17-07-2018 13:50 | Tomas Urban | Assigned To | KristÃ³f Szabados => Jacob Wieland - Spirent
17-07-2018 16:45 | Jacob Wieland - Spirent | File Added: CR7763.docx |
17-07-2018 16:46 | Jacob Wieland - Spirent | Note Added: 0015149 |
19-07-2018 16:02 | Jacob Wieland - Spirent | Relationship added | related to 0007785
20-07-2018 09:01 | Jacob Wieland - Spirent | File Added: CR7763-2.docx |
20-07-2018 09:01 | Jacob Wieland - Spirent | Note Added: 0015185 |
20-07-2018 09:01 | Jacob Wieland - Spirent | Assigned To | Jacob Wieland - Spirent => KristÃ³f Szabados
20-07-2018 09:01 | Jacob Wieland - Spirent | Status | assigned => confirmed
17-11-2018 15:18 | KristÃ³f Szabados | Note Added: 0015278 |
17-11-2018 15:18 | KristÃ³f Szabados | Status | confirmed => resolved
17-11-2018 15:18 | KristÃ³f Szabados | Resolution | open => fixed
17-11-2018 15:18 | KristÃ³f Szabados | Assigned To | KristÃ³f Szabados => Jens Grabowski
05-01-2019 14:06 | Gyorgy Rethy | Note Added: 0015321 |
05-01-2019 14:06 | Gyorgy Rethy | Status | resolved => closed
05-01-2019 14:06 | Gyorgy Rethy | Assigned To | Jens Grabowski => Gyorgy Rethy
05-01-2019 14:06 | Gyorgy Rethy | Fixed in Version | => v1.3.1 (ongoing)

Notes

(0015102)
KristÃ³f Szabados
15-05-2018 09:34

Please note that this would enable fuzz testing in TTCN-3. As a fuzzer algorithm could automatically craft and send messages involving invalid, unexpected or even random data as the input for the SUT.

(0015131)
Jens Grabowski
16-07-2018 14:06

STF discussion: Interesting feature, feature should be made for flexible. Feature should go to Advanced Matching Ext. package.

(0015149)
Jacob Wieland - Spirent
17-07-2018 16:46

uploaded a first draft of a description of mutation annotation templates

(0015185)
Jacob Wieland - Spirent
20-07-2018 09:01

please review the latest draft

(0015278)
KristÃ³f Szabados
17-11-2018 15:18

looks fine for me, can be included in the next version.

(0015321)
Gyorgy Rethy
05-01-2019 14:06

Added to draft v1.2.2 as new clause 5.6!