ETSI's Bug Tracker - Part 01: TTCN-3 Core Language
View Issue Details

ID | Project | Category | View Status | Date Submitted | Last Update
0006586 | Part 01: TTCN-3 Core Language | Technical | public | 11-07-2013 10:57 | 09-04-2014 14:11

Reporter | Gyorgy Rethy |
Assigned To | Ina Schieferdecker |
Priority | high | Severity | minor | Reproducibility | always
Status | closed | Resolution | fixed |
Platform |  | OS |  | OS Version |
Product Version |  |
Target Version | v4.6.1 (published 2014-06) | Fixed in Version | v4.6.1 (published 2014-06) |
Clause Reference(s) | 16,1.2, Annex C
Source (company - Author) | STF460

Summary | 0006586: How to encode to/decode from XML values containing non-ASCII characters?

Description

Issue is raised by 3GPP TF160, see mail from Wolfgang Seka:
Hi GyÃ¶rgy, Ina, Jens, Tomas,
I've still have another issue regarding XSD on my list:
Firstly just for clarification - I guess the common way to encode an XSD type (lets call it MyXSD_Type) is
var template (value) MyXSD_Type v_XML_Message := cs_XML_Template;
var charstring v_EncodedXML := oct2char(bit2oct(encvalue(v_XML_Message)));
and in the other direction
var MyXSD_Type v_XML_Message;
var bitstring v_Bitstring := oct2bit(char2oct(v_EncodedXML));
decvalue(v_Bitstring, v_XML_Message);
Even though we can encapsulate that, from a user's point of view it is a bit "bulky" so I wonder whether it would be werth to introduce special encoding/decoding functions for this purpose
(not that urgent but at least nice-to-have).
But there is another issue related to this: In the XSD files which we are using there are often extensions so that there is an "any" place-holder in the base definitions which gets populated which specific types by an extension.
Typically the place-holder is converted to an "XSD.String" field in the TTCN. Now it seems that different tools have different definitions for XSD.String: as long as it results in charstring there is no problem and the extension can be encoded/decoded by using encvalue/decvalue as shown above. But it seems that XSD.String is not always "charstring" but may also be defined as something like "universal charstring" what causes a problem with char2oct in the above.
(that would be another reason to introduce specific functions to encode/decode XSD types as these functions may use XSD.String as abstract type)
So - what is the generic solution ??
Best regards
Wolfgang

Steps To Reproduce |
Additional Information |
Tags | No tags attached.

Relationships

related to | 0006706 | closed | Gyorgy Rethy | Encoding in oct2unichar, unichar2oct, encvalue_unichar, decvalue_unichar

Attached Files | CR6586.doc (226,304) 11-07-2013 19:13 http://oldforge.etsi.org/mantis/file_download.php?file_id=2842&type=bug CR6586-v2.doc (235,008) 27-08-2013 17:26 http://oldforge.etsi.org/mantis/file_download.php?file_id=2863&type=bug

Issue History
Date Modified | Username | Field | Change
11-07-2013 10:57 | Gyorgy Rethy | New Issue |
11-07-2013 10:57 | Gyorgy Rethy | Clause Reference(s) | => 16,1.2, Annex C
11-07-2013 10:57 | Gyorgy Rethy | Source (company - Author) | => STF460
11-07-2013 10:59 | Jacob Wieland - Spirent | Status | new => assigned
11-07-2013 10:59 | Jacob Wieland - Spirent | Assigned To | => Jacob Wieland - Spirent
11-07-2013 11:17 | Gyorgy Rethy | Priority | normal => high
11-07-2013 11:17 | Gyorgy Rethy | Target Version | => v4.6.1 (published 2014-06)
11-07-2013 11:34 | Jacob Wieland - Spirent | Note Added: 0011536 |
11-07-2013 11:38 | Gyorgy Rethy | Note Added: 0011537 |
11-07-2013 11:38 | Jacob Wieland - Spirent | Note Added: 0011538 |
11-07-2013 19:13 | Jacob Wieland - Spirent | File Added: CR6586.doc |
11-07-2013 19:14 | Jacob Wieland - Spirent | Note Added: 0011561 |
11-07-2013 19:14 | Jacob Wieland - Spirent | Assigned To | Jacob Wieland - Spirent => Ina Schieferdecker
11-07-2013 19:14 | Jacob Wieland - Spirent | Status | assigned => confirmed
26-08-2013 15:49 | Ina Schieferdecker | File Added: CR6586-v2.doc |
26-08-2013 15:49 | Ina Schieferdecker | Note Added: 0011599 |
26-08-2013 15:50 | Ina Schieferdecker | Status | confirmed => assigned
26-08-2013 15:50 | Ina Schieferdecker | Assigned To | Ina Schieferdecker => Jacob Wieland - Spirent
26-08-2013 16:42 | Jacob Wieland - Spirent | Note Added: 0011603 |
27-08-2013 15:30 | Ina Schieferdecker | File Deleted: CR6586-v2.doc |
27-08-2013 15:31 | Ina Schieferdecker | File Added: CR6586-v2.doc |
27-08-2013 17:23 | Ina Schieferdecker | File Deleted: CR6586-v2.doc |
27-08-2013 17:23 | Ina Schieferdecker | File Added: CR6586-v2.doc |
27-08-2013 17:25 | Ina Schieferdecker | File Deleted: CR6586-v2.doc |
27-08-2013 17:26 | Ina Schieferdecker | File Added: CR6586-v2.doc |
27-08-2013 17:30 | Jacob Wieland - Spirent | Assigned To | Jacob Wieland - Spirent => Ina Schieferdecker
27-08-2013 17:30 | Jacob Wieland - Spirent | Status | assigned => confirmed
27-08-2013 18:46 | Ina Schieferdecker | Note Added: 0011614 |
27-08-2013 18:46 | Ina Schieferdecker | Status | confirmed => resolved
27-08-2013 18:46 | Ina Schieferdecker | Resolution | open => fixed
27-08-2013 18:46 | Ina Schieferdecker | Fixed in Version | => v4.6.1 (published 2014-06)
27-08-2013 18:47 | Ina Schieferdecker | Status | resolved => closed
09-04-2014 14:11 | Gyorgy Rethy | Relationship added | related to 0006706

Notes

(0011536)
Jacob Wieland - Spirent
11-07-2013 11:34

STF-discussion: we propose to add two new predefined functions
decvalueuchar(in universal charstring encoded_value,
inout any_data_type decoded_value,
in charstring string_encoding) return integer
encvalueuchar(in any_data_type val, in charstring string_encoding) return universal charstring
Maybe the names could be chosen as less of a mouthfull: encuchar, decuchar maybe or value2uchar and uchar2value
For completeness sake, we probably should also add two other conversion functions.
uchar2oct(in universal charstring str, in charstring string_encoding) return octetstring
oct2uchar(in octetstring str, in charstring string_encoding) return universal charstring
For the string_encoding parameters, a list of predefined allowed/standardized values and their respective meanings shall be added. Tool providers could be free to add additional, non-standardized allowed values that their tool supports (which could then be standardized later).
If these functions exists, we have to equivalences:
decvalueuchar(cstr, val, enc) =^= decvalue(oct2bit(uchar2oct(cstr, enc), val)
encvalueuchar(val, enc) =^= oct2uchar(bit2oct(encvalue(val)), enc)

(0011537)
Gyorgy Rethy
11-07-2013 11:38

STF discussion 2013-07-11:
Add new encvalueuchar and decvalueuchar predefined functions. encvalueuchar will encode TTCN-3 values into a universal charstring, as if oct2uchar(bit2oct(encvalue(v_XML_Message)),UTF8) was used (just TTCN-3 donâ€™t have oct2uchar today) and decvalueuchar will do the opposite. So, one can convert between TTCN-3 representations of XML values and an XML value in one step.
The type of encoding shall be defined by an additional argument, which will be a predefined enumeration; currently we are considering UTF8, UCS-2 (used by SMS) and UCS-4.

(0011538)
Jacob Wieland - Spirent
11-07-2013 11:38

Also, the string_encoding parameter could be made optional, i.e. given a default value, probably "utf-8".

(0011561)
Jacob Wieland - Spirent
11-07-2013 19:14

added oct2unichar, unichar2oct, decvalue_unichar, encvalue_unichar. References to character encoding standards are missing.

(0011599)
Ina Schieferdecker
26-08-2013 15:49

Please check the small changes

(0011603)
Jacob Wieland - Spirent
26-08-2013 16:42

No problem with me if the standard-references are not necessary. The encoded unicode (UCS-4) values were generated by use of wikipedia.

(0011614)
Ina Schieferdecker
27-08-2013 18:46

As proposed