Escaping Special Characters

Special characters, for example the newline character 0x0A or 0x0D, or language-specific symbols, can corrupt exported data files. They can break lines or cause problems during data loading within the ingestion process.

To avoid these issues, you can escape (replace) these characters with different characters/bytes that do not cause issues during data ingestion.

To set up the escaping function, in the FEX main menu, execute the step Setup > Parameters setup.

Every parameter that is relevant for escaping must have the prefix ESCAPE_ in its name.

The setup consists of three parameters:

  1. ESCAPE_DATATYPE: Specifies the data type that is escaped during the export. It can contain multiple values separated by semicolons.
    Example: CHAR;STRING

  2. ESCAPE_HEX_<number>: Defines the escaping at the byte level. It is mainly used for whitespace characters such as newline / horizontal tab or controlling bytes. Each line must contain a single key–value pair.
    Format: key;value where both key and value are single hexadecimal values according to ASCII. Key represents the to be replaced character, and value represents the replace with character. The values are separated by a semicolon.
    Example: Escaping the newline character with space: 0A;20

  3. ESCAPE_STR_<number>: Defines the escaping at the character or string level. It is used for replacing characters (e.g. ß, Ä) that can be specified as characters/strings. Each line must contain a single key–value pair.
    Format: key;value where both key and value are a character or string. Key represents the to be replaced string, and value represents the replace with string. They are separated by a semicolon.

    Example of escaping ß with ss: ß;ss

    Example of setup for escaping the fields CHAR and STRING:

  • Newlines for Windows and Unix (0x0A), carriage returns for Windows and Mac (0x0D) and bytes (0x90) will be escaped with space (0x20).

  • Sharp s (ß) will be escaped with double s (ss).