# Special Non-English Character Issue in downloaded csv files using system.perspective.download

**URL:** <https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580>\
**Category:** Ignition\
**Created:** [April 19, 2022, 7:12am UTC](https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580 "2022-04-19T07:12:35Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shouvik\_Roy](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@Shouvik\_Roy](https://forum.inductiveautomation.com/u/Shouvik_Roy)\
**Post date:** [April 19, 2022, 7:12am UTC](https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580/1 "2022-04-19T07:12:35Z")

</div>

Hello Team,  
we are facing some issue when opening downloaded csv files (data read from SQL DB) using system.perspective.download that contain Non-English special characters.  
The characters are not correctly denoted after the csv file is opened.  
Need some solution or alternative suggestion on the issue.

Thank You

---

<div class="post-metadata">

**Author:** ![Transistor](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/transistor/32/3380_2.png) [@Transistor](https://forum.inductiveautomation.com/u/Transistor)\
**Post date:** [April 19, 2022, 8:06am UTC](https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580/2 "2022-04-19T08:06:42Z")

</div>

You need to figure out where the problem is: in the database or in the Ignition read and download.

When you look in the database directly is the encoding correct?

Which database type are you using?  
What character encoding are you expecting?

---

<div class="post-metadata">

**Author:** ![Shouvik\_Roy](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@Shouvik\_Roy](https://forum.inductiveautomation.com/u/Shouvik_Roy)\
**Post date:** [April 19, 2022, 10:45am UTC](https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580/3 "2022-04-19T10:45:57Z")

</div>

In Database the characters are shown correctly also there is no error when we read into ignition using system.dataset.toCsv method , only issue occurs when we open that file in microsoft excel.  
We require ANSI encoding. Currently it is UTF-8 encoding.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [April 19, 2022, 12:36pm UTC](https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580/4 "2022-04-19T12:36:09Z")

</div>

> [@Shouvik\_Roy](#):
>
> contain Non-English special characters

> [@Shouvik\_Roy](#):
>
> We require ANSI encoding

These two requirements are fundamentally incompatible. ANSI doesn't have many (any?) non-english characters.

---

<div class="post-metadata">

**Author:** ![JordanCClark](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/jordancclark/32/83362_2.png) [@JordanCClark](https://forum.inductiveautomation.com/u/JordanCClark)\
**Post date:** [April 19, 2022, 2:59pm UTC](https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580/5 "2022-04-19T14:59:40Z")

</div>

> [@Shouvik\_Roy](#):
>
> We require ANSI encoding.

Is the file used elsewhere and you really need ANSI, or are you just opening it in Excel? If the latter, you'll need to add a byte-order marker to the file:

```python
csvData = u'\xef\xbb\xbf' + system.dataset.toCSV(dataset)

```

---

<div class="post-metadata">

**Author:** ![Amy\_Felix](https://avatars.discourse-cdn.com/v4/letter/a/919ad9/32.png) [@Amy\_Felix](https://forum.inductiveautomation.com/u/Amy_Felix)\
**Post date:** [April 24, 2023, 10:43pm UTC](https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580/6 "2023-04-24T22:43:14Z")

</div>

What does the **'\xef\xbb\xbf'** mean on the snip of code?

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [April 24, 2023, 11:00pm UTC](https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580/7 "2023-04-24T23:00:13Z")

</div>

Do you mean the `\xef` and similar escape sequences? Described here:

[https://docs.python.org/2.7/reference/lexical\_analysis.html#string-literals](https://docs.python.org/2.7/reference/lexical_analysis.html#string-literals)

Or do you mean the [Byte Order Mark](https://en.wikipedia.org/wiki/Byte_order_mark) itself?

---

<div class="post-metadata">

**Author:** ![nideyijuyidong](https://avatars.discourse-cdn.com/v4/letter/n/7cd45c/32.png) [@nideyijuyidong](https://forum.inductiveautomation.com/u/nideyijuyidong)\
**Post date:** [April 25, 2023, 3:33am UTC](https://forum.inductiveautomation.com/t/special-non-english-character-issue-in-downloaded-csv-files-using-system-perspective-download/58580/8 "2023-04-25T03:33:22Z")

</div>

ANSI is a collection of multiple encodings(GB2312、GBK、GB18030、Big5、Shift\_JIS and so on), and different languages correspond to different encodings.  
I did a little test. I created a GB2312 (ANSI for Simplified Chinese) txt file and opened it with Notepad. Then it shows that the encoding is ANSI. Therefore, the Simplified Chinese Windows operating system thinks that GB2312 is ANSI.  
For ANSI users of other languages, I think setting the encoding to the corresponding language's encoding should do the trick.
