# Looking for a way to import csv values without changing certain characters like "°"

**URL:** <https://forum.inductiveautomation.com/t/looking-for-a-way-to-import-csv-values-without-changing-certain-characters-like/107601>\
**Category:** Ignition\
**Tags:** scripting, ignition81\
**Created:** [June 19, 2025, 11:14pm UTC](https://forum.inductiveautomation.com/t/looking-for-a-way-to-import-csv-values-without-changing-certain-characters-like/107601 "2025-06-19T23:14:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![r123](https://avatars.discourse-cdn.com/v4/letter/r/838e76/32.png) [@r123](https://forum.inductiveautomation.com/u/r123)\
**Post date:** [June 19, 2025, 11:14pm UTC](https://forum.inductiveautomation.com/t/looking-for-a-way-to-import-csv-values-without-changing-certain-characters-like/107601/1 "2025-06-19T23:14:55Z")

</div>

I've tried changing a few different things but nothing's working. I changed the format that the Excel VBA is using when saving the CSV file but then that added quote marks in the CSV which I couldn't get ignition to deal with.

On the ignition side I've tried changing the encoding to a few different things but the best I can do is change the character I end up with from "�" to "�"

I'm sure I'm missing something simple but I don't know why it's so hard to just give me the character that's actually in the CSV file?

```python
			try:
				# This is the direct implementation of your suggestion:
				# Use io.open with 'utf-8-sig' to handle the BOM
				f = io.open(filePath, 'r', encoding='utf-8-sig')
				REST.Utilities.Tag_Functions_logger.info("Opened CSV file with utf-8-sig encoding: {}".format(filePath))
			except Exception as e:
				REST.Utilities.Tag_Functions_logger.error("Error opening file with utf-8-sig: {}".format(e))
				REST.Utilities.Tag_Functions_logger.error("Traceback: {}".format(traceback.format_exc()))
				REST.Utilities.Tag_Functions_logger.error("Attempting fallbacks (cp1252, latin-1) if utf-8-sig fails.")
				
				# --- Fallback if io.open with utf-8-sig explicitly fails ---
				# This block is what my previous solution focused on more
				opened_successfully_fallback = False
				for encoding_attempt in ['cp1252', 'latin-1']: # Removed utf-8 as utf-8-sig failed already
					try:
						f = codecs.open(filePath, 'r', encoding=encoding_attempt) # codecs.open can be more reliable for fallbacks in Jython 2
						f.read(1024) # Force a read to test encoding
						f.seek(0)
						REST.Utilities.Tag_Functions_logger.info("Opened CSV file with {} encoding: {}".format(encoding_attempt, filePath))
						opened_successfully_fallback = True
						break
					except UnicodeDecodeError as e_fallback:
						REST.Utilities.Tag_Functions_logger.error("Failed to decode CSV with {}. Error: {}. Trying next encoding.".format(encoding_attempt, e_fallback))
					except Exception as e_fallback_gen:
						REST.Utilities.Tag_Functions_logger.error("An unexpected error during fallback with {}: {}".format(encoding_attempt, e_fallback_gen))
						REST.Utilities.Tag_Functions_logger.error("Traceback: {}".format(traceback.format_exc()))
				
				if not opened_successfully_fallback:
					REST.Utilities.Tag_Functions_logger.error("Could not open CSV file with any of the attempted encodings. Returning.")
					return

```

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [June 20, 2025, 12:25am UTC](https://forum.inductiveautomation.com/t/looking-for-a-way-to-import-csv-values-without-changing-certain-characters-like/107601/2 "2025-06-20T00:25:58Z")

</div>

The file is probably encoded in 8859-1, not utf8.

I'd be using java libraries, not jython, when dealing with encodings.

Start with [StandardCharsets](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/charset/StandardCharsets.html).

---

<div class="post-metadata">

**Author:** ![r123](https://avatars.discourse-cdn.com/v4/letter/r/838e76/32.png) [@r123](https://forum.inductiveautomation.com/u/r123)\
**Post date:** [June 20, 2025, 12:42am UTC](https://forum.inductiveautomation.com/t/looking-for-a-way-to-import-csv-values-without-changing-certain-characters-like/107601/3 "2025-06-20T00:42:05Z")

</div>

Notepad++ says it's ANSI and I tried changing it to UTF-8. That is when it added quotes around each of the values and I was having trouble finding a way to deal with that. CSVs and Excel always seem so squirrely, like when you open a CSV and Excel immediately tries to reformat some of your values and so on.

Thanks for the info and I'll see if I can make sense of standard character sets. 🙂

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [June 20, 2025, 1:38am UTC](https://forum.inductiveautomation.com/t/looking-for-a-way-to-import-csv-values-without-changing-certain-characters-like/107601/4 "2025-06-20T01:38:46Z")

</div>

Consider reading the source excel file with the Apache POI libraries instead of passing through CSV. CSV is a lossy format.