Translating project

Superb. Thank you very much. :thumb_left:

I can say now that Excel exporting CSV with unicode stinks. Same for WPS office. You can export to a Unicode text file, but holy cats! that hoses over the English portions.

OpenOffice has better options, IIRC, so I’ll see where that leads…

Have I mentioned how much I dislike playing with Unicode? :smiling_imp:

I think I’ve got it. I know it will work with two-byte encoding, anyway. Haven’t had a chance to check it out with 3- and 4-byte encodings yet. I’ll hammer those out as needed.

Attached is a window you can import that will handle the Unicode. There are also a sample input file to reference (created in OpenOffice), and a resulting output file.

To make the CSV file in a table format and working with Unicode, I’m going to suggest OpenOffice. It has more (and generally better) options for Unicode, and free is not a bad price. Just be sure to save it in UTF-8 :wink:

Notepad++ is an okay option too, but a grid view to edit is not available.

[attachment=2]Unicode Translator_2016-02-09_1052.proj[/attachment]

@JordanCClark

Thank you for your effort. :thumb_left:

I tried it and it works (I didn’t tried it extensively).
But one thing I can’t figure it out:
How the hell can I get all my English texts from my Ignition project to openOffice?
If I understand this correctly, I must create csv file (in openOffice), put in all English texts, save it in UTF-8 format, send it to Russians (with the instructions about openOffice and UTF-8).
OK, maybe I can (and they will also, I hope) swallow this procedure…

But how to get all texts from Ignition to openOffice? :scratch:

Updated the window to give you options for both directions (properties <-> csv)

EDIT: Remembered I forgot to add the space substitutions :blush: Fixed now.

EDIT TO THE EDIT: Never mind. Removed file… Read further for explanation. :wink:

OK, kudos for your effort. I tried this, but…
This is your file output_ru.properties:

#Locale: ru #Fri Feb 05 12:01:06 EST 2016 Logout=\u041b\u043e\u0433\u0438\u043d Login=\u0412\u044b\u0445\u043e\u0434 This\ is\ a\ test\ for\ a\ full\ sentence\ translation.=\u042d\u0442\u043e \u0442\u0435\u0441\u0442 \u0434\u043b\u044f \u043f\u043e\u043b\u043d\u043e\u0433\u043e \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0430 \u043f\u0440\u0435\u0434\u043b\u043e\u0436\u0435\u043d\u0438\u0439.
When I put it in your project in properties->csv, this is csv file it outputs:

#Locale: ru, #Fri Feb 05 12:01:06 EST 2016, Logout,\u041b\u043e\u0433\u0438\u043d Login,\u0412\u044b\u0445\u043e\u0434 This\\ is\\ a\\ test\\ for\\ a\\ full\\ sentence\\ translation.,\u042d\u0442\u043e\ \u0442\u0435\u0441\u0442\ \u0434\u043b\u044f\ \u043f\u043e\u043b\u043d\u043e\u0433\u043e\ \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0430\ \u043f\u0440\u0435\u0434\u043b\u043e\u0436\u0435\u043d\u0438\u0439.
I think, this is unusable… :scratch:

Oh yeah. Just had a “duh” moment. I didn’t need to do anything with the spaces. It was already part of the export.

Well, as we say, it’s why we call it debugging… :laughing:

Here’s the original one. Before I broke it.
[attachment=3]Unicode Translator_2016-02-13_1200.proj[/attachment]
[attachment=2]2016-02-16_10-42-18.jpg[/attachment]

EDIT: It occurs to me (maybe another “duh” moment) that you may have had this in mind. This one should take care of that for you.
[attachment=1]Unicode Translator_2016-02-16_1134.proj[/attachment]
[attachment=0]2016-02-16_11-34-07.jpg[/attachment]

Testing with Japanese (Hiragana). So it works with 3-byte UTF-8 :slight_smile:

[quote=“JordanCClark”]EDIT: It occurs to me (maybe another “duh” moment) that you may have had this in mind. This one should take care of that for you.
[attachment=4]Unicode Translator_2016-02-16_1134.proj[/attachment]
[attachment=3]2016-02-16_11-34-07.jpg[/attachment]

Testing with Japanese (Hiragana). So it works with 3-byte UTF-8 :slight_smile:[/quote]

Yes, that’s what I need. :slight_smile:
But… it’s not working.
The simple test should be:

  • I create .PROPERTIES file from translation manager and convert it to .csv with your PROPERTIES->CSV function
  • convert created .csv file (without editing it) back to .PROPERTIES with your CSV->PROPERTIES function
  • import .PROPERTIES file to translation manager

The .PROPERTIES file, created from translation manager and .PROPERTIES file, created with your CSV->PROPERTIES function are not the same.

In file from translation manager, every space and special character is preceded by \ (backslash), and in your file spaces remain as spaces (without \ before space). So, when I try to import your file in translation manager, I get … not right… :slight_smile:
There are also no = signs in your file (where is no translation for english text).

In the attachment are three files: one from my translation manager, one from your PROPERTIES->CSV function and third from your function CSV->PROPERTIES.

Thanks for the properties file you’re working with. That helped tremendously!

All right. After finding all the escaped characters (well, all I could find anyway), and going back and using Java’s regex instead of Jython’s, I think we now have a fully working version.

Let me know how it works for you.

THAT’s IT! :prayer:

Now it working, like it should. :smiley: :thumb_left:

Thank you very much.
I think, this would help other people also… at least, when they come to the point of sending files to the other countries with another language…

What it’s not clear to me is, what were people at IA thinking, when they developed this translation export/import functions… why they didn’t implement something like this already? :scratch:

[quote=“zxcslo”]
What it’s not clear to me is, what were people at IA thinking, when they developed this translation export/import functions… why they didn’t implement something like this already? :scratch:[/quote]

I believe the intent was to handle all the translation entry from within Ignition. What we did is rather like Mythbusters: using tools and materials in ways for which they were not intended. :laughing:

As we saw in our exercise here, the problem with trying to make something absolutely universal is that the universe is a really big place. :wink:

Throw in locales and different ways to delineate thousands separators and decimal places in numbers, and the ways that Java/Jython/Windows/Linux/etc. handle such things (or not handle, as the case may be)-- I get headaches just thinking about it.

Anyway, glad it’s working for you! :slight_smile:

1 Like

For anyone interested in the future: I created a Python tool to convert the Translation Manager exported XML file to CSV and back from CSV to XML. You can check it out on GitHub: https://github.com/crooke/ignition-translation-converter

2 Likes

Here is the original project again. Thanks, @JordanCClark again for his help.
Ignition Localization PROPERTIES converter_2016-02-19_1258.proj (16.9 KB)

@ecrook
While you have all my respect for your contribution :thumbsup: , I would suggest making this work in Ignition. Not everyone is running Linux or have python installed (like me).

Huh??
I didn't see anything Linux-specific in Eric's code on github. Whille every Linux distro I know has python in its base packages, python is just as popular on Windows.

1 Like

Thanks @JordanCClark for your work on this. I have the same issues that @zxcslo has with HMI users in other countries. Excel is the lowest common denominator, and we are looking into getting translations from Ignition into Excel-based CSV and then back into Ignition. In addition, we leave the site where SCADA is located (often without Internet connection) and the customer needs to be able to translate something customer specific. So there must be an easy method to have customers translate things. I am hoping I don’t have to resort to our old method of storing our translations in SQL, and making SQL calls to change the text on the screens. Any thoughts?

translationManager.proj (11.8 KB)

This is why I like Ignition, it’s so open that you can plug in your own code to achieve this.

Here, I made a page (as template, so it can be reused over projects) that loads all terms and translations to a certain language into a table. The table can be edited directly, or exported and imported (to CSV in this case, but Excel would also work).

Disclaimer: I’m not responsible for any loss of data (translation strings or other), please make a project backup before using it.

4 Likes

I had been working this up on and off (mostly off) for a few months. This works with XML exports of the Translation Manager, and will handle multiple locales at once. You will need a Microsoft Azure account, but the free version is, as of this writing, 2 million characters per month.

  1. Export XMLs from Translation Manager

  2. Run the script in whatever fashion you choose.

  3. Resultant files will be in the same directory.

  4. Import the translated files back into the Translation Manager.
    NOTE: Currently, the locales are not being correncly parsed when importing back in. (see this post). But, manually selecting the locale dropdown will let you work around it. :slight_smile:

import javax.xml.parsers
import javax.xml.transform
import java.io.FileOutputStream as FOS
import httplib, urllib, uuid, codecs, os, glob

# **********************************************
# *** Update or verify the following values. ***
# **********************************************

# Replace the subscriptionKey string value with your valid subscription key.
subscriptionKey = 'put_your_own_key_here'

host = 'api.cognitive.microsofttranslator.com'
path = '/translate?api-version=3.0'

#*************************************************

filepath = system.file.openFile()
# Flag to tell it we are processing all of the files are just the selected one.
allFiles = True

if filepath != None:
  # Base file name without locale
  baseName = filepath[:-7]
  # Extension of the file
  extension = filepath[-4:]
  if allFiles == True:
    fileFilter = baseName + '*' + extension
  else:
    fileFilter = filepath

  # create file list 
  fileList = glob.glob(fileFilter)

  # create list of locales
  langList =  [f.replace(baseName,'').replace(extension, '')[1:] for f in fileList]

  # create language parameters to use with Microsoft Translate
  params = '&to='+'&to='.join(langList)
  
  # open the selected file and extract keys to be translated
  dbf = javax.xml.parsers.DocumentBuilderFactory.newInstance()
  db = dbf.newDocumentBuilder()
  dom = db.parse(filepath)
  doc = dom.getDocumentElement()
  entries = doc.getElementsByTagName('entry')
  keyList =[]
  for i in range(entries.getLength()):
    node = entries.item(i)
    keyList.append(node.getAttribute('key'))
	
  # header for Microsoft Translate API call
  headers = {
        'Ocp-Apim-Subscription-Key': subscriptionKey,
        'Content-type': 'application/json',
        'X-ClientTraceId': str(uuid.uuid4())
    }
  # serialize dictionary for use in Microsoft Translate API call
  requestBody = []
  for key in keyList:
    requestBody.append({'Text' : key})
  content = system.util.jsonEncode(requestBody)
  
  # send request to Microsoft Translate  
  conn = httplib.HTTPSConnection(host)
  conn.request ("POST", path + params, content, headers)
  # get response
  response = conn.getresponse()
  data = response.read()
  translatedData = system.util.jsonDecode(data)

  # create dictionary with all translated data
  dictOut = dict((key,{}) for key in keyList)
  for key, t in zip(keyList, translatedData):
    for language, translation in zip(langList, t['translations']):
      dictOut[key][language]=translation['text']

  # create output XML files      
  for lang in langList:
    fileName = baseName + '_' + lang + '_output' + extension
    

    dbf = javax.xml.parsers.DocumentBuilderFactory.newInstance()
    db = dbf.newDocumentBuilder()
    dom = db.newDocument()
	# XML root
    root = dom.createElement('properties')
	#create 'comment' element with locale
    e = dom.createElement('comment')
    e.appendChild(dom.createTextNode('Locale: '+lang))
	# add 'comment' element to root
    root.appendChild(e)
	# add translations
    for key in keyList:
	  # create entry element
      e = dom.createElement('entry')
	  # add 'key' attibute with original text
      e.setAttribute('key', key)
	  # add translated data for the key entry
      e.appendChild(dom.createTextNode(dictOut[key][lang].decode('utf-8')))
	  # add 'entry' element to root
      root.appendChild(e)
	#close root
    dom.appendChild(root)
    
	# prepare xml file properties
    tr = javax.xml.transform.TransformerFactory.newInstance().newTransformer()
    tr.setOutputProperty(javax.xml.transform.OutputKeys.INDENT, 'yes')
    tr.setOutputProperty(javax.xml.transform.OutputKeys.METHOD, 'xml')
    tr.setOutputProperty(javax.xml.transform.OutputKeys.ENCODING, 'UTF-8')
    tr.setOutputProperty(javax.xml.transform.OutputKeys.DOCTYPE_SYSTEM, 'http://java.sun.com/dtd/properties.dtd')
    tr.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "0")
	# convert XML data to a DOM source for writing to a file
    domSource = javax.xml.transform.dom.DOMSource(dom)
	# prepare FileOutputStream
    streamFile = FOS(fileName)
	# create StreamResult from FileOutputStream
    streamOut = javax.xml.transform.stream.StreamResult(streamFile)
	# write the DOM source to a file
    tr.transform(domSource, streamOut)
	# close the file
    streamOut.getOutputStream().close()

Disclaimer below, but here’s the TLDR version: It works for me, YMMV. Consider yourself warned.

Disclimer: While Jordan has made every attempt to ensure that the information contained in this post has been obtained from reliable sources, Jordan is not responsible for any errors or omissions, or for the results obtained from the use of this information. All information in this post is provided “as is”, with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information, and without warranty of any kind, express or implied, including, but not limited to warranties of performance, merchantability and fitness for a particular purpose. In no event will Jordan, his related partnerships or corporations, or the partners, agents or employees thereof be liable to you or anyone else for any decision made or action taken in reliance on the information in this post or for any consequential, special or similar damages, even if advised of the possibility of such damages.

4 Likes

Raising a thread from the dead here, but I just pushed a commit that will probably be in 8.0.2 which should help all of you out.

With this commit, exports of translations to property files now show non-ASCII characters in a human-readable way (ie. мир instead of \u043c\u0438\u0440)

Imports of property files with work either way (you can mix and match in the same file if you need to)

The net effect of that is you no longer need to have your translators deal with XML, which should make everyone happier.

What about v7.9? It will also get that?

I didn’t see any note on the ticket that we’re planning on backporting the feature.