Unicode ftp-8 TCP communications

I’ve set up an EMPERIA in-line printer as a TCP device and communication was working well. Then they wanted to put up and down arrows in the text being printed. When I added an up arrow, the printer showed this as a ‘?’ so it was not getting the coding correct. The documents say it is utf-8 coding for the command language.
In the printer, I set a variable up and put in an up arrow (U+2191 on their popup keyboard) and then read out the variable to Ign. What I got was ‘→ . If I copy that into the message I send to the printer, it does receive it as an arrow. I have tried some things I found online for encoding to utf-8 but nothing helps if I start with an actual arrow in the sting. I thought I would just put in a brute force decoding of the string before sending it and substitute in the characters it wants but I ran into a problem. This all happened on v8.0.13 so I updated to 8.1.14 last night. Now when I pull in a variable from the printer with a symbol, I actually see the symbol in the string, not the hieroglyphics I was getting before. Now I can’t even use that trick to get the ‘code’ for additional symbols. Can anyone point me in the right direction on how to encode these things properly?

How are you definining your strings, exactly?
Python 2 requires that strings that contain non-ASCII be prefixed with u to mark them as unicode literals, e.g.:
variable = u"â†"

I gave them a button to add arrows when setting the text to print.
newMsg = oldMsg + u’\u2191’
This shows an up arrow in the string but when I look in the printer it is a ‘?’.
if instead I do this:
newMsg = oldMsg + ‘↑’
it shows up in the printer as an arrow, but the messages in my db look like ↑↑Q↑↑ which nobody will understand. If I know how to get from u’\u2191’ to ↑ I can put an interpreter behind the scene, just before sending it to the printer. But I can’t find anything online. I assume it represents the utf-8 ‘0xE2 0x86 0x91’ for an up arrow, but I’m not sure how to translate to that and substitute it into the string as a character. Brute force would work if I knew the down arrow version of ↑ but then I would still only be handle those two characters. Tomorrow, they will want something else.

FYI, even in the post, you are not seeing what I am really pasting in there. It looks like this while I am editing.

Let’s step back a bit - can you share the full code you’re using to write to this printer, and/or any docs you might have from the printer? An encoding loss could be happening at multiple steps along the process.

There are a lot of steps that passes this thing around. A lot of this was an attempt to make it clear to future me.

I have a loop that constantly polls for data. In that loop I also have a step that checks if manual commands are in que to be sent. In a button, I add this manual command to that que to set a variable in the printer.

	var = self.getChild("Dropdown").props.value
	val = self.getChild("TextField_0").props.text
	#val = unicode(val, 'utf-8')
	#val = u'\u2191'
	if len(val) > 0:
		newRow = ['VAR_SET', var, val]
		oldDS = system.tag.read('[default]Printers/MPERIA/Send/PollingMessages').value
		newDS = system.dataset.addRow(oldDS, newRow)

Then it is picked up in the while loop (in an asynchronous thread function defined in a button for now) and then calls a script to build the msg in the format that the printer expects.

elif index == 5:
					# special command (manual stuff)
					# pull the command off of the top of the stack and send it
					# then remove the command. 
					specialMsgs = system.tag.read('[default]Printers/MPERIA/Send/PollingMessages').value
					if specialMsgs.rowCount > 0:
						cmd = specialMsgs.getValueAt(0, 0)
						p1 = specialMsgs.getValueAt(0, 1)
						p2 = specialMsgs.getValueAt(0, 2)
						msg = MPERIA.msg(cmd, p1, p2)
						removed = system.dataset.deleteRow(specialMsgs, 0)
						system.tag.write('[default]Printers/MPERIA/Send/PollingMessages', removed)

The script in the project Library that builds the message:

def msg(name, p1, p2):
	ACK = chr(6) 
	# Acknowledge. Sent from the Controller to indicate the success of a 
	# command sent to the Controller
	NAK = chr(21)
	# Negative acknowledge. Sent from the Controller to
	# indicate the failure of a command sent to the Controller.
	ETB = chr(23)
	# End of Transmission Block. Used as the final character in
	# all commands sent to the Controller, and as the end of data
	# sent from the Controller as a response to a command.
	ESC = chr(27)
	# Escape. Used as the starting character in all commands
	# sent to the Controller.
	FS = chr(28)
	# Unit Separator. Separates the command and arguments
	# sent to the Controller, and multiple responses sent from
	# the Controller as a response to a command.
	GS = chr(29)
	# Group Separator. Separates components of command
	# arguments into groups.
	RS = chr(30)
	# Record separator used to separate groups into records.
	US = chr(31)
	# Unit separator used to separate records into units.
	if p1 == None:
		msg = ESC + name + ETB
	elif p2 == None:
		msg = ESC + name + FS + p1 + ETB
		msg = ESC + name + FS + p1 + FS + p2 + ETB

	return msg

Then at the end of the loop

				if index > 0:
					sentTS = system.date.now()
					result = system.tag.writeBlocking(['[default]Printers/MPERIA/Writable'],[msg],1000)

Command-Protocol-Manual-11.2.x.pdf (407.1 KB)

I think you want to send your data to the WritableBytes tag, not the Writeable tag, which does some automatic encoding of data. If you’re trying to send specific data, that’s your best bet.

I changed all of the chr() statements to unichr() and changed the final statement to write the tag as
u = msg.encode(‘utf-8’)
result = system.tag.writeBlocking([’[default]Printers/MPERIA/Writable’],[u],1000)
and now I see a rectangle in the printer instead of the ‘?’

A rectangle on the actual output label? That would (to me) indicate that the printer doesn’t have a font to use to render the character.

I assume you are correct.
I’m going down the road you suggested for the writablebytes tag. When I created the device I was on an older version that did not have that feature so I didn’t even know that was an option. I’ll figure that out and mark as solution if it works. Thanks.

1 Like

I am writing to the WritableBytes tag and I am getting the same results as before.
I am converting to bytes using this:
def toBarray(string):
from java.lang import *
from org.python.core.util import StringUtil

#Convert string into a byte array
bArr = StringUtil.toBytes(string) #Read string into byte array
return bArr

I received a suggestion from MPERIA:

Can you try this in Hex?

1B 76 61 72 5F 73 65 74 1C 76 61 72 31 1C E2 86 91 17
Seen as ascii:
var_setvar1 ↑

Here is a convert from u+ (2191) to UTF8

I cannot figure out how to send his string of hex.
I do know that when I put var_setvar1 ↑ (with the ascii commands send as chr equivalents, ie unichr( 27) for )
I get
array(‘b’, [27, 83, 69, 84, 95, 86, 65, 82, 28, 118, 97, 114, 49, 28, 63, 23])

is it possible to send the hex version to the writeablebytes tag?

Again, when I put in the strings to this forum they look fine until I post it. Here is what the blurb from MPERIA actually looks like:

try using the array library

import array

stringIn = 'ABCD ↑'

# For display purposes. Printing the array will return decimals.
hexValues = [elem.encode('hex') for elem in stringIn]

bArray = array.array('B', stringIn)

print hexValues
print bArray


['41', '42', '43', '44', '20', 'e2', '86', '91']
array('B', [65, 66, 67, 68, 32, 226, 134, 145])
1 Like

That may have put me on to something.
This reflects how I am adding the arrows to my strings and the results I get using your code:
import array
a = u’\u2191’
strIn = 'ABCD ’ + a
print strIn
hexValues = [elem.encode(‘hex’) for elem in strIn]
print hexValues

[‘41’, ‘42’, ‘43’, ‘44’, ‘20’, ‘e28691’]

I’m not sure if you just used the keyboard shortcut to type your up arrow, but that is not an option for this application. It appears the way I am doing it is causing an error.

Try wrapping your string with a bytes() function

import array

stringIn = bytes(u'ABCD \u2191')

hexValues = [elem.encode('hex') for elem in stringIn]

bArray = array.array('B', stringIn)

print hexValues
print bArray

That worked.
Now I know how to recreate the hex code they want me to send which is good progress but I don’t seem to be able to send that to the WriteableBytes tag of my TCP device.

Thanks Jordan, your suggestion led to the solution. I ended up not needing the hex values at all.
I was sending my strings to be printed to a script to turn them into a byte array.
def toBarray(string):
from java.lang import *
from org.python.core.util import StringUtil

#Convert string into a byte array
bArr = StringUtil.toBytes(string) #Read string into byte array
return bArr

I was sending them with the line
bArr = MPERIA.toBarray(msg)
I changed it to this:
bArr = MPERIA.toBarray(bytes(msg))

now it works. It seems strange to need to change a string to bytes before sending it to a script I found that is supposed to make bytes itself but I’m not poking it with a stick.

Unicode can be annoying. I’ve spent many hours pulling my hair out over similar stuff. :roll_eyes: