New in 8.1.28: Custom Touchscreen Keyboard Support

[PGriffith]

Introducing a (long awaited) new feature for Vision that should help a specific subset of users a whole lot; totally custom touchscreen keyboards. Introduced primarily as an entrypoint for international support, this feature morphed a bit during development into something a bit more powerful. So, how does it work?

As a developer, your introduction to this feature will be the new Keyboard Layout editor accessible via the Tools menu in the Designer (while you're in the Vision workspace). This new tool window allows you to create, modify, and import/export custom keyboard layouts across your entire gateway:

You can also hit the preview button to get an instant, live preview of the currently selected keyboard layout:

Which is a good introduction to the end user facing portion of the feature - the brand new, totally reworked touchscreen keyboard.
Some obvious and not so obvious changes:

• The new layouts, including a new english layout, are greatly simplified to present a more mobile-like appearance.
• The new keyboard dialog is much smarter about how much screen real estate it uses, and how it's presented.
• From every keyboard layout, you can switch to a numeric/symbols layout that's common across all languages:
  
  
• Accent keys are now supported! Long-press on any key with accent characters defined and a popup window will open allowing you to enter any alternate variations of that key:
  
  • For anyone concerned about long presses on their particular HMI/touch panel, there are two 'escape hatches' for this new feature;
    1. The long press delay (defaulting to 1000ms) can be configured per-client with the ignition.touchscreen.longPressDelay system property; any value <= 0 will disable the long press behavior entirely.
    2. The 'hand/finger' button at the bottom right of the keyboard acts as a 'sticky' press, and automatically makes the next key clicked invoke the accents popup:
       
• To make it easier to remove any existing text in the field, there's now a clear all button at the end of the preview text field
• To make navigation easier within the text body, Shift + Arrow Keys will jump to the start or end of the current text, and the arrow keys can be held (with the same configurable initial long press delay) to repeatedly navigate within the text field.
• Similarly, the backspace key can be held down to repeat.

Now, for the best part: Very quickly in development, we realized that it simply wasn't practical for IA to offer every possible keyboard variation our users might expect. So, while we are now providing new installs of Ignition with Spanish, French, Italian, and German layouts, we also made these keyboard layouts totally freeform and configurable. There are no significant restrictions placed on the number or location of keys, which we hope will lead to some interesting possibilities. Keyboard layouts are configured in a simple JSON scheme that we hope will lead to publicly available alternative layouts hosted on the Ignition Exchange, or on these forums, or elsewhere in the community.
The default provided layouts (as of the initial release of this feature) are available below:

• fr_azerty.json (4.6 KB)
• es_qwerty.json (4.6 KB)
• de_qwertz.json (4.0 KB)
• it_qwerty.json (4.2 KB)

As well as the two new English layouts; one "modern" and one designed to closely emulate the legacy keyboard to ease the transition for your end users:

• en_us_compat.json (6.0 KB)
• en_us.json (4.2 KB)

In addition to these examples to start from, we have a well-formed JSON schema that defines the structure of a keyboard layout, which can be integrated with external editors to help create keyboard layouts outside of Ignition; this schema can also be exported from the Designer using the {S} button at the bottom of the layout list.

• keyboard_schema.json (7.1 KB)

The actual structure of a keyboard layout should hopefully be fairly self explanatory, though:

{
  "alias": "DE",
  "id": "73a221f3-e2de-4b5b-97a4-cecc32cc14ad",
  "labels": {
    "BACKSPACE": "Backsp",
    "CANCEL": "Cancel",
    "SHIFT": "Shift",
    "SPACE": "Space",
    "SUBMIT": "Enter"
  },
  "name": "de_qwertz",
  "rows": [],
  "supportedLanguages": [
    "de"
  ],
  "title": "Deutsch"
}

From top to bottom:

• alias is the short name to display on the keyboard layout switcher button
• id is a randomly generated UUID that will be automatically updated for you to keep track of layouts as they are updated on the gateway.
• labels is an object where certain recognized keys can be used to localize your keyboard layout. All system keys can be given an alternate name via the labels object; any keys not present will be given default English labels.
• name is a programmatically-friendly name for the keyboard layout, used in a few scenarios detailed below.
• rows is an array of arrays of keys, which can be structured in any of these three ways:
  • A string literal member of the enum "BACKSPACE", "SUBMIT", "SHIFT", "CANCEL", "CAPSLOCK", "NUMERIC_LAYOUT" will create a system key at that location
  • A simple key can be created with an object with at least one member lowercase:

    {
      "lowercase": "a"
    }
    

    Add an uppercase key to define an alternate form to use when the shift or capslock keys are active.
  • Or, more complex keys can be defined with a full object pattern:

    {
      "lowercase": {
        "character": "e",
        "accents": ["ê", "é", "è", "ë", "ė"]
      },
      "uppercase": {
        "character": "E",
        "accents": ["Ê", "É", "È", "Ë", "Ė"]
      }
    }
    

    Keys can also be defined with an alias, allowing the typed 'character' to be different than the label for the key:

    {
      "character": "\t",
      "alias": "Tab"
    }
    

    A few additional notes:
    • There are no restrictions on where system keys are placed. They can go in the middle of a line, be repeated on each end, or whatever your heart desires.
    • There are no limitations on how many characters a single key can "press" in one operation, and no restrictions that the uppercase or lowercase variants of a key have to be related in any way.
• supportedLanguages is an array of IETF language tags that this keyboard claims to support.
• title is a human-friendly (localized) string that will be used on the layout switching dialog.

Within this structure of rows of keys, there are essentially no restrictions; a keyboard layout can be as simple

Or complex

as you like (though, you likely won't want to go over 4 rows of keys, just for the sake of screen real estate).

So, you've got all these new keyboard layouts, how do you actually take advantage?
Well, for the majority of cases, hopefully, you simply upgrade, import or create layouts for whatever languages you need to support, and that's it. Everywhere the touchscreen keyboard is automatically invoked by Ignition, a check will be made first to see if there's a keyboard layout that supports the current system user's locale. If one is found, it will be returned preferentially. If you did want to invoke a particular keyboard layout, there are two options:

1. From scripting, with the new system.gui.showTouchscreenKeyboard() function that accepts keyword arguments:
   system.gui.showTouchscreenKeyboard("abc", fontSize=24, layout="de_qwertz")
   • As a corollary, there's a new system.vision.getKeyboardLayouts() function that returns the underlying KeyboardLayout objects; this can be used to retrieve all of the keyboard layouts on the current system, for instance:
     [layout.name for layout in system.vision.getKeyboardLayouts()]
2. Within Vision, every component with touchscreen keyboard support has a new property, 'Touchscreen Keyboard Layout', which can be set (or bound to) the name of any keyboard layout on the system (or left blank for the default, locale-aware behavior):
   

[meredithj]

This looks awesome. Are we able to use any Unicode characters / emojis?

[PGriffith]

Yes, any Unicode character will work. Most emoji don't currently render in our embedded font, but we're treating that as a bug. The characters do still insert into the text field, though. Other characters such as most of the Chinese, Japanese, Korean language families will need you to install additional font extensions manually, which is documented in the manual.

[jaydeepcheruku]

This is great @PGriffith but quick question, is there an easy way to set default layout to "en_us_compat" instead of "en_us" without going to each text/numeric field and changing "Touchscreen Keyboard Layout" property? Meaning option to change default layout at gateway or even at project level. Even a way to switch the default keyboard layout order to have "en_us_compat" first is ok.

[PGriffith]

No, not currently. As a short term stopgap, you could go through windows in the designer and recursively apply the layout you want using adapted code from this post:

Upgrade version makes buttons dissapear (possible bug?)

For anyone finding/being linked to this thread in the future; if you want a ‘middle ground’ solution between a global UI manager key and setting every button up with scripting, you could use a technique like this to recursively modify every button in the window (e.g. in visionWindowOpened): from javax.swing import AbstractButton from com.inductiveautomation.ignition.client.IgnitionLookAndFeel import setBlendMode from de.javasoft.plaf.synthetica.util.Synthetica2DUtils import BlendMode def getCo…

from com.inductiveautomation.vision.api.client.components.touchscreen import TouchscreenMouseListener

def getComponents(window):
	def walkComponents(component):
		for component in component.components:
			yield component
			walkComponents(component)
	
	return walkComponents(window.rootContainer)
	
isTouchscreenComponent = lambda component: isinstance(component, TouchscreenMouseListener)

def setKeyboardLayout(component, layout):
	component.keyboardName = layout

win = system.gui.getWindow("WindowName")
for component in filter(isTouchscreenComponent, getComponents(win)):
	setKeyboardLayout(component, "en_us_compat")

[jaydeepcheruku]

Thanks a lot @PGriffith This works for now.

[Andrew_Collier]

This is a great new feature, and makes restricting data entry so much easier. One question though, are there any plans to make the Space and the !123 buttons removable as an option?

[PGriffith]

That's an interesting thought. It hadn't really come up in our internal brainstorming about this feature, but it definitely makes sense. I'll make a note of it, but I can't guarantee timely implementation

[Dave_Forbes]

I exported the en_us_compat and then imported it as a custom keyboard with a different name. Now all the clients default to the custom one.

[c.turmel]

Is there anyway to change the font size of the keyboard layout?

[PGriffith]

As of 8.1.34, there's two ways to customize the font size.
Across your entire project, via the new project property:
https://docs.inductiveautomation.com/display/DOC81/Vision+Project+Properties#VisionProjectProperties-VisionUserInterfaceProperties
Or locally on a particular client by providing an override in the Vision Client Launcher's advanced configuration:
https://docs.inductiveautomation.com/display/DOC81/Vision+Client+Parameters#VisionClientParameters-KeyboardFontScaling