Nightly builds are back & project resource primer

Hi everyone,

We’ve turned the nightly betas back on, and soon we’ll be opening up the beta program to the public. First I just wanted to thank you all for participating in this private beta phase, we really appreciate all the feedback. Secondly, I wanted to take a minute to explain what we’ve been up to these past few weeks and the changes you can expect now that the builds are flowing again.

We turned off the nightlies as we worked on merging in perhaps the most ambitious and important change we’ve ever made to the core of the Ignition platform. While this body of work is really hundreds of changes across every part of Ignition, I like to think of it as three distinct features that all intertwine into a broad overhaul of Ignition’s project subsystem: project storage, project inheritance, and concurrent editing.

First of all, projects are now stored directly on the filesystem of the Ignition Gateway, under your data/ directory in a subfolder called “projects”. Each project is organized as a collection of files that are organized in a coherent and predictable way. Each project resource is represented as a two or more files. One file is called resource.json and contains some meta information about the resource. The other file may be called whatever the relevant module wants it to be called. For example, perspective views are called view.json and script modules are called code.py. Any changes you make in the Designer will be written to these files, and any changes made by external forces to the files will be detected and represented in your project. There are many benefits from storing projects in this way. Resources can now be edited, migrated and synchronized externally, using any kind of tooling that deals with the filesystem. Most notably, this means that external source control tools, such as git, can be integrated. There is still some work to be done on our part for this integration to be smooth (e.g. a post-save scripting hook in order to auto-commit), but the hard part (for us) is done.

Second, projects can now inherit resources from a parent project. The organization of these inheritance links is completely under your control. By putting the inheritance design of projects in your hands, we’ve created the opportunity for dramatically more powerful and flexible gateway designs. Your project inherits all of the resources from its parent project and that project’s parent, etc all the way up the chain until it finds a project with no parent specified. Each project along the way can then define its own additional resources, as well as optionally override any resource that it was inheriting. The built-in “global” project is now gone, because its purpose has been superseded by the new inheritance system.

This change has significant consequences for several types of resources in Ignition. We have managed to maintain backwards compatibility by doing the following:

  • Anything that was in your old “global” project will be imported and put into a new project called “global”, and all your other projects will be configured with this new global project as their parent. That way, anything that was previously available to the project in the global space (scripts, templates) will still be available, because they have been inherited.
  • Scripts that existed in shared.* and project.* will continue to live there. However, it is important to realize that these are now simply standard script packages (a.k.a.folders), and are no longer managed by the system. A new install would not have these packages, though of course you’re free to continue to use this naming convention, if desired.
  • Alarm pipelines will be placed in a new “alarm pipeline” project. In general, pipelines are simply like other resources now, and may reside in any project. You’ll notice that the project name is part of the pipeline path on the tag configuration, but if the project is not specified (as it won’t be for any existing tag from Ignition 7), the system will automatically look in the special upgrade project for pipelines.
  • Executable resources, such as Transaction Groups and Run-Always SFCs, will only execute in non-inheritable ”leaf” projects.
  • There is a new setting under Gateway Settings for the “Gateway Scripting Project”. This is where you can specify a project to be used for gateway resources that live outside of projects, such as Tags. This setting will default to the global project upon upgrade, allowing tags that use the runScript function to continue to access the scripts that they previously accessed through the global project.

Third, we’ve totally redesigned how the concurrent design experience works. In Ignition 7, concurrent designers working in the same project were kept safe by means of edit locks. If one person had a resource open for editing, other designers were then locked out of it. This was somewhat problematic. When people left their designers open and went to lunch, resources stayed locked. Furthermore, sometimes it’s hard to tell when a resource is “open” and when it isn’t. When you do a search and replace, have you “opened” all the resources, thereby locking them all? Should you be able to search into a resource somebody else has locked? Most important of all, this locking strategy is completely incompatible with the new filesystem storage concept. After all, external forces may alter the filesystem at any time, outside of Ignition’s control.

Now in Ignition 8, the designer uses a totally lock-less concurrent editing strategy. The way this works is that when you save, the save also sends a signature of the resource as it was when you loaded it. This way, we can have 100% confidence whether or not another designer (or someone in the filesystem) has made change since you loaded that resource. If not - your save continues and the resource is modified. If a conflict is detected, your entire save is rejected atomically, and you are presented with a UI in the designer to resolve the conflicts. Currently, this means choosing whether to accept your local changes or the remote changes per conflicting resource. In the near future this system will be further improved with two additional features: the ability to merge your local changes with the remote changes for text-based resources, and additional information in the designer warning you before you save that you are editing a resource that someone else is also editing or has already made changes to.

Finally, outside of the changes to the project resource system, we have continued to work like crazy on many other outstanding issues. Some of the most important recent fixes include:

  • Redundancy now works again, with a newly reworked and improved state synchronization system.
  • Tag history can now be queried through a remote tag historian connection to an Ignition 7 system.
  • The gateway’s web user interface went through another major round of design improvement.
  • Many of the designer’s user interfaces that were missing in the beta have been created (e.g. Perspective’s tag history binding configuration interface).
  • You now have the ability to flatten vector images. Full drawing support is still a ways off, but in the meantime this feature is very useful to create overlays for symbols.
  • Many, many other fixes and improvements.

Thanks for your patience as we get this release ready. We hope you enjoy these new features.

26 Likes

Wonderful update. I think an emerging strategy is to install Ignition 8 on a new server and have it connect to an existing Ignition 7 server and subscribe to the tags and history,etc. This is because multiple sites may have local Ignition servers for machine control and these maybe large installs that have been updated over multiple Ignition versions. Can you provide somefeedback about how this senario will work? Is it the same as connecting a remote tag provider like we currently do and how would an Ignition 8 project allow the user to redirect to an Ignition 7 project?

We imagine this will be common as well (using Perspective as the visualization layer). It would be done as you guessed - with remote tag providers.

This isn’t going to be possible. Vision re-targeting won’t work between Ignition 7.x and Ignition 8.x.

Absolutely fantastic! Toys, toys, toys! And it’s not even my birthday!

Seriously, though. Source control has been on my wish list for a long time.

3 Likes

You guys are making America great again!

4 Likes

Seriously, though. Source control has been on my wish list for a long time.

Yeah, it’s a big deal, and I wanted to mention that before the final release we hope to get most resources into readable text formats (not vision, unfortunately) and add some more advanced project lifecycle hooks to try to allow you to script automatic commit/push, etc.

You guys are making America great again!

Haha… thanks?

3 Likes

Good stuff. Thanks for the update.