Best Practices for SVN/GIT on projects

Ignition
1

I am now experimenting with keeping my data/projects directory as a git repository, similar to described in this thread.

It’s still a bit of a bumpy process, but it definitely feels more precise (i.e. safe) than the prior methods of maintaining project history.

One question I’m still trying to figure out is if there are any sub-directories or files that we should add to the .gitignore files. At the moment, I’m suspicious of the conversion-report.txt file and the .resources directory. Can anyone provide guidance on this topic, please?

3 Likes
2

Hmm. Yes, ignoring conversion-report.txt and the .resources folder is a good start.

I can’t think of any others off the top of my head.

3

So, I’m piecing together a project update gateway event script. Here’s the first working revision:

projects_directory = '/var/lib/ignition/data/projects'

def project_updated():
	current_gateway = system.tag.readBlocking(['[System]Gateway/SystemName'])[0].getValue()
	current_project = system.util.getProjectName()
	update_message = 'Updated {g}:{p}'.format(g=current_gateway, p=current_project)
	print(update_message)
	add_command = 'git -C {pd} add .'.split(' ')
	add_command[2] = projects_directory
	system.util.execute(add_command)
	commit_command = 'git -C {pd} commit -m {msg}'.split(' ')
	commit_command[2] = projects_directory
	commit_command[5] = update_message
	system.util.execute(commit_command)

Yes, my parameter replacement in the command strings has hard-coded indices at the moment. I was originally doing format() before split(), but bumped into issues with it splitting my commit messages. This is just my quick-and-dirty fix at the moment.

Some questions I have:

  • Is that a good way to access git? I don’t want to have to install/maintain any libraries, so I suspect that means I’ll be calling the command line git to do the work.
  • Is there some way to auto-detect the projects directory path? I couldn’t find anything with a few minutes of searching the manual and forum.
  • Is there any way to grab more info about the project change that triggered the event? I would like to have information like which user triggered the save and which project components were updated.

EDIT: Something isn’t quite right with this script since the commit doesn’t always happen. I’m guessing the calls to system.util.execute don’t block for the command to finish. Is that possibly the issue?

4

What you’ve got there is essentially “correct”, for the limited implementation of the project update script we currently have - we’re definitely planning to revisit in future, and add some of the “obvious” information like project name, modification actor (if available! remember that changes could come directly from the filesystem), etcc.
I think you’ll need to use something like Python’s builtin subprocess module (eg, subprocess.call()) rather than system.util.execute() to make the git call block.

5

Interesting chasing-my-own-tail moment: I just did a git pull in the projects directory of one server to bring it in sync with the other, which changed some files, which triggered the project change event, which then tried to add/commit everything!

In this particular case it was okay since the projects directory was in sync with a commit (i.e. the add/commit both did nothing). But, I’m pretty sure there would be some very ugly consequences in many other cases…

Let this be a warning to any other people following this path: don’t enable a project change script like I have above if you will be doing any work in the projects directory that could ever possibly leave the git repo in an incomplete state. I can’t imagine the chaos this would cause if I were doing a git merge in the projects directory and hit a conflict.

I imagine a reasonable solution would be a lockout memory tag that indicates that git add/commit should be disabled for the time being. A developer would turn it on before doing git work on the command line, and the script above would do nothing when it was set. This would allow the setting to be changed without changing the project properties.

1 Like
6

Incidental nice features: switching from system.util.execute to subprocess.call will include the output of the git command in the wrapper log, and switching from print to a system.util.getLogger object also adds in more hints about the source of the trigger.

INFO   | jvm 1    | 2019/11/27 16:54:11 | I [c.revision_helper             ] [16:54:11]: Updated ignition1:customer project-name=customer, request-origin=1.2.3.4, session-user=jbuser, session-project=customer, session-id=34947258
INFO   | jvm 1    | 2019/11/27 16:54:11 | [master e78f6c9] Updated ignition1:customer
INFO   | jvm 1    | 2019/11/27 16:54:11 |  4 files changed, 11 insertions(+), 8 deletions(-)
I
7

Hmm, you could try something like this to get your own access to that meta-information. It’s not guaranteed to be available, but you can try:

from org.slf4j import MDC
user = MDC.get("session-user")
if user is not None:
# you have an actual username
8

Should probably use org.slf4j.MDC instead.

1 Like
9

In case anyone lands here in the forum first, the IA team just recently put out an excellent whitepaper discussing issues and methods related to deploying Ignition on revision control like git:

10

Hi everyone! Future me, leaving more breadcrumbs of knowledge for others who follow…

As tempting as it may be, do NOT add resources.json or thumbnail.png to your gitignore, both will cause things to break. (tested circa Ignition 8.1.7)

Ignoring resources.json files will cause (sometimes cyclical) thrashing of the project files whenever you sync to a new checkout, but more importantly will totally break the newer method of saving named SQL queries to disk.

Ignoring thumbnail.png files would’ve been nice to avoid putting binaries in git, but the first time you update a server and gain views without those files, your log will be full of lines like this:

Error creating digest for resourceManifest=data/projects/Support/com.inductiveautomation.perspective/views/Page/Monitoring/resource.json, file=thumbnail.png 
Error reading data for file thumbnail.png for resource at com.inductiveautomation.perspective/views/Page/Monitoring 
###Fatal-An error occurred while reading project resources from disk. Disk-to-memory synchronization will not occur!

I hope that maybe IA can resolve the second and allow Ignition to sync up projects from disk without PNG files, but we’ve gotta leave them in for now.

11

No, we have no intention of doing this.

A resource is comprised of everything inside the resource directory. You should ignore nothing.

We are, however, likely going to stop writing back an "externally modified" entry to the resource upon scanning it and noticing the signature is incorrect. We'll just let it be wrong in the file and the in-memory / UI representation will still indicate there is a mismatch when necessary.

2 Likes
12

I for one love this. Our devs commit to Azure dev ops and I get the thumbnail of the view along with the code diff. Lots of time I don't even need to open the designer to review changes. Just signoff right from the git commit.

13

Any news on the gateway event script? I have been looking for a script to run on gateway (ignition docker filesystem repo + git) for updating GitLab on save etc.

This is not documented in the Best Practice etc, only showing how to work with git on local PC.

Best regards
mwd123

14

What are you looking for, exactly?
The project update script was updated in 8.1.14 to have additional contextual information; actor and resources:
https://docs.inductiveautomation.com/display/DOC81/Gateway+Event+Scripts#GatewayEventScripts-UpdateScript

15

What I was looking for actually was this:

logger = system.util.getLogger("GW_UpdateScript")
logger.info("Running git update!")
project = system.util.getProjectName()
path="/usr/local/bin/ignition/data/projects/"
import time
try:
	system.util.execute(["git","config","--global","user.name","XXX - XXX"])
	system.util.execute(["git","config","--global","user.email","XXX@XXX.com"])
	system.util.execute(["git","-C",path+project,"add","."])
		time.sleep(5)
	dateStr=str(system.date.now())
	system.util.execute(["git","-C",path+project,"commit","-m","Designer save @ "+dateStr])
	time.sleep(5)
	system.util.execute(["git","-C",path+project,"push"])
	logger.info("git update done!")
except Exception as e:
	logger.infof("error, errorMsg: \n",e)

Edit: Now added system.util.getProjectName and it seems to work fine.

16

You are taking your chances with system.util.execute. I don't think it waits for the process to run, so those three commands could execute out of order. Use java's ProcessBuilder to ensure things are sequential (and you can check for success).

{ Also, please format your code properly. Edit your comment, highlight the code, and click the </> button. }

1 Like
17

Fixed the code highlight, thanks :slight_smile: new to the forum and sadly don't have enough coding experience for doing anything else than a simple bash-script :frowning:

But with that in mind I assume its smarter to call the bash-script instead. Will change to that! Thanks for pointing this out.

18

Oh, and if you make sure things are sequential, you won't need to use sleep(), which is evil and should never be used in Ignition. /:

19

time.sleep(5) is in the Best Practices from IA for using git.
Sadly I don't know how to code it on my own, abit surprised that there is no real good working example that covers what I do in the script I posted earlier.

20

Oy!

Since I don't have to do this myself, I hadn't run across that. IA should fix that. :frowning_face: