Performance issues

Hello,

I’ve been running the client performance tool and haven’t seen anything really. Except the fact that screen also stops working when I’m having the issue.

I have a project that is relatively small. 16 - 18 main screens and about the same number of popups. I have two screens that are using the Template Canvas component to show dynamically generated scrolling list of template components. They use some predefined UDTs to build the list of components. If I switch to and from these screens I will fairly quickly get to a point to where the client slows down to a halt. I can’t even close it. When I started writing this post I had pressed the button to switch to one of the less complicated windows and it still hasn’t shown. The circle in the upper right corner is there doing nothing, The Performance monitor page is stalled as well.

In Task Manager the java instance is using lots of CPU, and PageFaults, relatively little memory but growing, Threads and handles are remaining about the same.

CPU > 70%, Private Working Memory 534MB, Page Faults 1750424, Handles 740, Threads 51, I/O Read 62657, Write 72936, Other 755149

None of the pages cache so that I can do initialization code that NEEDS to be done every time the window is opened, but not every time the window is exposed like building the template list.

I was hoping that somebody could offer some suggestions.

It’s probably something silly I’m doing…

Thank you,
Bill

Are you swapping between the windows or opening multiple instances of the same window? It’s possible you have the windows opened multiple times, all running the same code and bogging it down. Try using system.nav.swapTo() or system.nav.swapWindow().

https://docs.inductiveautomation.com/display/DOC79/system.nav.swapTo

The most common reason I’ve seen for such problems is insufficient discipline in propertyChange events. Usually a missing if event.propertyName=='something': check or an assignment to a property that triggers a recursive propertyChange. Add print statements to all of the events in the troublesome templates and windows to see what’s happening in the diagnostic console.

pturmel, – Boy you’re being tough on me today.

First, I always check event.propertyName :wink: The propertyChange event handler for this template is below, however the template does contain other templates that have their own propertyChange event handlers.

The other thing, is that if it was because of the looping property changes I would expect that it would happen more frequently, and not just start to go slower and then get stuck. Since I sent the first message, I left the thing there and 3 hours later it was still spinning.

I would not be against the idea that there was an endless loop going on, but I’m not sure what, and the log messages that I have are not being triggered during the issue.


logger = event.source.getLogger("propertyChange")

#logger.info("PropertyChange Name = %s, Value %s --> %s" % (str(event.propertyName), str(event.oldValue), str(event.newValue)))

if event.propertyName in ( 'editor', 'componentRunning' ):
	logger.info("Component Running")
	event.source.ComponentInitialized = True

elif event.propertyName == "OnDeckRecipeNumber":
	logger.info("PropertyChange Name = %s, Value %s --> %s" % (str(event.propertyName), str(event.oldValue), str(event.newValue)))
	event.source.setOnDeckOrCurrentFlags(event.newValue, event.source.CurrentRecipeNumber)

elif event.propertyName == "CurrentRecipeNumber":
	logger.info("PropertyChange Name = %s, Value %s --> %s" % (str(event.propertyName), str(event.oldValue), str(event.newValue)))
	event.source.setOnDeckOrCurrentFlags(event.source.OnDeckRecipeNumber, event.newValue)

elif event.propertyName in ( "BinId", "FieldId", "VarietyId" ):
	if event.source.ComponentInitialized and event.oldValue == 0 and event.source.ShearSettings == 0:
		logger.info("PropertyChange Name = %s, Value %s --> %s" % (str(event.propertyName), str(event.oldValue), str(event.newValue)))
		binId = event.source.BinId
		fieldId = event.source.FieldId
		varietyId = event.source.VarietyId
		if event.propertyName == "BinId":
			binId = event.newValue
		elif event.propertyName == "FieldId":
			fieldId = event.newValue
		elif event.propertyName == "VarietyId":
			varietyId = event.newValue
		if binId > 0 and fieldId > 0 and varietyId > 0:
			logger.info("Initializing Values")
			event.source.ShearSettings = 1
			event.source.ManualTank = False
			if event.source.HaveSizer:
				event.source.RatioDiameterMode = 1
				event.source.RatioValue = event.source.DefaultRatioValue
				event.source.DiameterValue = event.source.DefaultDiameterValue

elif event.propertyName == "SelectBox":		# Check Box Indicator Field (Somebody else changed it)
	if event.source.SelectRecipe != event.newValue:
		logger.info("PropertyChange Name = %s, Value %s --> %s" % (str(event.propertyName), str(event.oldValue), str(event.newValue)))
		event.source.SelectRecipe = event.newValue
		
elif event.propertyName == "SelectRecipe":	# Check Box Control Field (Checkbox was clicked or pressed)
	if not event.newValue:	# Transition from True to False
		if event.source.IsOnDeckJob:
			logger.info("PropertyChange Name = %s, Value %s --> %s ==> Remove from On Deck" % (str(event.propertyName), str(event.oldValue), str(event.newValue)))
			event.source.SelectBox = False	# The propertyChange for SelectBox will clear the On Deck Recipe for me (I hope)
	else:
		if not event.source.IsOnDeckJob and not event.source.IsCurrentJob:
			if event.source.OnDeckRecipeNumber <= 0:
				logger.info("PropertyChange Name = %s, Value %s --> %s ==> Put On Deck" % (str(event.propertyName), str(event.oldValue), str(event.newValue)))
				def loadOnDeck(event=event):
					event.source.loadOnDeckRecipe(event.source.RecipeNumber)
					
				system.util.invokeLater(loadOnDeck, 100)
			else:
				system.gui.errorBox("Deselect the On Deck grading job before putting another job On Deck")
				def resetSelectRecipe(event=event):
					event.source.SelectRecipe = False
					
				system.util.invokeLater(resetSelectRecipe,100)
	
elif event.propertyName == "ClearRequested":
	if event.newValue:
		logger.info("PropertyChange Name = %s, Value %s --> %s" % (str(event.propertyName), str(event.oldValue), str(event.newValue)))
		def processClearRequested(event=event):
			event.source.processClearButton()
		
		system.util.invokeLater(processClearRequested, 500)

I’m using a Tab Menu component with the Navigation Mode set to “Swap to window” When I have checked the open windows, I only have the Title Window, the Navigation Window and the Current Window open. I also have all the windows set to not cache so they are closed and reopened each time.

To figure out what’s going on when the client’s locked up, grab a thread dump from the hung client/designer using jstack - then upload it/post it here and we/I can take a look at what’s going on on your system.

Heh. :grin:
No offense intended. BTW, I'd take up the offer to paw through your thread dump -- that's a pain in the $%^&*.

1 Like

Ok, first thing… I always use the client launcher with no default application. I was thinking about what was going on, and was concerned that it seemed like a java memory starvation type of problem. So I went into the launch.xml file and modified the init-heap to be 256M, the max-heap to be 2048M, then added the “-XX:MaxPermSize=128M” to the jvm-args tag. They were all empty before.

While I don’t see those commands on the command line for the java executable, I also haven’t had it bog down since those changes.

In watching the Performance Diagnostic window I see the memory usage chart go up to around 600M then drop back down to 100M while cycling through windows.

Does the idea of this being an available memory issue make sense?

I have to work on a different project for the rest of the week (LabVIEW is a pain), but next week I’ll be working on this one again, if I see similar behavior I will revisit this.

PGriffith, thanks for the information about jstack and the offer to help examine the results…

And one other thing… I am using the invokeLater method in order to reduce race conditions when I’m setting fields that are also being watched with the propertyChanged event handler.

I looked in the clientlauncher.log and saw that it was configured for -Xms32M -Xmx256M and now it is -Xms256M -Xmx2048M.

So I definitely upped the max and starting memory for the client.

My Designer sessions are/were -Xms64m -Xmx1024M and I haven’t restarted it since I made the change.

I use the clientlauncher on the HMI screens as well, but use the command line. I believe it’s already set higher there, but I have had one report from them with the HMI locking up like this, so when I’m there next week I’ll double check the settings and the available memory on the HMI.

Like I said, I’ll revisit this if the problem reoccurs, otherwise thanks for all the suggestions!

Bill

Well, I’m back at the client, and had some problems again with the client slowing down during operation.

Even with a fresh start the user is complaining that it is taking a while to load the screen I was feeling was the main culprit. This screen has a Template Canvas with 20 templates loaded, and it takes more than 3 seconds before the canvas has refreshed when the screen is loading. I changed this screen back to be cached, but that didn’t change the behavior of the canvas. I switched it from a dynamically built list of templates to a hard coded list.

Any suggestions on a way to have a scroll-able list of templates that doesn’t take a while to reload?

I didn’t get a stack trace because I couldn’t access the OS once the screen basically locked up.

Don't use any UDT properties. Replace with tagpath string properties and indirect bindings to the specific items in any UDT that a specfic template needs.

Thanks, But I’m not using the UDT structure at all for these templates. I am using Indirect Tag access for all the tags being utilized.

This template does use 6 other templates in constructing itself.

I had thought it was one of the sub-templates where it had 3 local properties that were bound to a simple Id, Name sql lookup, but I rearranged that to do InvokeLater to populate the properties and got rid of the automatic SQL query. I even went so far as to disable the SQL entirely and didn’t notice any load time difference.

All the tags except 2 are memory tags.

Total number of tags in use are about 20 per template.

And it is loading 20 templates.

Without you sharing your project, I don’t know what else to suggest.