There are several posts about displaying Cognex images and overlays on the ignition screen. There are a few people selling modules and there is the popular web module that can display the SVG itself. We have used the web module in the past and it sort of works for what we want. I don’t like it because its another module that must be purchased and we’ve had some flaky stuff happen on Ignition upgrades.
I thought I would share my current approach and maybe it helps some others that might be asking the same question down the road.
First, the application: We have a new machine that has 4 Cognex Insight 7000 series cameras doing something undisclosed for IP reasons. To help support the remote site that will be running this new machine, we would like to store the images for a period of time before removing them, along with key information used to run the part (ID, offsets, angles, and other values from the PLC and Cognex cameras). We would also like to display the images to the operator on the HMI.
Next, the setup: We are running an CompactLogix PLC with a PC running a 2 client license of Ignition (there other reasons why we did not use the sites main Ignition server). On this PC we also have a SQL express instance running (also, other reasons why we chose not to use the sites SQL Cluster). Essentially, we chose to have this machine entirely stand-alone.PC also has IIS FTP setup or Filezilla (both tested and both work, I like IIS because i don’t like any more 3rd party software than I must have).
The operation:
-
As parts enter the machine Cognex Insight cameras aquire an image and pass variables to the PLC via Ethernet/IP.
-
The camera also uploads a copy of the image and overlay data to an FTP.
-
Gateway script runs every 500 ms to check FTP location and if file is present, inserts the bytes into a SQL table. I chose to use a SQL table to hold the binary data because the files are relatively small less than 1MB and I can hold a lot of them before it makes a noticeable impact on the SQL database size. I also feel it is easier to manage the records this way that in a file system. One quick Delete statement and I can clear out all the old records. I chose to do this as a gateway script because whether the client is open or not, the images are being stored.
-
In addition, the SVG file is stored in plain text in a varbinary(max) field in the same record with the image. The DB is also storing date-time info, flags for retaining images (preventing deletion), etc.
-
The paintable canvas has a custom property that has the current picture ID from the DB. The overview page that has the Paintable Canvas has a timer that runs every 500ms to check to see if there is a new record in the database. If a new record exists, the ID is updated and a call to a new extension function called “Draw” on the paintable canvas is called. Yes, I could have the GW script update a gateway tag that would trigger a Draw to remove the timer, and I might still, but its working for now.
-
The Draw function on the PC loads the image into the cache-img and also takes the SVG data in the DB and parses it into a dataset custom property on the PC, called Overlay Data. Another post made reference to how big the SVG standard was and maybe while true, Cognex utilized only a few primitive shapes from the SVG standard and they do it very consistently. The code below will not load any SVG, but should (and is for me) working with Cognex. SVG files are xml-type file that hold the overlay information. I chose to use the elementTree libraries and regular expression libraries that are included in Ignition to parse the SVG files.
-
Note: With Cognex version 4.8 and higher, the overlay graphics cannot be saved in the same file as the image. One major reason for this is that if you want to store the image and re-run it on an emulator offline to diagnose problems, you want the raw image without the pesky overlay messing up the evaluation, hence the reason to have the raw image and all the overlay graphics separate.
-
When the PC has it’s repaint event triggered, it reloads the cached image and the OverlayData custom property and draws those images. It was very important to not have any SQL calls or files being read as the repaint event needs to happen very quickly. Use of the cached image and the overlay dataset minimize the time required to process the repaint event.
I hope this helps to anyone thinking of trying the same thing. I like this approach because it means i do not have to purchase and maintain an extra module or 3rd party module and it uses native functions and Java 2D graphics. It also gives the user a little more control over how and what are displayed, but it does require the developer to understand a little more about how things like SVG graphics and Java 2D graphics work.
Anyone can feel free to roast me if they see an issue with the solution or if you have a potential improvement, please pass it along.
With the pervasive nature of Vision in industrial applications and with Cognex being the largest global manufacturer of machine vision, I think it would be nice to see a component from Ignition to display and interact with these devices. The solutions out in industry to allow the operators and technicians view images in realtime leave a lot to be desired and require having a computer hooked up running their programming software or purchasing an expensive monitor like the Cognex VisionView appliance or the add-on for a Rockwell panelview (where the user has to exit the Rockwell software to windows CE and launch the Cognex software). It creates a muh richer and meaningful user experience if the images and data from the Cognex cameras are able to be displayed in an Ignition window and images stored for troubleshooting.
This method also opens the opportunity to do some crazy stuff in the future, like the vision fails to find a feature, the user could click on the paintable canvas and the X,Y location can be transformed back to camera coordinates and used as an alternate method to find a pattern.
Draw extension function on Paintable Canvas:
def Draw(self):
def Run(s=self):
s.Refresh = s.Refresh +1
def checkAttrib (attributes,strname):
val = None
if strname in attributes.keys():
val = attributes[strname]
return val
def parseSVG (data, tag, attrib,text):
tag = tag.lower().strip()
# print attrib
if tag == 'text' or tag == 'circle' or tag == 'line' or tag == 'rect' or tag == 'path' or tag == 'polyline':
x = checkAttrib(attrib,'x')
y = checkAttrib(attrib,'y')
stroke=checkAttrib(attrib,'stroke')
x1 = checkAttrib(attrib,'x1')
y1 = checkAttrib(attrib,'y1')
x2 = checkAttrib(attrib,'x2')
y2 = checkAttrib(attrib,'y2')
h = checkAttrib(attrib,'height')
w = checkAttrib(attrib,'width')
fill = checkAttrib(attrib,'fill')
StrokeW = checkAttrib(attrib,'stroke-width')
d = checkAttrib(attrib,'d')
r= checkAttrib(attrib,'r')
cx= checkAttrib(attrib,'cx')
cy= checkAttrib(attrib,'cy')
transform = checkAttrib(attrib,'transform')
style = checkAttrib(attrib,'style')
id = checkAttrib(attrib,'id')
points = checkAttrib(attrib,'points')
if text == None or text == '':
text = None
font = None
fontsize = None
if tag == 'text':
params = style.split(";")
for param in params:
lstparam = param.split(":")
if lstparam[0] == "fill":
stroke = lstparam[1]
elif lstparam == "font-family":
font = lstparam[1]
elif lstparam[0] == "font-size":
fontsize = int(lstparam[1].replace("pt","").strip())
row = [tag,x,y,x1,y1,x2,y2,h,w,StrokeW,stroke,d,fill,r,cx,cy,transform,style,id,points,text,font,fontsize]
data.append(row)
return data
from java.io import ByteArrayInputStream
from javax.imageio import ImageIO
from java.awt import Image
from java.awt.image import BufferedImage
from java.awt import Color
print "Draw"
canvas = self
Error = 0
image = system.db.runScalarPrepQuery("select image from images where image_id = ?",[self.ID])
if Error == 0 and image <> None:
#set canvas properties
bais = ByteArrayInputStream(image)
bImageFromConvert = ImageIO.read(bais)
canvas.putClientProperty("img-cached",bImageFromConvert)
ID = self.ID
overlay = system.db.runScalarPrepQuery("select SVG_Overlay from images where image_id = ?",[ID])
headers = ["tag","x","y","x1","y1","x2","y2","h","w","StrokeW","stroke","d","fill","r","cx","cy","transform","style","id","points","text","font","fontsize"]
data=[]
import xml.etree.ElementTree as ET
root = ET.fromstring(overlay)
#for elem in root:
# print elem
for child in root:
tag = child.tag[child.tag.find("}")+1:]
data = parseSVG (data, tag, child.attrib,child.text)
for x in child:
tag = x.tag[x.tag.find("}")+1:]
data= parseSVG (data, tag, x.attrib,x.text)
SVGData = system.dataset.toDataSet(headers, data)
self.OverlayData=SVGData
system.util.invokeLater(Run,100)
else:
image = BufferedImage(300,300,BufferedImage.TYPE_BYTE_GRAY)
graphics = image.createGraphics()
graphics.setPaint(Color( 255,255,255))
graphics.fillRect ( 0, 0, image.getWidth(), image.getHeight() )
canvas.putClientProperty("img-cached",image)
headers = ["tag","x","y","x1","y1","x2","y2","h","w","StrokeW","stroke","d","fill","r","cx","cy","transform","style","id","points","text","font","fontsize"]
data=[]
SVGData = system.dataset.toDataSet(headers, data)
self.OverlayData=SVGData
system.util.invokeLater(Run,100)
Repaint event on Paintable Canvas
from java.awt import Color
from java.awt import GradientPaint
from java.awt.geom import GeneralPath
from java.awt.geom import Rectangle2D
from java.awt.geom import Ellipse2D
from java.io import ByteArrayInputStream
from javax.imageio import ImageIO
from java.awt import Image
from java.awt import RenderingHints
from java.awt.image import BufferedImage
from java.lang import Math
from java.awt.geom import AffineTransform
from java.awt.geom import Line2D
from java.awt import BasicStroke
from java.awt import Font
import re
def getColor(strColor):
import system
try:
h = strColor.lstrip('#')
RGBColor = tuple(int(h[i:i+2], 16) for i in (0, 2, 4))
except:
RGBColor = (0,0,255)
return system.gui.color(RGBColor[0],RGBColor[1],RGBColor[2])
image = event.source.getClientProperty("img-cached")
print "repaint"
if image <> None:
g = event.graphics
#g.setRenderingHint(RenderingHints.KEY_ANTIALIASING,RenderingHints.VALUE_ANTIALIAS_ON)
#Get details of paintable canvas
cwidth = event.source.getWidth()
cheight = event.source.getHeight()
scale = AffineTransform()
#Get details of paintable canvas
iwidth = image.getWidth()
iheight = image.getHeight()
bounds = Rectangle2D.Float(0,0,iwidth,iheight)
boundsNew=scale.createTransformedShape(bounds).getBounds2D()
xr = cwidth/boundsNew.getWidth()
yr = cheight/boundsNew.getHeight()
if xr<yr:
rr = xr
else:
rr=yr
if rr> 1.00:
rr = 1.00
scale.scale(rr,rr)
boundsNew=scale.createTransformedShape(bounds).getBounds2D()
yo = (cheight -boundsNew.getHeight())/2
xo = (cwidth -boundsNew.getWidth())/2
print xo,yo
t = AffineTransform()
t.translate(xo,yo)
t.concatenate(scale)
g.drawImage(image,t,event.source)
print t
############### DRAW SVG DATA
SVGData = event.source.OverlayData
g.setTransform(t)
line = Line2D.Float(0,0,50,50)
g.setStroke(BasicStroke(10))
g.draw(line)
for row in range(SVGData.rowCount):
if SVGData.getValueAt(row,"tag") == 'line':
g.setColor(getColor(SVGData.getValueAt(row,"stroke")))
x1 = float(SVGData.getValueAt(row,"x1"))
y1 = float(SVGData.getValueAt(row,"y1"))
x2 = float(SVGData.getValueAt(row,"x2"))
y2 = float(SVGData.getValueAt(row,"y2"))
line = Line2D.Float(x1,y1,x2,y2)
g.setStroke(BasicStroke(int(20)))
g.draw(line)
if SVGData.getValueAt(row,"tag") == 'circle':
g.setColor(getColor(SVGData.getValueAt(row,"stroke")))
x = float(SVGData.getValueAt(row,"cx"))
y = float(SVGData.getValueAt(row,"cy"))
r = float(SVGData.getValueAt(row,"r"))
circle = Ellipse2D.Float(x-r,y-r,r+r,r+r)
g.setStroke(BasicStroke(int(SVGData.getValueAt(row,"StrokeW"))))
g.draw(circle)
if SVGData.getValueAt(row,"tag") == 'text':
g.setColor(getColor(SVGData.getValueAt(row,"stroke")))
g.setFont(Font("Ariel", Font.PLAIN, int(24)))
x = float(SVGData.getValueAt(row,"x"))
y = float(SVGData.getValueAt(row,"y"))
text = SVGData.getValueAt(row,"text")
g.drawString(text,x,y)
if SVGData.getValueAt(row,"tag") == 'polyline':
g.setColor(getColor(SVGData.getValueAt(row,"stroke")))
g.setStroke(BasicStroke(1))
# Get Transform and Rotate
TR = SVGData.getValueAt(row,"transform").strip()
loc = TR.find("rotate")
Transform = TR[:loc].strip().replace('translate(','').replace(')','')
Rotate = float(TR[loc:].strip().replace('rotate(','').replace(')',''))
Tx = float(Transform.split(' ')[0].strip())
Ty = float(Transform.split(' ')[1].strip())
t2 = AffineTransform(t)
t2.rotate(Math.toRadians(float(Rotate)),Tx,Ty)
t2.translate(Tx,Ty)
g.setTransform(t2)
poly = GeneralPath()
points = SVGData.getValueAt(row,"points")
i=0
for j in points.split(' '):
t4 = j.split(',')
if i == 0:
poly.moveTo(int(t4[0]),int(t4[1]))
else:
poly.lineTo(int(t4[0]),int(t4[1]))
i+=1
poly.closePath()
if SVGData.getValueAt(row,"fill") != None:
g.setColor(getColor(SVGData.getValueAt(row,"fill")))
g.fill(poly)
g.setColor(getColor(SVGData.getValueAt(row,"stroke")))
g.draw(poly)
g.setTransform(t)
if SVGData.getValueAt(row,"tag") == 'path':
g.setColor(getColor(SVGData.getValueAt(row,"stroke")))
g.setStroke(BasicStroke(int(SVGData.getValueAt(row,"StrokeW"))))
poly = GeneralPath()
d = SVGData.getValueAt(row,"d").strip()
subd = re.findall(r"[a-zA-Z][ ]*[0-9]*[.]*[0-9]*[,]*[0-9?]*[.]*[0-9]*",d)
for cnt in subd:
cmd = cnt.strip()[0]
if cmd == 'M':
points = re.findall(r"[0-9]*[.]*[0-9]",cnt)
poly.moveTo(float(points[0]),float(points[1]))
if cmd == 'L':
points = re.findall(r"[0-9]*[.]*[0-9]",cnt)
poly.lineTo(float(points[0]),float(points[1]))
if cmd == 'z' or cmd == 'Z':
poly.closePath()
if SVGData.getValueAt(row,"fill") != None:
g.setColor(getColor(SVGData.getValueAt(row,"fill")))
g.fill(poly)
g.setColor(getColor(SVGData.getValueAt(row,"stroke")))
g.draw(poly)
TF = AffineTransform()
g.setTransform(TF)
line = Line2D.Float(0,30,50,50)
g.setColor(system.gui.color(0,0,255))
g.setStroke(BasicStroke(1))
g.draw(line)