Meh. I wouldn’t do expensive interpolation. When each element of the heat map is larger than one pixel, I would just paint the larger element with the single color. If too sharp to use directly, I’d apply a cheap gaussian blur.
3 Likes
I tried to do the Gateway & Client Setup, but somehow Ignition Script Console is not reading the easysvg and troubleshooting the gatewayscript without something to check is quite hard since I will need to update it.
../lib/core/common and the /user-lib/pylib/ I tried putting the file here but it doesn’t seem to recognize it… I have also converted it from zip to .modl file to work but it doesn’t seem to be able
Code
import easysvg
import base64
import jnumeric.PyMultiarray as np
class heatPalette:
'''
Calculate the RGB value of a point on a gradient.
Usage: heatPalette(minval, maxval, [colors])
minval - minimun value to scale to
maxval - maximum value to scale to
colors - A list of RGB colors delineating a series of
adjacent linear color gradients between each pair.
Example: scale the values between 20 and 35 to a three color gradient.
p = heatPalette(20, 35, ['#0000FF', '#FFFF00', '#FF0000'])
After the instance is created, call it with a value to get the scaled color
p(20) returns '#0000FF'
p(25) returns '#AAAA55'
'''
def __init__(self, minval, maxval, colors):
self.minval = minval
self.maxval = maxval
self.colors = []
for color in colors:
if type(color).__name__ == 'str':
c = self.hex_to_rgb(color)
elif type(color).__name__ == 'list':
c = tuple(color)
elif type(color).__name__ == 'tuple':
c = color
self.colors.append(c)
def __call__(self, value):
return self.calc(value)
def calc(self, value):
import sys
# Smallest possible difference.
epsilon = sys.float_info.epsilon
# Keep value within limits
if value < self.minval:
value = self.minval
if value > self.maxval:
value = self.maxval
'''
Determine where the given value falls proportionality within
the range from minval->maxval and scale that fractional value
by the total number in the "colors" pallette.
'''
i_f = float(value-self.minval) / float(self.maxval-self.minval) * (len(self.colors)-1)
'''
Determine the lower index of the pair of color indices this
value corresponds and its fractional distance between the lower
and the upper colors.
'''
# Split into integer & fractional parts.
i, f = int(i_f // 1), i_f % 1
# Does it fall exactly on one of the color points?
if f < epsilon:
return self.rgb_to_hex(self.colors[i])
# Otherwise return a color within the range between them.
else:
(r1, g1, b1), (r2, g2, b2) = self.colors[i], self.colors[i+1]
print (r1, g1, b1), (r2, g2, b2)
print int(r1 + f*(r2-r1)), int(g1 + f*(g2-g1)), int(b1 + f*(b2-b1))
return self.rgb_to_hex((int(r1 + f*(r2-r1)), int(g1 + f*(g2-g1)), int(b1 + f*(b2-b1))))
def hex_to_rgb(self, value):
'''
Convert a hex RGB value to a tuple of 0-255 values
'''
value = value.lstrip('#')
return tuple(int(value[i:i+2], 16) for i in (0, 2, 4))
def rgb_to_hex(self, rgb):
'''
Convert an RGB tuple to a hex string
'''
return '#%02x%02x%02x' % rgb
# Convert to pyDataSet as easier to iterate
pyData = system.dataset.toPyDataSet(Row0.value.getValue())
sensornumber_row = 32
sensornumber_col = 32
new_row_size = 94
new_col_size = 94
row_pixel_spread = 2
col_pixel_spread = 2
resize=np.zeros((94,94), 'f')
imageid= 1
for m in range( 0, sensornumber_row):
for n in range (0, new_col_size):
count = n % (col_pixel_spread + 1)
if (count ==0):
resize[int((m*(row_pixel_spread + 1)))][n] = pyData[m][int((n/(col_pixel_spread+1)))]
if count!=0:
GRADIENT = pyData[m][int((n/(col_pixel_spread+1))+1)] - pyData[m][int((n/(col_pixel_spread+1)))]
GRADIENT = GRADIENT/(col_pixel_spread+1)
resize[int((m*(row_pixel_spread + 1)))][n] = (pyData[m][int((n/(col_pixel_spread+1)))] + GRADIENT*count)
for n in range( 0, new_col_size):
for m in range (0, new_row_size):
count = m % (row_pixel_spread + 1)
if (count == 0):
resize[m][n] = resize[m][n]
if (count != 0):
m1 = int(int(m/(row_pixel_spread+1))*(row_pixel_spread+1)+(row_pixel_spread+1))
GRADIENT = resize[m1][n] - resize[int(int(m/(row_pixel_spread+1))*(row_pixel_spread+1))][n]
GRADIENT = GRADIENT/(row_pixel_spread+1)
resize[m][n] = (resize[int(int(m/(row_pixel_spread+1))*(row_pixel_spread+1)),n] + GRADIENT*count)
#Define palette for heatmap
palette = heatPalette(20, 30, ['#0000FF', '#FFFF00', '#FF0000'])
# Width, Height of each cell in heatmap
cellSize = (50, 50)
# Define padding area aroung the heatmap
padding = 50
# Create SVG
svg = easysvg.SvgGenerator()
svg.begin(cellSize[0] * (new_col_size - 2) + 2 * padding, cellSize[1] * new_row_size + 2* padding)
for i, row in enumerate(resize):
for j, col in enumerate(list(row)[2:]):
color = palette(col)
svg.rect(j*cellSize[0] + padding, i * cellSize[1] + padding, cellSize[0], cellSize[1], color)
svg.end()
svg_string = svg.get_svg()
b64 = base64.b64encode(svg_string)
system.db.runPrepQuery("INSERT INTO 'svgtable' (`ID`, `converttime`, `base64svg`)", [NULL, current_timestamp(), b64])
The retrieving portion for the Client is quite clear, I might need to do some modification to the string to remove “/u003d/u003d” just in case
It should to go into /user-lib/pylib/site-packages. You may also need to restart the gateway to get it to update.
Hi,
Just wondering where the SVG will live after creation.
Thanks
I'd recommend as a blob in a database table.
Sorry, I was not very clear with my question. When you assigned the SVG to the image component as follows:
self.getSibling('Image').props.source = 'data:image/svg+xml;base64,' + b64
Where does the generated SVG file live? Is this SVG file generated by the script only a one-time thing if I do not choose to store it in the DB?
Yes, that particular example is creating a data URI; the SVG is that base64 string, and the user agent (browser, mobile device, whatever) knows how to parse that encoded string back into an actual useful image. Useful for testing, but can be a performance cliff.
1 Like
I see that it can cause performance issue when the data points exceed a certain number. Do you have other recommendations as to how to handle it? This method does have better performance than using the native Perspective table or chart.
It depends on how dynamic your data is. There's a lot of range between "dynamically calculate the image every time" and "cache the image forever".
If you do need it to be dynamic, you might want to take a look at the Webdev module (not free) or @pturmel's Blob Server module (free). With Phil's module, you could pre-calculate your image on some trigger with scripting and store it to the DB, then use Phil's module to 'serve' it up for as many clients as needed. With Webdev, you can run a script every time the image is requested, allowing you to implement any caching strategy you want.
1 Like
Is this available in Maker?
Looks like it is.
1 Like
The data is very dynamic. The client wanted to choose the time range, query the data, and build the heat map right then and there. So it does not make sense to pre-calculate the image and store it to the DB in this case. If I choose to go with Webdev, will I just utilize the script that @JordanCClark provided to generate the data URL in the Webdev environment and then return it to the image component?
You would skip this part. You would return the binary image data directly with the appropriate content type.
That makes sense. I am guessing the following code does not give me the binary image data:
svg = easysvg.SvgGenerator()
system.perspective.print(cellSize[0] * (data.columnCount) + 2 * padding)
system.perspective.print(cellSize[1] * data.rowCount + 2* padding)
svg.begin(cellSize[0] * (data.columnCount) + 2 * padding, cellSize[1] * data.rowCount + 2* padding)
for i, row in enumerate(data):
for j, col in enumerate(list(row)[2:]):
color = palette(col)
svg.rect(j*cellSize[0] + padding, i * cellSize[1] + padding, cellSize[0], cellSize[1], color)
svg.end()
svg_string = svg.get_svg()
How would I get the binary data from this?
Thanks
Actually, for that, just return the svg_string and let WebDev encode it for you.
It is not working for me. Not sure if it is because the dataset is too big. I got an error that the localhost is unable to handle this request. Should I not have done the computing in the Web Dev? I am trying to create a heat map image with at least 500 columns X 200 rows of data.
code in Web Dev:
#query data from database
startTime = '2023-04-07 03:58:16.093'
endTime = '2023-04-07 04:40:51'
parameter = {'startDate': startTime, 'endDate': endTime}
ds = system.db.runNamedQuery('SlitRollReport','masterRollData', parameter)
colsRemove = ['Accumulated_Length','t_stamp','Scan_Count','Specification_Maximum','Specification_Minimum']
colsKeep = list(ds.columnNames)
for col in colsRemove: colsKeep.remove(col)
ds = system.dataset.filterColumns(ds, colsKeep)
data = system.dataset.toPyDataSet(ds)
print data
#_________________________________________________________________________________
import easysvg
import base64
class heatPalette:
'''
Calculate the RGB value of a point on a gradient.
Usage: heatPalette(minval, maxval, [colors])
minval - minimun value to scale to
maxval - maximum value to scale to
colors - A list of RGB colors delineating a series of
adjacent linear color gradients between each pair.
Example: scale the values between 20 and 35 to a three color gradient.
p = heatPalette(20, 35, ['#0000FF', '#FFFF00', '#FF0000'])
After the instance is created, call it with a value to get the scaled color
p(20) returns '#0000FF'
p(25) returns '#AAAA55'
'''
def __init__(self, minval, maxval, colors):
self.minval = minval
self.maxval = maxval
self.colors = []
for color in colors:
if type(color).__name__ == 'str':
c = self.hex_to_rgb(color)
elif type(color).__name__ == 'list':
c = tuple(color)
elif type(color).__name__ == 'tuple':
c = color
self.colors.append(c)
def __call__(self, value):
return self.calc(value)
def calc(self, value):
import sys
# Smallest possible difference.
epsilon = sys.float_info.epsilon
# Keep value within limits
if value < self.minval:
value = self.minval
if value > self.maxval:
value = self.maxval
'''
Determine where the given value falls proportionality within
the range from minval->maxval and scale that fractional value
by the total number in the "colors" pallette.
'''
i_f = float(value-self.minval) / float(self.maxval-self.minval) * (len(self.colors)-1)
'''
Determine the lower index of the pair of color indices this
value corresponds and its fractional distance between the lower
and the upper colors.
'''
# Split into integer & fractional parts.
i, f = int(i_f // 1), i_f % 1
# Does it fall exactly on one of the color points?
if f < epsilon:
return self.rgb_to_hex(self.colors[i])
# Otherwise return a color within the range between them.
else:
(r1, g1, b1), (r2, g2, b2) = self.colors[i], self.colors[i+1]
#print (r1, g1, b1), (r2, g2, b2)
#print int(r1 + f*(r2-r1)), int(g1 + f*(g2-g1)), int(b1 + f*(b2-b1))
return self.rgb_to_hex((int(r1 + f*(r2-r1)), int(g1 + f*(g2-g1)), int(b1 + f*(b2-b1))))
def hex_to_rgb(self, value):
'''
Convert a hex RGB value to a tuple of 0-255 values
'''
value = value.lstrip('#')
return tuple(int(value[i:i+2], 16) for i in (0, 2, 4))
def rgb_to_hex(self, rgb):
'''
Convert an RGB tuple to a hex string
'''
#Update the heat Map
#_________________________________#
# #
# INITIAL SETTINGS #
#_________________________________#
Min = 29
Max = 31
# Define palette for heatmap
palette = heatPalette(Min, Max, ['#FF0000', '#00FF00', '#00D900', '#FFFF00'])
# Width, Height of each cell in heatmap
cellSize = (30, 30)
# Define padding area aroung the heatmap
padding = 20
#this is the original script that creates a 32 x 32 pixels image
# Create SVG
svg = easysvg.SvgGenerator()
svg.begin(cellSize[0] * (data.columnCount) + 2 * padding, cellSize[1] * data.rowCount + 2* padding)
for i, row in enumerate(data):
for j, col in enumerate(list(row)[2:]):
color = palette(col)
svg.rect(j*cellSize[0] + padding, i * cellSize[1] + padding, cellSize[0], cellSize[1], color)
svg.end()
svg_string = svg.get_svg()
return {'bytes': svg_string}
1 Like
Check your gateway logs to see the actual error.
Error from the gateway logs:
INFO | jvm 4 | 2023/04/28 16:05:43 | I [p.ClientSession ] [20:05:43]: WebSocket disconnected from session. session-project=SlitRollReport
INFO | jvm 4 | 2023/04/28 16:05:46 | <PyDataset rows:217 cols:512>
INFO | jvm 4 | 2023/04/28 16:05:46 | after svg begin
INFO | jvm 4 | 2023/04/28 16:05:47 | end svg
INFO | jvm 4 | 2023/04/28 16:05:47 | Bad Base64 input character at 0: 60(decimal)
it appears that the error occurred because the last line of code
return{'bytes': svg_string}
You shouldn't need any base64 with WebDev, as Phil alluded to.
base64 was not used in my code for WebDev. In the example, I returned the svg_string as bytes unless I misunderstood something here.