View Dataset from script console

Is there an easy way to view a dataset from the script console while developing?

1 Like

The easiest way I know of would be to iterate through it and print it .

Check the following page for an example.

That’s what i ended up doing, thank you.

Sorry, I'm on vacation, but family is napping, so I get to play. Yes, this is how I relax. Don't judge.
EDIT: Added option for printing from Perspective. Sorry, not quite ready for prime time. :roll_eyes:

def printDataSet(datasetIn):
	from itertools import izip
	import ast
	if 'PropertyTreeScriptWrapper$ArrayWrapper' in str(type(datasetIn)):
		headers = datasetIn[0].keys()
		data = []
		for row in datasetIn:
			data.append([item.value for item in row.values()])
		
		datasetIn = system.dataset.toDataSet(headers, data)
		
	# Get Row and Column counts
	nrows = datasetIn.getRowCount()
	ncols = datasetIn.getColumnCount()
        		        			
	# Get max length of row labels
	rowLen = len(max(['row']+[unicode(i) for i in range(nrows)], key=len))
	# Get column names
	colNames = datasetIn.getColumnNames()
	# initialize lists to process columns
	headerList = []
	colData = []
	maxLen = []
	# Process columns
	for i, col in izip(range(ncols), colNames):
		# Get column as list, converting all elemenst to unicode strings
		colIn = ([unicode(element) for element in list(datasetIn.getColumnAsList(i))])
		# Get max lentgh of the column, including the column name
		maxLen = len(max([col]+colIn, key=len))
		# Append data as left justified strings.
		# ljust() will automatically pad to the max string length
		colData.append([element.ljust(maxLen) for element in colIn])
		# Append column name, also using the max string length
		headerList.append(col.ljust(maxLen))
	# Concatenate the header string and print it.
	headerString= 'row'.ljust(rowLen) + ' | ' + ' | '.join(headerList)
	
	print headerString
	# Print a bunch of dashes the same length as the header string
	print'-' * len(headerString)
	# Print the rows
	for row in enumerate(izip(*colData)):
		print unicode(row[0]).ljust(rowLen) + ' | ' + ' | '.join(row[1])

Sample output:

row | fgPartNum | Motor    | Harness    | Latch      | DCM        | PSAT       | Antenna | Speaker    | Chassis    | PO     | Description         
--------------------------------------------------------------------------------------------------------------------------------------------------
0   | 4CJ9902   | N/A      | 68343451AA | 68367059AA | 68203031AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
1   | 4CJ9904   | N/A      | 68343389AA | 68367059AA | 68203031AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
2   | 4CJ9906   | N/A      | 68483665AA | 68367059AA | 68203031AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
3   | 4CJ9908   | N/A      | 68505567AA | 68367059AA | 68203031AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
4   | 4CJ9910   | N/A      | 68457400AA | 68367059AA | 68203031AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
5   | 4CJ9914   | N/A      | 68483666AA | 68367059AA | 68203031AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
6   | 4CJ9916   | N/A      | 68483667AA | 68366541AA | 68203031AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
7   | 4CJ9918   | N/A      | 68448401AA | 68366541AA | 68203031AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
8   | 4CJ9920   | N/A      | 68506501AA | 68367059AA | 68203031AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
9   | 4CJ9922   | N/A      | 68343451AA | 68367059AA | 68203033AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
10  | 4CJ9924   | N/A      | 68343389AA | 68367059AA | 68203033AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
11  | 4CJ9926   | N/A      | 68483665AA | 68367059AA | 68203033AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
12  | 4CJ9930   | N/A      | 68457400AA | 68367059AA | 68203033AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
13  | 4CJ9932   | N/A      | 68505567AA | 68367059AA | 68203033AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
14  | 4CJ9934   | N/A      | 68483666AA | 68367059AA | 68203033AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
15  | 4CJ9936   | N/A      | 68483667AA | 68366541AA | 68203033AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
16  | 4CJ9938   | N/A      | 68448401AA | 68366541AA | 68203033AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
17  | 4CJ9940   | N/A      | 68506501AA | 68367059AA | 68203033AA | 68370843AA | N/A     | N/A        | 68426067AA | 123456 | LH FRONT            
18  | 4CJ9952   | CM094380 | 68343453AA | 68366543AA | N/A        | N/A        | N/A     | 68426487AA | 68426071AA | 123456 | LH REAR AUTO DOWN   
19  | 4CJ9954   | CM094380 | 68447658AA | 68366543AA | N/A        | N/A        | N/A     | 68426487AA | 68426071AA | 123456 | LH REAR AUTO DOWN   
20  | 4CJ9956   | CM094380 | 68483657AA | 68366543AA | N/A        | N/A        | N/A     | 68347029AA | 68426071AA | 123456 | LH REAR AUTO DOWN   
21  | 4CJ9958   | CM094360 | 68447659AA | 68366543AA | N/A        | N/A        | N/A     | 68347029AA | 68426071AA | 123456 | LH REAR AUTO UP/DOWN
22  | 4CJ9960   | CM094360 | 68447660AA | 68366543AA | N/A        | N/A        | N/A     | 68459976AA | 68426071AA | 123456 | LH REAR AUTO UP/DOWN
23  | 4CJ9962   | CM094380 | 68343449AA | 68366543AA | N/A        | N/A        | N/A     | N/A        | 68426071AA | 123456 | LH REAR AUTO DOWN   
17 Likes

I get NameError: global name 'tagIn' is not defined, first line under your for i, col in zip(range(ncols), colNames):

Change tagIn to datasetIn

1 Like

Yep. Script updated.

@JordanCClark
Why don’t I see a return in the function?
I thought functions had to have returns.
I tried it, and it works, but I am confused.


util.printDataSet(datasetIn)

Is util a good name?


What else do you have stashed?

In general, functions don't have to have a return. The return is only useful if you want to... return something.
Now, in the case of python, it turns out they will always return something. If you don't specify what, they will return None.

If it's an utility function, then sure. I have a folder named utils with utility scripts in it.

2 Likes

Most of what I have is a bit too specific to my needs, but those could be useful:

utils.dataset
def to_dicts(ds):
	columns = system.dataset.getColumnHeaders(ds)
	return [dict(zip(columns, row)) for row in system.dataset.toPyDataSet(ds)]

def to_string(ds):
	headers = ds.getColumnNames()
	widths = [len(str(ds.getRowCount()))] + [
		max(
			max(len(str(value)) for value in ds.getColumnAsList(c)),
			len(headers[c])
		) for c in range(ds.getColumnCount())
	]
	headers = [""] + headers
	ds = system.dataset.toPyDataSet(ds)
	return (
		" | ".join(h.ljust(w) for h, w in zip(headers, widths)) + '\n'
		+ "-" * (sum(widths) + 3*(len(widths)-1)) + '\n'
		+ '\n'.join(" | ".join(str(v).ljust(w) for v, w in zip([i] + [r for r in row], widths)) for i, row in enumerate(ds))
	)

def print_ds(ds):
	print to_string(ds)
utils.tags
from com.inductiveautomation.ignition.common.model.values import BasicQualifiedValue, QualityCode

base_logger = system.util.getLogger("Utils").createSubLogger("Tags")

def force_quality(paths, quality):
	"""
	Force a quality on all the tags in {paths}.

	Values and timestamp are left unchanged.
	quality can either be a QualityCode, or a string that will be used to build a QualityCode.
	If quality is a string and not a valid QualityCode, QualityCode.Uncertain will be used instead.
	"""

	logger = base_logger.createSubLogger("ForceQuality")
	logger.info("forcing quality {} on the following tags: {}".format(quality, ", ".join(paths)))

	if not isinstance(quality, QualityCode):
		quality = getattr(QualityCode, quality.title(), QualityCode.Uncertain)

	tags = system.tag.readBlocking(paths)
	values = [BasicQualifiedValue(tag.value, quality) for tag in tags]
	utils.tag.write(paths, values, logger)

def write(paths, values, logger):
	write_return = system.tag.writeBlocking(paths, values)
	bad_writes = ["{} ({})".format(path, quality) for path, quality in zip(paths, write_return) if not quality.isGood()]
	if bad_writes:
		logger.warn("failed writes: {}".format(bad_writes))


def write_async(paths, values, logger):
	def log(write_return):
		bad_writes = ["{} ({})".format(path, quality) for path, quality in zip(paths, write_return) if not quality.isGood()]
		if bad_writes:
			logger.warn("failed writes: {}".format(bad_writes))
	system.tag.writeAsync(paths, values, log)
utils.test
import ast

"""
Testing framework

Import test in your file: from utils.test import test

Decorate your functions to be tested:
@test
def test_foo():
    foo = "foo"
    bar = get_bar()
    assert foo == bar, "expected {}, got {}".format(foo, bar)

Call the function individually, or use run_tests() to run all the marked functions in modules:
run_tests(module_1, module_b)
"""

class TestFinder(ast.NodeVisitor):
	def visit_Module(self, node):
		self.test_funcs = []
		self.generic_visit(node)

	def visit_FunctionDef(self, node):
		if "test" in (deco.id for deco in node.decorator_list):
			self.test_funcs.append(node.name)
	
	def get_testfuncs(self):
		return self.test_funcs

def test(func):
	"""
	Decorator to mark functions to be tested.
	
	Simply import the decorator (from Tests.utils import test) and add @test before the function
	"""

	def wrapper():
		try:
			func()
		except AssertionError as e:
			message = "failed ({})".format(e)
		else:
			message = "passed"
		return message
	return wrapper


def run_tests(*modules):
	"""
	Run all the functions marked as tests (with the test decorator) in every packages passed as parameter.
	"""
	for m in modules:
		print("{}\ntesting functions in {}...".format("-"*20, m.name))

		tree = ast.parse(m.code)
		f = TestFinder()
		f.visit(tree)
		funcs = f.get_testfuncs()

		if not funcs:
			print("No functions marked for testing found in {}".format(m.name))
		for f in funcs:
			try:
				message = getattr(m, f)()
				print("{:.<40}{}".format(f, message))
			except BaseException as e:
				print(repr(e))

I run this in the script console like this (basic example):

image

2 Likes

I didn’t understand what those do Pascal.


# Get max lentgh of the column, including the column name
maxLen = len(max([col]+colIn, key=len))

I don’t understand what this is doing exactly.
I tried looking up how key=len works.
Tried testing it.

L1=['spider', 'moose', 'horse', 'ant', 'dog', 'cat']
L3= sorted(L1,key=len)
print (L3)
['ant', 'dog', 'cat', 'moose', 'horse', 'spider']

Still confused about it. I expected to see the long words first and in alphabetical order.
Output was not alphabetized at all. Seemed like it is like setting the thing to sort by.
However, the code uses key=len inside of the max(), so I am even more confused.

That’s Jordan’s code. But I’ll try to crack this case !

key being an argument (or a key word argument) of those functions, it might have a different effect from one function to another. Just because it has the same name doesn’t mean it will have the same effect.

In the case of sorted, well… It tells the function what to sort by.
So if you’re telling it to sort by length, there’s no reason it should be alphabetically.
Also, it sorts by ascending order by default (try sorted([5, 2, 3, 1, 4])). If you want it to sort in descending order, you’ll need to pass it another argument: sorted(seq, reverse=True)

In the case of max, it’s actually pretty much the same thing. The key argument tells it what to use to compare elements. So if you give it key=len, it will use the return of the len function called on each element to evaluate what’s the biggest value.
It’s very close to what key does with sorted, as you could get the max value using sorted:
max_value = sorted(seq, key=len)[-1]

2 Likes

As Pascal mentioned, key tells what to use to perform the sort.

Here’s some extra examples. I changed up your list a bit to illustrate:

L1=['spider', 'mooselings', 'ant', 'Zebra',  'dog', 'cat']

# Sort by alphanumeric value
print sorted(L1)
print max(L1)
print len(max(L1))
print '------------'

# Sort by length of the string.
print sorted(L1, key = len)
print max(L1, key = len)
print len(max(L1, key = len))

print '------------'

# Sort by alphabetical by using the lower case versions
print sorted(L1, key = str.lower)
print max(L1, key = str.lower)
['Zebra', 'ant', 'cat', 'dog', 'mooselings', 'spider']
spider
6
------------
['ant', 'dog', 'cat', 'Zebra', 'spider', 'mooselings']
mooselings
10
------------
['ant', 'cat', 'dog', 'mooselings', 'spider', 'Zebra']
Zebra
1 Like

I did not realize that lowercase had a higher value than uppercase alphanumerically.

If you look at the values in an ASCII table, you can get an idea of how characters compare.

3 Likes