Hi,
We would love to test our Apache IoTDB Tag Historian module under more usual loads for IoTDB and therefore would love to create something round 500000 tags that we can store in IoTDB. I know there are the simulated devices available, but creating so many tags would take for ever to setup.
Is there a way to create a large number of simulated tags for load testing? (Maybe even using an external process on a separate device)
Chris
If I remember right the programmable device simulator can be generated from a CSV, maybe can write a script to generate a massive CSV for you to then generate these?
Hi Hayden,
that sounds like a reasonable plan ... and running a CSV would probably reduce the load of generating hundreds of thousands of simulated devices.
Do you have any pointers to this programmable device simulator?
Chris
Probably best adding some tags on the interface, then exporting the CSV to get the format right. Write a script to generate a load more tags, then reimport it.
Hi Hayden,
thanks for that ... I was already watching the video ... helps a lot 
Chris
One thing however that I don't see, is that in this I can setup loads of tags for the OPC-UA simulator device, however in order to make this data go to the tag-historian, I usually need to go to the graphical editor, create an ignition tag, select the OPC-UA tag as source and then configure it to go to my historian.
I can't see this part being "automatable".
Do you have any idea how I could programm creating these simulated tags going to the historian?
Bulk/programmatic creation via system.tag.configure | Ignition User Manual
Most advanced Ignition users would write a jython script to instantiate many tags with system.tag.configure(). It would be most efficient if done with a UDT that has many member tags, preset to historize to your new historian, preferably expression tags that generate some pseudo-random or cyclic data.
Aaaaah ... that sounds a lot more interesting
I'll look into it ... thanks for the pointers.
Chris
Hmpf ... why can't anything I'm trying to do be simple?
So ... I've been struggling a bit as it seems that if I browse tags the ones with history only have an "attribute" of "history" (or something similar). if I browse the historic tags I get only the history settings ... so what I tried doing, is go through the tags and look for the ones with "history" attribute, then browseHistoricTags using the path I got there ... unfortunately I keep on getting errors that "system" is not defined. Even if I use the example on this page:
Has anything changed? It seems as soon as I use "system" inside a locally defined function, I get the errors.
This error usually comes from scoping issues. Where exactly are your "locally defined functions" ?
I just added a script to gateway event of "timer" and if I simply paste the example:
def browse(path='histprov:DB:/drv:controller:default:/tag:simulator'):
for result in system.tag.browseHistoricalTags(path).getResults():
print result.getPath()
if result.hasChildren():
browse(result.getPath())
browse()
I get that error.
Gateway events have weird scoping rules, inherited from older versions. Don't code there.
Instead, put your code in the script library, and call it from the gateway event.
Alternatively, you can generate many instances of a UDT with
UDT Multi-Instance Wizard | Ignition User Manual (inductiveautomation.com)
Yep... gateway events should be a single line callout to a project library script.
Well I gave up on the whole script things as I thought I'd need to invest quite a bit of time to find out how to do it ... instead I wrote a little program, that generates a json file that has a configurable number of historian tags in them. With this I was able to test my Apache IoTDB historian with 161000 tags ... guess when importing another 500000 I was a bit greedy ... now the UI application doesn't seem to like me anymore
Here's the code for my little tool:
package com.timecho.iotdb.ignition.tools.taggenerator;
import java.io.PrintStream;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Random;
public class TagGenerator {
static Device[] devices = {
new Device("Realistic0", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic0", "Float8", "Ignition OPC UA Server"),
new Device("Realistic1", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic1", "Float8", "Ignition OPC UA Server"),
new Device("Realistic2", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic2", "Float8", "Ignition OPC UA Server"),
new Device("Realistic3", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic3", "Float8", "Ignition OPC UA Server"),
new Device("Realistic4", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic4", "Float8", "Ignition OPC UA Server"),
new Device("Realistic5", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic5", "Float8", "Ignition OPC UA Server"),
new Device("Realistic6", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic6", "Float8", "Ignition OPC UA Server"),
new Device("Realistic7", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic7", "Float8", "Ignition OPC UA Server"),
new Device("Realistic8", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic8", "Float8", "Ignition OPC UA Server"),
new Device("Realistic9", "AtomicTag", "opc", "ns=1;s=[Sample_Device]_Meta:Realistic/Realistic9", "Float8", "Ignition OPC UA Server")
};
public static void main(String[] args) throws Exception {
Random random = new Random();
if(args.length < 2 || args.length > 3) {
System.out.println("Usage: TagGenerator {num-total-tags} {num-tags-per-folder} ({output-file-name})");
}
int numTags = Integer.parseInt(args[0]);
int numTagsForFolder = Integer.parseInt(args[1]);
PrintStream out = System.out;
if(args.length == 3) {
out = new PrintStream(Files.newOutputStream(Paths.get(args[2])));
}
int folderIndex = 0;
out.printf(
"{\n" +
" \"tags\": [\n" +
" {\n" +
" \"name\": \"Benchmark%08d\",\n" +
" \"tagType\": \"Folder\",\n" +
" \"tags\": [\n", folderIndex);
int curNumTagsInFolder = 0;
for(int i = 0; i < numTags; i++) {
int deviceIndex = random.nextInt(10);
Device curDevice = devices[deviceIndex];
out.printf(" {\n" +
" \"valueSource\": \"%s\",\n" +
" \"opcItemPath\": \"%s\",\n" +
" \"dataType\": \"%s\",\n" +
" \"historyProvider\": \"Apache IoTDB History Provider\",\n" +
" \"name\": \"BenchmarkTag%08d\",\n" +
" \"historyEnabled\": true,\n" +
" \"tagType\": \"%s\",\n" +
" \"opcServer\": \"%s\"\n" +
" }%s\n", curDevice.getValueSource(), curDevice.getOpcItemPath(), curDevice.getDataType(),
curNumTagsInFolder,
curDevice.getTagType(), curDevice.getOpcServer(),
(curNumTagsInFolder == numTagsForFolder || i == (numTags - 1)) ? "" : ",");
curNumTagsInFolder++;
if(curNumTagsInFolder > numTagsForFolder) {
curNumTagsInFolder = 0;
folderIndex++;
out.printf(
" ]\n" +
" },\n" +
" {\n" +
" \"name\": \"Benchmark%08d\",\n" +
" \"tagType\": \"Folder\",\n" +
" \"tags\": [\n", folderIndex);
}
}
out.print(
" ]\n" +
" }\n" +
" ]\n" +
"}\n");
if(out != System.out) {
out.flush();
out.close();
}
}
}
It's not super battle proven ... I guess there are probably edge-cases, that make it produce invalid JSON, but I didn't run into any issues.
