Ignition Edge Store and Forward

[PRAMANJ]

@R.Alamsha

Moral of the story: Ban polling and use PUSH only technology. Follow the old fashioned PSTN Telemetry SCADA rules for IOT SCADA.

1. PubNub uses push technology.
2. The messages are sent in compressed form on exception so where are the MB's of data coming in to picture! PubNub delivers less less than data-gram size messages (~1500bytes) most efficiently as a single packet. A data-gram size message can represent a large data when its compressed and sent on exception basis from RTU. Where is the question of bandwidth chocking! I don't understand!
3. PubNub cost is peanuts (1 million messages are free every month, plus there is an SLA and guarantee for QOS and builtin data historian on their server. FTP also works on internet which you have to pay for. FTP is for transfer of large files (MBs to GBs) not few byte files! Also FTP needs to be supported on client and server (RTU and CCC) and will have a large foot print! Can your RTU support that?
4. If customer is happy with ftp then why to look for an alternative technology. Let others also use your approach you you can commercialize or publish it!

[R.Alamsha]

@PRAMANJ

PubNub cost is peanuts (1 million messages are free every month,

PubNub looks like a dream come true, sounds very interesting and pricing is affordable. I love it because they repeatedly claim "PUSH" technology, inline with my data collection philosophy. I have to discuss with them before proposing it to the client.

[PRAMANJ]

@R.Alamsha

I have to discuss with them before proposing it to the client.

They do have customer specific custom service offering for higher reliability and QoS. You should definitely talk to them if you are serious!

[R.Alamsha]

Onsite connectivity for remote stations is on a private mobile network package. If i connect that mobile network at each site to PubNub, the cost will shoot up and hence not viable. Apart from this, coordinating with the ISP for internet connection at each site is a nightmare. So, we will stick on with the old Telemetry and FTP S&F model. At CCC, we have state of the art internet connectivity. PubNub looks like an excellent solution for high availability web services from CCC. Revenue generation through web service is the life line of this project. I will look into this technology seriously. Thanks for this valuable suggestion.

[PRAMANJ]

@R.Alamsha

So, we will stick on with the old Telemetry and FTP S&F model.

That's fine as long as it works and meets your requirements. I am not conversant with FTP over Telemetry if you can give some references I can explore it.

[R.Alamsha]

@PRAMANJ

I am not conversant with FTP over Telemetry if you can give some references I can explore it.

The objective of FTP based S&F is to collect and download the remote time stamped RTU datalogs into the SCADA Tag Historain . This can be accomplished in 2 ways.

OPC UA alternatives (aka Kepserver)?

This script will parse your CSV and enter it into the ignition historian with your RTU timestamp. #convert the csv into an ignition dataset import csv path = system.file.openFile("csv") if path != None: f = open(path,'rb') reader = csv.reader(f) csvlist = list(reader) ds=[] row = 0 for x in csvlist: if row == 0: headers = x if row > 0: ds.append(x) row += 1 csvdataset = system.dataset.toDataSet(headers, ds) #parse the dataset and enter the data into the historian for row in r…

1. Run jython script on Ignition and download as explained above.
2. Import the csv files directly into the database like MySQL / PostgreSQL etc.

I haven't tested option 1 yet. Personally, i prefer option 2, because MySQL takes care of rejecting duplicate tags automatically and it's very fast. When you automate this process, your script must select the latest table partition and import.

Automatic IOT data collection by FTP

I would like to automate historical data collection (data logs) from 100’s of remote RTUs. These devices can send timestamped CSV/JSON files upon exception or periodically by secured FTP to designated folders on a Linux server. I want to monitor/scan these folders periodically for any new file arrival and automatically import/merge these data logs to Ignition Historian. I found these 2 links interesting. http://ignitiontips.blogspot.ae/2016/08/import-historical-data-from-csv-to.html I am NO…

[R.Alamsha]

@PRAMANJ

That's fine as long as it works and meets your requirements

I have forgotten to mention this point. The main reason for avoiding internet based automation for remote sites is, every site needs an IP address. VPN routers can help but again automating this process for 1000's of sites from CCC needs a very reliable VPN service provider which is very expensive. Where as, a private mobile network data channel gives private IP by default, deployment is very fast, 100% secured and connectivity is instantaneous. No need for any VPN router or Dynamic DNS. Downside is cost and limited bandwidth. Both can be very well controlled as discussed above.