# Websocket for real time high performant SCADA **URL:** \ **Category:** General Discussion\ **Created:** [May 29, 2018, 4:56am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194 "2018-05-29T04:56:31Z")\ **Posts on this page:** 20\ **Page:** 3
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 1, 2018, 4:57pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/41 "2018-06-01T16:57:49Z")
> [@PRAMANJ](#): > > Performance wise java module should be faster as it's I compiled code than python but I haven't tried python approach Python code also is using the same websocket approach like you do, i think. Is that correct?. ---
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 1, 2018, 7:04pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/42 "2018-06-01T19:04:39Z")
> [@Kyle\_Chase](#): > > MQTT/Pubnub/HTTP are all better protocols to use for this, as opposed to FTP. I was also once very much excited about MQTT. When i insisted our RTU vendor for MQTT embedded RTUs, this is what he replied: "We implemented MQTT in our RTUs and carried out field study for more than a year. We studied thousands of transactions and found MODBUS TCP and FTP were miles ahead and 100% outperformed MQTT hands down on every transaction. We have decided NOT to pursue MQTT any more for our RTU business. MQTT is definitely light weight and may be suitable for home automation business, smart sensors and embedded devices like thermostats, temperature sensors and so on. If you want MQTT RTUs, please find some other supplier. We are NOT interested. Thanks". ---
**Author:** ![Kyle\_Chase](https://avatars.discourse-cdn.com/v4/letter/k/9de0a6/32.png) [@Kyle\_Chase](https://forum.inductiveautomation.com/u/Kyle_Chase)\ **Post date:** [June 1, 2018, 7:31pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/43 "2018-06-01T19:31:37Z")
> [@R.Alamsha](#): > > You are right. FTP is easy, ubiquitous, available in every RTU and a long haul data transfer workhorse. For example, you and i got our 300+ mb Ignition by FTP. Ofcourse, entire world does this way and it works perfectly every time. If any network failure happens, my torrent downloader retries and continues downloading from where it was left. This solution is available for uploading also, since decades. > > Is this kind of robust file transfer possible with MQTT?. BTW, how many RTUs do you think have got MQTT?. I have not seen FTP in any RTU that I have used in production. So maybe this is an Asia thing. What link are you using to download Ignition from? I have never seen an FTP link for an Ignition binary, IA's website uses HTTP for file transfer. Also, you mention torrents. Well, the BitTorrent protocol is not FTP, and I have never seen BitTorrent on an RTU either. As for file transfer over MQTT, yes, it is possible and reliable. We have transferred video, images, VPN configuration files, SSL certificates, and even firmware upgrades over MQTT. ---
**Author:** ![Kyle\_Chase](https://avatars.discourse-cdn.com/v4/letter/k/9de0a6/32.png) [@Kyle\_Chase](https://forum.inductiveautomation.com/u/Kyle_Chase)\ **Post date:** [June 1, 2018, 7:47pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/44 "2018-06-01T19:47:25Z")
> [@R.Alamsha](#): > > Collect IOT data from JSON API's servers and csv files sent by thousands of RTUs by FTP. The FTP files are encrypted as per the latest internet standards and the data collector must run the file parsing decryptor supplied by the client and serve the data. Yes, this is possible with Ignition, the built in FTP libraries, and python. > [@R.Alamsha](#): > > Process it on the ML/AI engine server supplied by the client( this is supplied by IBM. May be Watson engine). Yes, Ignition can more this information into Watson > [@R.Alamsha](#): > > Stream the processed data and alarms on JSON API (like online trading servers) for 1000's of mobile and remote control centre consumers. The CCC server will be deployed on a private cloud at CCC connected to high speed streaming networks like PubNub and so on. Yes, MQTT, PubNub, HTTP REST, can all move data and provide endpoints to be consumed by other applications. > [@R.Alamsha](#): > > 1. Serve the data on "MODBUS TCP Master/Slave" for SCADA consumers like Ignition, WinCC OA and so on. Ignition can talk Modbus as a master natively already, and implementing a slave would be relatively easy as well. > [@R.Alamsha](#): > > 1. Scalable, unlimited tags and clients capability. (Ignition is best qualified here) Yes, Ignition is good at this, and they are getting better with every release. > [@R.Alamsha](#): > > RTU's are deployed on a private mobile network which charges by data transfer volume. To keep the monthly bills within the budget, RTUs are designed to report by exception and go offline like old fashioned PSTN lines. Whenever they connect to the MODBUS data server at CCC, all RTU tags are automatically updated and also a timestamped encrypted csv file is transferred by FTP. So, the data collecting server or SCADA stations won't be able to periodically scan the remote RTUs to validate the tags.\_ Having Ignition act as an FTP client to the RTUs is possible, but based on the report by expection statement, it seems like you want the RTU to be a client to an FTP server. Either way, Ignition can do this, either as a client, or by watching an local directory on the server exposed as an FTP target. Ignition already supports Modbus as a master, and a slave can be built as required. > [@R.Alamsha](#): > > SCADA stations can scan the front end "MODBUS TCP Master/Slave" data server at CCC as usual to avoid tag fault errors and disruption of services. _In fact, the front end server simulates the remote RTUs._ Same as my other Modbus comments. This is possible. > [@R.Alamsha](#): > > **Retain and serve the last value of all tags to SCADA consumers upon reboot or network failure etc.The data servers shall be fault tolerant and shall continue to serve the data seamlessly on the same IP to data consumers, in case of any changeover or maintenance** Ignition retains values of tags already, but some sort of persistence layer could be used, a SQL server, a NoSQL database, etc. > [@R.Alamsha](#): > > 1. **The SCADA database will have 2 timestamps, from the origin and destination. SCADA station must display and serve all alarms and history with both timestamps.** Yep, all comes down to database design. This is possible. > [@R.Alamsha](#): > > Kindly keep in mind, the data collection and processing load is very high on the JSON API and FTP file parsing servers. There are some high volume data server agents contracted by the client. Most of the JSON payloads served by tham are deeply nested from health services, financial services and so on. The objective of this project is to correlate all the information with the help of ML/AI algorithms and provide specialized services for 1000's of consumers. Clients pay big amount for this service and they demand high quality service. Define a high processing load. The JSON API would probably be a set of REST endpoints, and FTP, well that is just an FTP server with Ignition watching the folders, and processing on change. As stated previously, MQTT, PubNub, TCP/UDP streams, web sockets, all are possible with Ignition. > [@R.Alamsha](#): > > Needless to mention that multiple servers will be working in tandem on cloud to collect, process and serve the IOT data. _Please feel free to ask me, if you don't understand any part of this spec._ ---
**Author:** ![Kyle\_Chase](https://avatars.discourse-cdn.com/v4/letter/k/9de0a6/32.png) [@Kyle\_Chase](https://forum.inductiveautomation.com/u/Kyle_Chase)\ **Post date:** [June 1, 2018, 7:57pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/45 "2018-06-01T19:57:27Z")
Who is your RTU vendor? I have been building SCADA systems for 15 years, and MQTT is the biggest jump from an architecture standpoint in my career. Every system I have ever built would be magnitudes better using MQTT as opposed to Modbus or whatever other protocol we were using. I disagree with your vendor, and based on how many vendors there are adding MQTT to their products. I suspect that either your RTU vendor doesn’t know what they are doing, or they have alternative motives for not supporting MQTT. Their analysis of the protocol is just wrong. I am not saying remove Modbus/FTP, but I can not see how they came up with that conclusion. As an example, their has been a company using MQTT for over 20 years, in the Oil and Gas industry. I could be wrong, but they have 20,000 miles of pipeline, and the MQTT infrastructure has had 99.999% reliability over that time frame. I can assure you it is not just suitable for home automation and smart sensors. This installation has been pulling from (1,000s,10,000s, I can not remember the number, but a lot) of PLCs and RTUs for that period, and rather successfully. To sum this part of our conversation up, MQTT was built for industrial use, and there is a lot of successful use cases of MQTT in industry. ---
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 1, 2018, 8:56pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/46 "2018-06-01T20:56:31Z")
> [@post:44](#): > > Scalable, unlimited tags and clients capability. (Ignition is best qualified here) > [@Kyle\_Chase](#): > > Yes, Ignition is good at this, and they are getting better with every release. This is one point where Ignition beats every SCADA vendor hands down. I have strongly recommended Ignition for this and hope it goes well. > [@post:44](#): > > Collect IOT data from JSON API's servers and csv files sent by thousands of RTUs by FTP. The FTP files are encrypted as per the latest internet standards and the data collector must run the file parsing decryptor supplied by the client and serve the data. > [@Kyle\_Chase](#): > > Yes, this is possible with Ignition, the built in FTP libraries, and python. I agree with you. I think, IA can build a custom Ignition data server which simulates a big software Modbus Master/Slave RTU on Linux + retain last tag values + serve Alarms and Tag data as JSON Payload to IT servers + serve web pages to Ignition clients. Other things IT will take care. I have proposed a pilot project to start with and build upon it gradually. Let's see how it goes. Thanks for the wonderful discussion. ---
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 1, 2018, 9:14pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/47 "2018-06-01T21:14:47Z")
> [@Kyle\_Chase](#): > > As for file transfer over MQTT, yes, it is possible and reliable. We have transferred video, images, VPN configuration files, SSL certificates, and even firmware upgrades over MQTT. You love to use MQTT for file transfer and i love to use FTP. I will never use MQTT for file transfer. > [@Kyle\_Chase](#): > > To sum this part of our conversation up, MQTT was built for industrial use, and there is a lot of successful use cases of MQTT in industry. I am happy with Modbus + FTP RTU. I don't want MQTT RTU. For home automation projects, i 100% support MQTT. But for data collection, i will use only FTP. Again HA is not my core domain and i can't discuss much on this. ---
**Author:** ![PRAMANJ](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pramanj/32/66935_2.png) [@PRAMANJ](https://forum.inductiveautomation.com/u/PRAMANJ)\ **Post date:** [June 2, 2018, 4:32am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/48 "2018-06-02T04:32:59Z")
For me as well the ftp bit instead of mqtt or pubnub was confusing ! If you are confident that your RTUs can communicate with a central computer using ftp then it's not difficult to interface it to ignition server using jython scripts on server or nodejs using JavaScript. I am a fan of JavaScript and modejs. As files are received from RTUs over ftp they can trigger an event on modejs to update ignition tag values in memory. Nodejs is required as JavaScript cannot do file iio , where as it's excellent in communication. We will not need the other modules of arscada and m scada , they will be optional As I had mentioned and you also suggested it's better to pilot it on a simple test case. Both approaches of jython and modejs can be tried. I can do the modejs approach faster and someone else can do the jython approach faster. At least we will understand each others requirements and approaches better. ---
**Author:** ![PRAMANJ](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pramanj/32/66935_2.png) [@PRAMANJ](https://forum.inductiveautomation.com/u/PRAMANJ)\ **Post date:** [June 2, 2018, 4:55am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/49 "2018-06-02T04:55:13Z")
In reply to you question "python scripts also use wesocket?' I guess python scripts on server will update ignition tag values using its appropriate communication library, (you can refer to it's documentation) to update tags values once ftp files are received . There is no wesocket involved here. Njscada also uses [socket.io](http://socket.io) only for communicating with its arscada or mscara clients. If we are not using these modules of njscada then you are using njscada as nodejs server for file io as JavaScript cannot do disk read write. You can use ignitions unlimited clients for CCC and mobile clients that it supports. ---
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 2, 2018, 5:04am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/50 "2018-06-02T05:04:40Z")
> [@PRAMANJ](#): > > If you are confident that your RTUs can communicate with a central computer using ftp then it's not difficult to interface it to ignition server using jython scripts on server or nodejs using JavaScript. > [@R.Alamsha](#): > > I think, IA can build a custom Ignition data server which simulates a big software Modbus Master/Slave RTU on Linux + retain last tag values + serve Alarms and Tag data as JSON Payload to IT servers + serve web pages to Ignition clients. Please go through my specs once again patiently and it's final. My reply above to Kyle\_Chase is what i wanted from Ignition and i hope IA can do it. Your JS/Python coding skills and deep knowledge on Ignition engineering will be a very good asset for any Ignition project. I have already decided to engage you, if everything goes well. ---
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 2, 2018, 5:24am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/51 "2018-06-02T05:24:20Z")
> [@PRAMANJ](#): > > In reply to you question "python scripts also use wesocket?' I guess python scripts on server will update ignition tag values using its appropriate communication library, > [@Kyle\_Chase](#): > > MQTT was built for industrial use, and there is a lot of successful use cases of MQTT in industry. [http://forum.inductiveautomation.com/t/mqtt-issues-in-ignition/16789/8](http://forum.inductiveautomation.com/t/mqtt-issues-in-ignition/16789/8) Please post your views there. Thanks. ---
**Author:** ![PRAMANJ](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pramanj/32/66935_2.png) [@PRAMANJ](https://forum.inductiveautomation.com/u/PRAMANJ)\ **Post date:** [June 2, 2018, 5:56am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/52 "2018-06-02T05:56:13Z")
Only missing link for the jigsaw puzzle is the FTP RTUs. Pubnub also provides interface with Watson and offers capability of unlimited storage of messages sent or received thru it on it's servers in cloud for persistence. But we are not yet decided if pubnub or mqtt are required at all if ftp is going the job! I will be glad to do a pilot with you on experimental basis. ---
**Author:** ![PRAMANJ](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pramanj/32/66935_2.png) [@PRAMANJ](https://forum.inductiveautomation.com/u/PRAMANJ)\ **Post date:** [June 2, 2018, 7:06am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/53 "2018-06-02T07:06:44Z")
But of course njscada can talk to your RTUs as well on [socket.io](http://socket.io) (web sockets) in addition to mscada and arscada clients (if any!). However On RTUs you have to send packets as stringified JSON files rather than ftp. We have see if nodejs supports FTP client and server npm then it can directly send receive ftp files directly to you RTUs. ---
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 2, 2018, 8:23am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/54 "2018-06-02T08:23:09Z")
> [@PRAMANJ](#): > > Only missing link for the jigsaw puzzle is the FTP RTUs Don't worry. That's my baby 🙂 > [@PRAMANJ](#): > > But of course njscada can talk to your RTUs as well on [socket.io](http://socket.io) (web sockets) I would like to call the front end data server as Commander 🙂 My first preference for Commander is customized Ignition. If IA can't do that, i will look into your offer. Nevertheless, your njscada experience elevates you as one of the best Ignition experts. My opinion is, you can make more money with Ignition than njscada. Ignition is like Rolls Royce. Whereas njscada is not even a product in the first place. It's a very good showcase of your coding skills. Can you submit an evaluation copy and compliance statement of njscada to the consultant?. Without this, it will never ever prequalify as a SCADA product anywhere in the world. ---
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 2, 2018, 12:29pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/55 "2018-06-02T12:29:40Z")
I think, you can repackage your njscada for home automation on Ignition Edge platform. If you like. we can discuss at the MQTT thread posted above. i think, you will get very good support from IA too. ---
**Author:** ![PRAMANJ](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pramanj/32/66935_2.png) [@PRAMANJ](https://forum.inductiveautomation.com/u/PRAMANJ)\ **Post date:** [June 2, 2018, 8:36pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/56 "2018-06-02T20:36:26Z")
Njscada is targeted as an add on to ignition. The front end ftp server can be built in njserver or a third party open source Linux based server that you have used in your project itself! From there data can be pumped in and and out of ignition using jython scripts or njscada. Another advantage of njscada will be additional interface of arscada and mscada clients if customer prefers. ---
**Author:** ![PRAMANJ](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pramanj/32/66935_2.png) [@PRAMANJ](https://forum.inductiveautomation.com/u/PRAMANJ)\ **Post date:** [June 2, 2018, 8:43pm UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/57 "2018-06-02T20:43:14Z")
Regarding your home automation market for njscada it's possible to integrate with ignition edge thru OPC server. Alternatively it can integrate directly with RTU s on SNMP or ftp protocols without the edge . ---
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 3, 2018, 2:48am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/58 "2018-06-03T02:48:10Z")
> [@PRAMANJ](#): > > Regarding your home automation market for njscada it's possible to integrate with ignition edge thru OPC server. How will you collect data from HA devices to the OPC server?. > [@Ignition Edge - Most powerful Home Automation server in the world](http://forum.inductiveautomation.com/t/ignition-edge-most-powerful-home-automation-server-in-the-world/17129/7): > > I have engineered decent Building Automation projects on Tridium, a very powerful BAS integration platform from Honeywell. So, i think i am qualified to speak on this subject. I have always been looking for a rock solid HA server for high end apartments, villas, bungalows and ranches of millionaires and billionaires. Till date i couldn’t find one, until it suddenly dawned on me “why not Ignition Edge?”. Read on… MQTT is emerging as the ultimate protocol for HA and IOT business because of it’s … ---
**Author:** ![PRAMANJ](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pramanj/32/66935_2.png) [@PRAMANJ](https://forum.inductiveautomation.com/u/PRAMANJ)\ **Post date:** [June 3, 2018, 4:46am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/59 "2018-06-03T04:46:55Z")
Sorry for my late and brief responses I am enjoying holidays in Europe till mid June. HA devices I guess communicate over zigbee,modbus, backnet ,modems and rtus We need drivers for these in opc ua server in ignition edge or njscada ---
**Author:** ![R.Alamsha](https://avatars.discourse-cdn.com/v4/letter/r/e5b9ba/32.png) [@R.Alamsha](https://forum.inductiveautomation.com/u/R.Alamsha)\ **Post date:** [June 3, 2018, 5:00am UTC](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194/60 "2018-06-03T05:00:43Z")
> [@PRAMANJ](#): > > HA devices I guess communicate over zigbee,modbus, backnet ,modems and rtus We need drivers for these in opc ua server in ignition edge or njscada No. All those projects have failed and trashed. I have updated my post above. Pls check it. [Previous page](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194.md?page=2) [Next page](https://forum.inductiveautomation.com/t/websocket-for-real-time-high-performant-scada/18194.md?page=4)