Kafka and Ignition 8

Ignition
1

Hello, I am trying to connect to my company’s Kafka instance from Ignition. We have connected to other pub/sub platforms like IBM MQ via MQTT but I cannot find how to connect to Kafka. Can anyone share how they were able to make this connection?

2

you can perhaps found some help at

1 Like
3

Hello. I just cloned the latest from GitHub today and imported it into Eclipse from there. I see 8 errors and 33 warnings. The errors are “A class file was not written. The project may be inconsistent, if so try refreshing this project and building it”. There are various types of warnings. Is there something missing? Refreshing didn’t solve anything.

4

@kevin.deyoung I have nothing to do with that module, wasn’t even aware of it till this thread, but just wanted to say that I was able to clone the repo an run mvn package and it built without error. Given that, I suspect that your issue is an environmental/IDE one. I haven’t used eclipse in a long time so I can’t help there, but can confirm that I was able to import the project into the free/open source version of Intellij community edition and it was able to build/run as I’d expect in that environment.

Not sure how actively that repo is watched/maintained, but it may be worth it to post an issue on the github repo, maybe someone else is using eclipse and can help with the setup/configuration.

5

I’m sure that you are correct on the environment issue. I’m not sure what version of the JRE is required and I am a novice at best when it comes to Eclipse & Maven. I’ll continue digging and see what I can figure out.

Thanks again.

6

I ran the module build using the AdoptOpenJDK 11.0.6. Given that Ignition is currently shipping with Java 11, you’ll want to use the same version for best compatibility.

JDK vendor shouldn’t matter: Azul, Amazon Coretto, AdoptOpenJdk, Oracle - all should be fine for developing and building the module, just make sure to note licensing implications if applicable to you/your organization.

7

this module just a consumer,but it import org.apache.kafka.clients.xxx.jar to ignition。
so we could make a producer in python just like below.enjoy!

#firstly install Kafka-Ignition-Module-signed.modl 
from java.util import Properties
from org.apache.kafka.clients.producer import KafkaProducer
from org.apache.kafka.clients.producer import ProducerRecord

producer = None
topic = "firstEvent"
servers = "192.168.2.174:9092" #"xxx:9092,1xxx:9092,xxx:9092"
props = Properties()
props.put("bootstrap.servers", servers)#xxx服务器ip
props.put("acks", "all")#所有follower都响应了才认为消息提交成功，即"committed"
props.put("retries", 1)#retries = MAX 无限重试，直到你意识到出现了问题:)
props.put("batch.size", 16384)#producer将试图批处理消息记录，以减少请求次数.默认的批量处理消息字节数
#batch.size当批量的数据大小达到设定值后，就会立即发送，不顾下面的linger.ms
props.put("linger.ms", 1)#延迟1ms发送，这项设置将通过增加小的延迟来完成--即，不是立即发送一条记录，producer将会等待给定的延迟时间以允许其他消息记录发送，这些消息记录可以批量处理
props.put("buffer.memory", 33554432)#producer可以用来缓存数据的内存大小。
props.put("key.serializer", "org.apache.kafka.common.serialization.IntegerSerializer")
props.put("value.serializer","org.apache.kafka.common.serialization.StringSerializer")
producer = KafkaProducer(props)
#print servers
#print producer
data = "this is the %s 0f 100 msg from ingition kafka producer"
try:
	for i in range(100):
		record = ProducerRecord(topic,1,data%(i+1))
		a = producer.send(record)
		#print a.isDone(),dir(a)
except:
	print "eeeeeeeeeeeeeeeeeeeeeeeee"
	raise
finally:
	producer.close()
8

Hello! I was hoping you could point me in the right direction, or had some more information on using MQTT to connect to IBM MQ. We already have used MQTT quite a bit, just curious how you were able to connect it to IBM MQ.

Thanks!