Hi there,
Working on migrating our existing MQTT emission over to Kafka Event Streams and I'm encountering some throughput issues. There doesn't seem to be too much documentation on the inner workings of the actual Kafka handler, but from the Event Streams UI the buffer is always maxing out. I'm seeing loads in my dev of about ~1k events per second and the buffer filling up quite a bit before it suddenly goes down to 0 and starts again. Is the handler batching events or is it sending each event as an individual message to Kafka serially? Would it be better to write a custom script to handle these events? Are we limited to just a single producer or can we simultaneously run multiple producers for a single event stream?
Any help is appreciated, thanks!