# KSQL Module For Filtering Kafka Data

**URL:** <https://forum.inductiveautomation.com/t/ksql-module-for-filtering-kafka-data/56294>\
**Category:** Module Development\
**Created:** [February 18, 2022, 1:38am UTC](https://forum.inductiveautomation.com/t/ksql-module-for-filtering-kafka-data/56294 "2022-02-18T01:38:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![nicholas.robinson](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nicholas.robinson/32/22188_2.png) [@nicholas.robinson](https://forum.inductiveautomation.com/u/nicholas.robinson)\
**Post date:** [February 18, 2022, 1:38am UTC](https://forum.inductiveautomation.com/t/ksql-module-for-filtering-kafka-data/56294/1 "2022-02-18T01:38:54Z")

</div>

Before I go down this road, I wanted to check: does anyone have an existing module for KSQL?

Background:

We have a topic that contains sensor data for ~6000 locations and is terabyte size when data is there for just 3 days. This means to get data out of it for a single location, use of Kafka consumer is not feasible because even if we polled for just the most recent record, the return will be huge and 99% of it will be data not relevant to the location which is running Ignition.

That same data also lives in data bricks and we do have a JDBC connection to that, however by the time the data meanders its way to data bricks, it is around 1-2 hours old.

In the web portal for Kafka (Lense) you can issue queries in SQL against a Kafka broker and get a fully relevant response (i.e. include store number and sensor key in the WHERE statement).

So in order to make pulling from humongous data into a single site Ignition possible, I am looking into KSQL because I think it has potential to be exposed as Jython scripting in Ignition.

I simply wanted to check here if anyone has already had this experience or one similar.

Thanks,

Nick

---

<div class="post-metadata">

**Author:** ![cody.tamaki](https://avatars.discourse-cdn.com/v4/letter/c/b487fb/32.png) [@cody.tamaki](https://forum.inductiveautomation.com/u/cody.tamaki)\
**Post date:** [February 18, 2022, 10:57pm UTC](https://forum.inductiveautomation.com/t/ksql-module-for-filtering-kafka-data/56294/2 "2022-02-18T22:57:56Z")

</div>

There is a project on Github that does a bit of Kafka integration but I don’t think it specifically supports KSQL.

> **[GitHub - IgnitionModuleDevelopmentCommunity/Kafka: Kafka extension for Ignition](https://github.com/IgnitionModuleDevelopmentCommunity/Kafka)**
>
> Kafka extension for Ignition. Contribute to IgnitionModuleDevelopmentCommunity/Kafka development by creating an account on GitHub.