Kindling - a collection of tools to help with Ignition

So the intermediary output from my script is a CSV like this:

2022-02-15 15:15:00,0
2022-02-15 15:20:00,0
2022-02-15 15:25:00,1
2022-02-15 15:30:00,3
2022-02-15 15:35:00,2
2022-02-15 15:40:00,1
2022-02-15 15:45:00,2

For each 5 minute interval there is a timestamp of when it starts and then a count of how many log entries were in that time period (after filtering).

The graphical output would ideally be like a time trend chart from Perspective/Vision. I’ve been too lazy to figure out how to get datetimes nicely on my X axis and instead groom them to Unix timestamps, but my graphs look like this from my spreadsheet software:

From this I can notice spikes, maybe notice patterns, and find interesting time periods to look at the log more closely. Like in this one, I might be curious why I had over 90 log messages in 5 minutes roughly around time 1644675000 (February 12, 2022 2:10:00 PM). (I use quite a lot…)


Hm, I like that idea a lot. I’ll have to think about how to implement it, but I’ll keep thinking about it.
I made a formal issue, if you’ve got a Github account feel free to add any thoughts: Graph-view (log events / time slice) · Issue #44 · paul-griffith/kindling · GitHub

1 Like

Hey Paul, I have a few requests:

  1. Are you able to add an option to remove the timestamp from the log comment? This makes it easier to see repeating errors.

  2. It looks like you’re converting the local timestamp into GMT?

from the raw wrapper:

INFO   | jvm 1    | 2022/03/08 19:19:44 | W [g.D.Route                     ] [08:49:44]: Error handling route. route=/fetch/:project/:uuid, route-group=perspective, route-path=/fetch/:project/:uuid

where 19:19:14 is the local time (+10:30) and 8:49:44 is GMT time

from Kindling:

Would you be able to add an option to select between GMT and local?

  1. Would you be able to add an “export to CSV” option or to Excel? Just an FYI, in the script I wrote to parse wrappers, I replaced all double quotes with single quotes so that I could wrap each row cell in double quotes, unless of course you use an obscure delimiter. I assume if you export to Excel directly though you wouldn’t need to do that.

Hmm… Maybe. It should be possible to extract it safely. I don’t know offhand whether the timezone ever differs from the actual wrapper log time.

Yeah, for lack of a better option. I can put in a timezone selector (that defaults to local); that’s probably the easiest thing. Timezone selector · Issue #47 · paul-griffith/kindling · GitHub

1 Like


Haha nice. My reasoning for this is so that we can more easily track using Excel which issues we’ve resolved and which we’re still looking into, and add additional columns etc.

The two timestamps are always different in my wrappers, with the main one that includes the date being local and the one in the log comment being GMT which only has the time part (which is a bit odd, especially with me being +9/10:30 different, the local date is very likely to be different to the GMT date, assuming of course that the gateway timezone is set to local time and not GMT.)

7 posts were split to a new topic: Windows Problems

Adds log ‘density’ display to scrollbar in log viewer (thanks for the idea, @justin.brzozoski)
This is an experimental/incubating feature. Currently, logs are aggregated into 2-minute buckets and rendered in the scrollbar:

I also changed the wrapper log parsing to remove the extra timestamps (@nminchin). And added a timezone dropdown at the top of the panel.


Thanks for adding timezone :slight_smile: and that density graph looks sweeeet :heart_eyes:

By dumb luck, the recent log I grabbed off one of our gateways to try out the new Kindling had one huge spike in density. I didn’t know that going in and was kinda confused why the graph was mostly empty when I opened it.

That one faint spike 70-80% of the way down just dwarfs everything else:

And that spike was from a mostly harmless bug in our code I’m off to go fix now… :slight_smile:

I’ve noticed similar things a few times; I might try to get fancier and do a log scale horizontally or cap the absolute magnitude. What I’m doing is basically manually rendering a graph and then scaling it down to fit inside the scrollbar, so when there’s lots of detail you can get some very faint lines.
I’ll probably add time-slice selection in the column settings in the next release - maybe 1, 2, 5, 10 minutes or something like that.

I suspect you also know this, but it’s difficult to tell where the log lines currently in the view exist on the time graph. The scrollbar is linear with number of lines, the time graph is linear with time, and they don’t align. I briefly (instinctively) tried to move the scroll bar on top of the spike, then realized that wasn’t going to work. I’m not sure what simple solutions exist for this issue. Maybe just highlight where the current viewable log lines are on the time graph in a second color? But that would require updating/refreshing your scaled graphic, so that might be more processing than it’s worth.

I don’t know what the good solution is here. Something to ponder for now.

I like Kindling a lot. Thank you for putting it together.

Some tools replace the scrollbar entirely with the mini map:

Some go side-by-side:

In each case the map has its own position indicator (not shown in first screenshot).

@PGriffith I couldn’t run it on windows

Can you provide any more information? The latest published version (0.2.1) works fine on my windows laptop. The tool I’m using, JDeploy, should automatically download an appropriate JDK for you.

0.2.2 is out now - a minor update to tweak the density display and fix the painfully-slow scroll speed of the logs view.

Hi @PGriffith is there a way to use Kindling to help us understand quarantine.xml files?
I often find things have been quarantined, but opening the file doesn’t help whatsoever as it is all encrypted. I just want to know what table in the database it is failing with!
Ideally also why it is failing…

Or do i need to get a different file for this analysis?

1 Like

The quarantine file is not generally very helpful (it’s missing too much information) but if you zip up the entire folder in the datacache directory in the install directory, you can open that zip using the ‘S+F Cache Zip File’ tool:

That should hopefully give you some insight into what’s going wrong.


Public version has jumped to 0.4.1, for reasons that aren’t very important.

Release 0.4.1 · paul-griffith/kindling · GitHub

Highlighting some changes:

  • Standalone executables are available for Windows and Linux users who don’t want to use JDeploy; see the releases page above
  • CSV and XLSX exports for some data formats (see #55, thanks to Josh Hansen)
  • ‘Legacy’ thread dump parsing from prior versions that didn’t output a JSON export (see #63, thanks again to Josh and to Corbin)
  • Support for pasting logs and thread dumps directly from the clipboard (see #65)
  • Added some UI and parsing for deadlocked threads (see #68, thanks again to Josh)

And some other miscellaneous UI tweaks and improvements on the backend. Anyone using JDeploy should automatically receive the update the next time they launch.


I’m not sure if it would even be possible but one thing that would be extremely useful is the ability to snapshot what queries are actually being run against a database connection.

Database Status Queries per second and long running aren’t very helpful to try and cut down on what is hitting certain databases.

Maybe it could turn on deeper logging for x amount of seconds, then parse it out the log?

Hmm. Currently Kindling doesn’t have any way to ‘reach in’ to a gateway and configure anything. Adding extra parsing to the logs to try to pull out patterns is doable, but tricky to generalize.