How to ensure module is actually removed on shutdown()

Module Development
1

Problem

We are working on a module that has unexpected behavior when restarted or uninstalled/reinstalled. I believe these issues are caused by resources that are not removed by the module shutdown() method, and I'd like to better understand how we can locate these resources to ensure they are all removed.

Background

The module we are working on throws errors in the logs and unexpected behavior occurs when the module is restarted or uninstalled/reinstalled. In every instance, this behavior is fixed by restarting the gateway.

From the SDK Programmer's Guide:

Important: You should take special care to remove all resources added by the module to the platform. If any resource is left uncollected, the entire module will remain in memory, leading to a memory leak, which will be particularly troublesome if the module is reloaded several times in one gateway session.

Progress made

I think the above statement typifies the issues we're seeing - code continues to execute on the gateway from the uninstalled module, which causes errors or unexpected behavior when the module is subsequently restarted. We have begun solving these issues by terminating the objects created - specifically, calling .shutdown() or .close() methods on some objects.

These attempts have been successful - the errors or unexpected behavior has ceased once that object is removed.

Question

How do we ensure that all resources have been removed during module shutdown()? I'm looking for a general approach to locate and solve these issues when they occur, as well as any coding best practices to prevent them from happening in the first place.

Thanks for taking a look!

2

There's no general panacea. Static analysis tools and IDE inspections can help identify possible leaks, but there's no single answer (even Rust programs can have memory leaks given sufficient programmer effort).

All that said... Module reloading is going away entirely in 8.3, so in this particular area you can just get to "good enough" for 8.1.

3

There's no general panacea. Static analysis tools and IDE inspections can help identify possible leaks, but there's no single answer

That's what I feared, but thanks for clarifying! I'll look into static analysis tools; is there anything you've used in the past you can recommend?

Module reloading is going away entirely in 8.3, so in this particular area you can just get to "good enough" for 8.1

This is the first I've heard of that - do you mean the module restart button is going away, or is something more fundamental changing?

even Rust programs can have memory leaks given sufficient programmer effort

Challenge accepted :sunglasses: Haha thanks for the laugh and the help!

4

We use SonarQube internally. It's... okay.

Module reloading as a concept is going away. The gateway will start with whatever modules were installed, and there's a new state of "pending install" or something like it that basically means "the next time you start the gateway, this module will be added to the list of modules."

But it won't be possible, through any mechanism, to restart the entire module. It's up to you if you want to try hot-reloading within your own stuff inside the module.

5

The gateway will start with whatever modules were installed, and there's a new state of "pending install" or something like it that basically means "the next time you start the gateway, this module will be added to the list of modules.

Fascinating. I take it that means a gateway restart will be required for all module installs in 8.3?

6

Yep. We're also definitely trying to reduce startup time as a consequence :smile:

2 Likes