I think this is probably a bug... I'm guessing that all the various things that actually reference the tag provider (like a project) haven’t opted themselves in to being a reference.
One issue though, when the tag provider is renamed, memory mode tags revert to their ‘Default‘ value. I assume this is due to them having a different reference in the database.
Is it possible to rename the tag provider but still keep the value they had before the renaming?