Also, according to the docs, the syntax is system.util.sendMessage(projectName, messageHandler, ...).
The first 2 parameters are not marked as optional, so try putting them in.
I would also check the scope is what you expect and want. Does the handler also work if you do: system.util.sendMessage('myMessageHandler', scope="S")
I have run into issues trying to send messages with payloads of instances of my own custom classes. I would just check that your payload is exactly the data and types you expect.
That cannot be supported safely for message handlers outside the leaf project that created the custom class instances. There is a real risk of large memory leaks.
It seems odd to me that you're saying it works, but it may also have to do with where you are calling it from. Since message handlers are a project resource, I seem to recall there being an undocumented behavior where if it is called from within the containing project then it will still work. Or something along those lines.
Project Name should definitely be included in the call.