Since your device doesn’t support the MaskWriteRegister function you’ll have to read, mask, then write the new whole register value via scripting.
An old post describing the scripting approach: Modbus UA
A newer post that uses Derived Tags: Troubles making a Derived Boolean Tag work as a mask for a Short Tag
The derived tag approach can be prone to issues when writing multiple bits at once. It requires that Optimistic Writes be enabled for the Tag Group the OPC tags reside in and there was a race condition we just fixed landing in 8.1.2.
edit: it is strange that the exception code is IllegalDataValue instead of IllegalFunction… can’t see if anything is weird because you didn’t upload the capture or show the request in your screenshot.