Okay, so there's a lot to unpack here.
This is because the data of the table is a list - not a Dataset. It's important to remember that targeted cell styling is not compatible with datasets, so you previously converted the data property of the table to a list of objects in order to apply styling.
You're encountering this error because you're treating your data in an inadvisable manner. I recommend only having one data source for area of data. In your current code, you're treating a view.custom.part.spec.data as the source for your table, but then you're treating the table as the basis for updating the view property. Try this instead:
rowIndex=event.row # row number of the commit
colName=event.column # column name of the commit
newValue=event.value
# Below, I've referenced the source-of-truth dataset instead of the "translated" props.data of the Table
data = self.view.custom.part.spec.data # Dataset [86R ? 8C]
newData = system.dataset.setValue(data, rowIndex,colName,newValue)
self.view.custom.part.spec.data = newData
Even this approach, however, assumes that the custom property is not a binding, which I suspect it is. If that property is the result of a binding, then you should be editing whatever source drives that binding (perhaps a query to update a SQL table).