Invisibilty difference for different sides?

I’m working on a module for double-blind version of a tactical game. My goal is to use the Invisibility function (unless someone knows a better way) to hide one side’s counters from another, as long as those counters are outside the LOS of the enemy. The Referee would have access/visibility to each side’s counters.

Because I would like for (e.g.) Side A to NOT know when its counters have been spotted, I don’t want to use the “semi-transparency” function of invisibility to show when the counter is hidden; so I set the opacity to 100%, which works fine. But then how can I tell if the counter is hidden or not? Is there a way to have the counter appear 100% opaque to the player, but not to the Referee? I have been trying to figure out if there is a way to use the InvisibleToOthers property to do this, but so far without success. Any idea appreciated; thank you!