A better idea would be to treat the spectators like a "team." Team chat from spectators would only be seen by other spectators, while public chat from spectators would be seen by everyone. Trust me, you don't want to get rid of the spectators' ability to communicate with in-game players.
[SELECT ALL] Code:
(spectating, so chat is hidden)
Player1: Hey, I'm going to have to close down this server, so you might want to end the CM while you have the time!
(playing)
Player2: So, Player3, how about this Clan Match? :D
Player3: Oh, it's great. Best ratio I've gotten in a long time, I'll make sure I screenshot it at intermission!
2 minutes remaining
*server disconnects; all players are kicked out*
Player2: FFFUUUU
Player3: FFFUUUU
Player1: I tried to tell you both...