When we do something wrong or something that can easily be perceived as wrong, we have the responsibility to apologiese & clarify or retract what we said - regardless of how the other party will react.
Telling the truth is not dependent on the listener’s response, is it? The truth must be told, even when people don’t believe it at first.