You have every right to air your side and to talk about it to whoever you want. It's a part of your life, and he has no right to take that from you. If he has a problem with the things you may or may not be saying, that is his problem, and it is up to him to manage the fallout. He does have the right to discuss with you how you may or may not be presenting him as one part of managing the issue, but he has no right to tear you down or to make you do anything.

As the song goes... "At first I was afraid, I was petrified" https://www.youtube.com/watch?v=RPCaH0ermEw