I think going at first with him just so he is comfortable and the counselor can meet you is not bad idea as my husband does that with me a lot. But after the first, at most second session, he will need to go it alone because as much as you think you are and can be 100% honest and open around your partner you see once they start hitting you with the hard shit you will find you can't say everything in front of them.

My husband had to go to a few sessions with me here and there to help with maintaining our marriage during the bad times. So occasionally popping in is a good idea if it starts getting rough for you both as a couple.