What's so is you feel uncomfortable and embarrassed and you believe it's because of the reasons you've given, but when you stated your reasons (in your letter) the problem didn't disappear. This is because you haven't gotten to the truth yet. Once you tell the truth, of what's really bothering you, you'll have an entirely new set of problems.

Experience tells me that you are not in communication with anyone. You have lapsed into doing your imitation of communication with your husband and daughter. How do we know? We know because when communication takes place problems disappear, conversely, when talking takes place problems persist.

It's most likely you have never been in communication with anyone and that the moments of love you've had with your husband were but unconscious happenings. I say this because once one knows how to cause communication to take place the ability stays with them for life. That is to say, all it takes is for one partner to intend for communication to take place and the other joins in the co-creation, or, they extract themselves from the conversation/relationship. I.e. "I don't want to talk about it." or, "Don't ever mention that again."

You can tell you've been stuck using the adversarial communication model because of the withholds. If you'll look back, even before you met your husband, but certainly on the first date, you found yourself withholding certain thoughts; this is one of the primary characteristics of the adversarial communication model. Most likely you had not had any experiences of communicating openly, honestly and spontaneously, (zero thoughts withheld) in your family; for most it sounds like a "real good idea: until they come across a thought they are afraid to communicate.

Withholding the first thought is the beginning of the commitment to mediocrity in a relationship. In fact, your breakdown in communication reveals that that's where you are now, back then, when he said something that was a clear indicator of where he was heading; had you been conscious, and operating from integrity, you would have said your truth and he would have realized that you would not be aligned with him on his path.

When you withheld certain thoughts from your husband during your first date you compromised your integrity. It's quite common for couples to have only a few peak experience of joy during their engagement and few, if any, thereafter. Neither of you have learned how to create and recreate joy at will through a single sit-down conversation.

What you're looking for is the first time in your life that you compromised your integrity; you let something slide. With him, you voiced an objection to moving into a neighborhood that was not comfortable, and you didn't make sure he "got" your consideration and so you still have it.  It could even be that you went along with a huge mortgage, one you've never been uncomfortable with, because you know that without his income you'd have to sell the house; alarm bells rang but you surrendered, ostensibly as a gesture of teamwork and partnership, but in truth surrendering is how you manipulate him into providing for you.  I suspect that you attracted and married someone who has been leading you in ways that conflict with your integrity. You both are presently ripe for bankruptcy; that is to say, your leadership-communication skills are supporting him in behaviors that lead to bankruptcy. That you intend he ignores your feelings suggests he believes that you'll submissively go along with it if he declares bankruptcy and causes others to lose money or, that wimpy you could be conned into forgiving him for infidelity? His treatment of you reveals that he is on the way down; such arrogance begs to be humbled. Put another way, you, using your leadership-communication skills, have been unconsciously intent on having him crash and burn so as to teach him what you need to learn.  

It could be said that you've been asked to wear fancy jewelry when in truth you're of the opinion that such ostentation is inappropriate in a world in which people are still homeless and hungry. Possibly you believe that what would make you happy is a life of sufficiency, one that supports generosity and community service, rather than showy accumulation and ostentation. For some it's uncomfortable to socialize with people who are unconscious, prattling about things that don't add to the aliveness of others. Few conversations at your present social block-parties make a positive difference in your community—evidenced by lots of badmouthing and gossip.

In any case, you've sold out. You've set it up for him to invalidate your brilliant intuition, your feedback, after which you made him wrong. This is abusive of you. It doesn't speak well of how you've been using your equally powerful leadership-communication skills; you've been leading him in a way that supports him in verbally abusing you, speaking condescendingly to you, ignoring your considerations. He's supposed to know that your happiness is of paramount importance. Men who commit themselves to serving their spouse are rewarded ten-fold. That you are unhappy and nothing you say can effect a change reveals that you both need relationship coaching. I recommend that you begin by doing The Clearing Process, by yourself, (it's free), one clearing per day for five days in a row.