Problems with step kids, and there mom - there seems to always be something!?

Long story short my husband and I have been married 4 years. He has a 9 and 7 year old, we have a 2 year old. After we got married he got an awesome job-but its shift work so every other weekend, holiday and half the summer I am the one who is the primary care taker for his kids. They hate me! Things were good with them while we dated and for the 1st year we were married, then here came all the rude and disrespectfull behavior. When I was prego I was shoved down out stairs, and broke my foot, a year later came a claim I slapped 1 of them in the face, DHS suspended my husbands visitation, pending the out come of a in home visit, which ended in our favor, especially when the DHS lady showed up with the info that it was the other child I slapped, we had proof from his dumb ex she had claimed it was the girl. I put my foot down and said I wasnt keeping them anymore-that lasted 15 months (he continued to talk to them via phone every day)