I'm 21 and don't want to babysit?

So, my boyfriend and I are 21. We moved in together a year ago. His mom is always calling to ask us to babysit his brother and sister! We live a couple cities away from them. I don't know if I'm being selfish or not, we have watched them a few times since we've moved in together; but it's so annoying to me that she calls and wants us to take them on our weekends. We work full time, and enjoy going out on Fridays and Saturdays with our friends. This has caused some turmoil between his mother and I, I feel she dislikes me now because of how often we turn down the babysitting. But we moved out ya know, it's not so much our responsibility anymore. What should we do?