I think he thought that it was his then so called wife Guddi touching him.
I think that baby bump was some pillow! Only when Vividha entered, Guddi became possessive about atharva. If you remember, even Vividha was comfortable when Ravish touched her waist during dances. And other times too.