I too had no issues with jagya leaving anandi and going to gauri as she was his adult choice.
What I did have issues with is the way he did it -- he first generated a story that she was not his equal by constantly berating her and picking at her 'negative' points to butress his argument that she was just a useless burden around his neck. Then he simply went with gauri, without even bothering to let anandi know that he no longer considers her as his wife. The way she finds out is that he enters into the haveli with another woman who is wearing a mangal sutra and sindoor.
I also have no issues with his leaving gauri and going back to anandi. We all have the right to change our mind. We make certain decisions when we have some information. Then when we get more information, we may make new decisions.
So he wants to leave gauri now ... then fine, okay.
But there is some sort of protocol as to how this relationship too has to be closed off. It cannot be just that easy that now he has decided that this relationship wont work, so he just boards a bus to Jayetsar after this, and that can be considered the end of the marriage even if it is just a 'so-called' marriage.