Asking whether they should be held to a higher moral standard is actually quite difficult to address. Higher than who? What the public often forgets is that the spotlight is upon politicians simply because of the job they do. Were the spotlight reversed, can any of us see ourselves as being morally better or worse than anyone else? Getting the job done and doing it with an honest intention is far more important than looking into all sides of morality, since morality is judged differently by different elements of society.
If their moral judgment gets in the way of doing the job to the satisfaction of the country, then of course those morals make their viability questionable. Who will believe a politician who lies about their private life? People say that they are not accountable for that element of their lives, though a liar is a liar, and the habit of dealing with problems in a dishonest manner is not restricted to politicians, but to all walks of life.