They all carry on a bit though don't they. It is only because their adultery was such front line news that it seems wrong. The others have been divorced, except for Edward, but give it time, and Diana wasn't a complete innocent. Although, to be fair, she was at first. If she had died while married to Charles then her successor whether an adulteress or not would be Princess of Wales. And don't forget that Charles was to blame as well as Camilla, so does that mean no-one wants him as King or that he can't be king because he has committed adultery. I hardly think so, when I look at past kings.
P.S. Personally I don't care, but find the argument interesting.