There is no debate about this. Children should not be taught to swear or use foul language. Parents, teachers and guardians have a responsibility to try and avoid exposure to such language.
To me the moral debatable dilemma arises when children are inadvertently exposed to such language by various, sometimes older siblings, family members or even random observations of people. Heck kids will eavesdrop on peoples conversations in a grocery checkout line and pickup something naughty that way.
The question is when a child does this – what do you do? One school of thought is that the word/behavior is inappropriate and you reprimand them for it immediately and sternly nip the habit in the bud.
The other school of thought is that from a small child's perspective it is innocent imitation. They may have an inkling that its bad, but their minds are clean and they are merely imitating without knowledge of context. They are not actually swearing or using foul language like a teen or adult would. Reprimanding them makes them want to do it more. It also impresses on the brain the notion of taboo. They become more aware that certain words and things are naughty or inappropriate. They slowly start thinking like us at an earlier age and developing notions of swearing and foul language. Hence, instead of preserving innocence you are actually taking it away. Some parents feel that for very young children you don't do anything – and start disciplinary behavior only when they are old enough.
The counter to that is that with media and societal exposure, other kids and siblings – it is not always innocent imitation but children are already aware of context. Moreover, not cutting them off in time could put them on their way to thinking that there is nothing wrong with such behavior and making a habit of it.
Similar two schools of thoughts on parenting also exist with things over little kids singing and dancing and imitating music videos of people like Britney Spears, Kesha, Lady Gaga, Katy Perry etc. A lot of the lyrical content is not really kid appropriate. What I love best is that 'I kissed a girl' won Nickelodeon Kids choice awards.