One view (not one that I completely/necessarily agree with):
Noam Chomsky once said that believing in freedom of speech means believing in the right of those with views which you despise to speak. Even hate speech is allowed in America as the First Amendment is designed so that if somebody uses racist language, it is better if everyone else uses their own freedom of speech to shoot it down, rather than having the government have to step in.
If the government steps in, it'd count as interference with people's political freedom and 'government oppression', and would set a precedent for the government to have to regulate speech. This would be called censorship.