“Public opinion is always more tyrannical towards those who obviously fear it than towards those who feel indifferent to it.”