“A father is very miserable who has no other hold on his children's affection than the need they have of his assistance”