More children are living with their divorced or never married dads in the last 40 years, but look at what happened during the recession– more divorced dads with children and a dramatic drop for unmarried fathers. The employment status of unmarried fathers may have something to do with this change. More US Census Bureau demographics about living arrangements of children.