The issue isn’t with a family renting out their old flat to people, or building a box of flats with their retirement money.

It actually is the issue if they can’t actually afford to manage/maintain the property. Maybe it’s not their fault they’re a slumlord, but they made a choice to use a property as a rental, and if they can’t keep up with upkeep… that’s on them.