Breast tissue changes as you age. Those changes cause the breasts to lose elasticity and fat. And that causes sagging. But aging isn't necessarily the reason; an extreme change in weight, pregnancy, breast-feeding or good old genetics can also cause sagging and drooping. How much time you spend not wearing a bra can do it too.