Isn't this wrong for this to happen?

It means you can't have a string of women and baby mama's. It doesn't necessary mean you have to marry. There are many verses that say if you can accept celibacy, it's the best.