Yes it does matter. A marriage in the sight of God is when two people commit the sexual act. Then they become one. Even if its just a casual one night stand. Therefore is you sleep with someone else afterwards you have committed adultery in Gods eyes. Gods seventh commandment reads “Thou shalt not commit adultery”. So if you sleep with ten people you have ten marriages and yet you have not married one.
If you are a virgin and you marry someone who is not, unknown to you, then you are breaking the commandment but you are innocent to the fact and Jesus will forgive you if you repent properly and sincerely. Stay away from this sin as this could lead to hell itself.