As I've written elsewhere regeneration is:

that activity of God wherein he radically transforms the moral fiber of a person through the unique work of the Holy Spirit. This transformation is analogous to a new birth where one begins his/her life (Jn. 3:3-7; 2 Cor. 5:17; Gal. 6:15; Tit. 3:5; Jm. 1:18; 1 Pt. 1:3; 1 Jn. 2:29)....Value systems are wholly renovated, not just altered. Old impulses and habits are replaced with new ones (Gal. 5:19-24; Col. 2:11-12) and a death takes place of the old self (nature), which was dominated by sinful desires and activities (Rom. 6:1-11-11; Gal. 2:20). In the place of the old life God renews the converted person and imparts new spiritual life never to be corrupted (1 Pt. 1:4).

How might this happen? How does the new displace the old? I once heard an analogy from chemistry that I thought was useful (albeit analogies are only useful for illustration). A mixture consists of not less than two chemical elements, each of which can be distinguished and separated from the other when combined in the same container. A compound, on the other hand, is the combination of two or more elements that are chemically bound to each other such that they cannot be extracted individually. In a compound the ratio of the individual elements remains constant (like water having 88.8% oxygen and 11.2% hydrogen), whereas in a mixture the ratio of the elements can vary.
Or, consider Aristotelian metaphysics. A compound could be likened to a picture. Once the picture is on the canvas, the paint cannot be extracted without compromise to the picture. The paint and the picture are inextricably tied together. The very essence of a thing, argued Aristotle, consists of the sum of its essential parts. When individual parts are combined in a compound (paint + picture), a new substance obtains. The paint in a sense becomes the picture.
Similarly, Augustine noted in Book XII, Chapter XXIX of his Confessions:

When a song is sung, the sound is heard simultaneously. It is not that unformed sound comes first and is then shaped into song....That is why a song has its being in the sound it embodies, and its sound is its matter. The matter is given form to be a song.

In other words, no sound, no song. No paint, no painting. Sound is to song as paint is to painting. The two elements, though separate, are so entangled that the one is bound up in the other.
Similarly with regeneration. When God's Spirit enters our souls (via faith in Christ) we become a new creation altogether. We are metaphysically changed from the inside out. When God's Spirit enters us we become an integrated being; a compound, metaphorically speaking, whereby God's Spirit unites with our human spirit to form one new substance. Though the uniqueness of each part (God's Spirit and ours) is evident, the predominating expression for those born from above is "Christ in you, the hope of glory" (Col. 1:27); our new life is now "hidden with Christ in God" (Col. 3:3). Thus, "if anyone is in Christ, he is a new creation; the old has gone, the new has come!"
Just thinking....
Paul
P.S. I refer to the human "spirit" not as a third part in the human make up, (also known as a trichotomous view of "body, soul, and spirit"), but only to refer to that immaterial aspect of our humanity; in essence our "soul." Humanity is a complex unity consisting of material and immaterial substance. Biblical usage suggests that soul and spirit are referring to the same aspect of our being.
[The quote from Augustine is from Paul Helm's Analysis 8 - The Gifts of the King. The original text can be found here.]