It may surprise you to learn that there is only one place where the Bible specifically refers to Jesus living in a person's heart. Writing to the Ephesians Paul prays that believers would be “strengthened with power through his Spirit in your inner being, so that Christ may dwell in your hearts through faith” (Eph 3:17). The Bible far more regularly speaks of God's Spirit living in the hearts of believers, such as in Rom 5:5 where it states that “God's love has been poured into our hearts through the Holy Spirit who has been given to us.”
So, when it comes to beginning a relationship with God through Jesus Christ, encouraging someone to “invite Jesus into your heart” can be misleading. God calls people to repent (i.e., turn away) from their sin and believe (i.e., trust) in Jesus Christ to receive forgiveness for their sins and the gift of eternal life (Acts 3:19–21; Rom 10:9–10; Eph 2:1–10; 1 Thes 1:2–10). Those who repent and believe become children of God (John 1:12–13), and because of that, “God has sent the Spirit of his Son into our hearts, crying, ‘Abba! Father!’ ” (Gal 4:6). So, unless a person is turning from their sin and trusting in what Jesus has done for them to be forgiven, they are not a Christian—even if they have “invited Jesus to live in their heart.”
Having said that, it is a glorious reality that the Spirit of Jesus Christ lives inside his people. When God created humanity, he dwelled with them in the Garden of Eden (Gen 3:8). Because of Adam's sin, humanity could no longer dwell in God's presence (Gen 3:22–24). Once the nation of Israel was formed God dwelled with his people, first in the tabernacle (Exod 40:34–38) and then in the temple (1 Kgs 8:1–11). But when they, like Adam, continued to sin, God's presence eventually left the temple (Ezek 10). So God promised a day when he would institute a new covenant (Jer 31:31–34), and as part of that new covenant he would put his Spirit inside his people (Ezek 36:27).
To make that a reality the Son of God took on flesh to dwell among us (John 1:14). Because of his death, resurrection, and ascension to heaven, he has sent his Spirit to dwell in his people as the new temple of God (Acts 2:14–36). All of this is in anticipation of the day when, in a new heaven and a new earth, “the dwelling place of God is with man. He will dwell with them, and they will be his people, and God himself will be with them as their God. He will wipe away every tear from their eyes, and death shall be no more, neither shall there be mourning, nor crying, nor pain anymore, for the former things have passed away” (Rev 21:3–4).
As believers that is our hope, “And everyone who thus hopes in him purifies himself as he is pure” (1 John 3:3). As God's temple we must pursue purity in thought, attitude, speech, and action (1 Cor 6:19–20). Through his Spirit, Jesus lives in us to fight sin (Rom 8:12–17) and empower us to serve one another in love (Gal 5:13–26). That is what it means for Jesus to live in our hearts by faith.