God bless you, brother. The question is not whether you presently attend a public church, but whether or not you have visibly forsaken the standards of faith which John describes in his epistle.
Test your heart. Do you desire to be under godly elders, worshiping Christ properly as a part of a visible body, and to benefit other believers locally? If you desire that, keep seeking it. For a time I was without a local church to attend because my conscience forbade me to attend the one I had been with... the pastor suggested we stop setting aside regular time each day for prayer, and to just "have faith that God will take care of things." It seemed he also supported persons who held onto corrupt entertainment to show that we had "grace", allowing an overt easy-believism. I wanted to be with a visible body with godly elders, and the Lord knew it. He revealed one a few months later.
What does the scripture say? Elijah went out to the wilderness, but never left the Shepherd. Even in the desert of solitude, the Lord provided his servant with a brook and bread from unlikely sources. At last the prophet was reunited with the faithful remnant of 7000, though for a time he wandered alone and knew God by not more than a small voice.
Regarding your text, 1 John 2:19, I believe the traditional understanding is "those who went out from unity in the doctrines essentially described by John in the 1st Epistle."
Matthew Henry comments,
They were once entertainers or professors of apostolical doctrine: "They went out from us (v. 19), from our company and communion;" possibly from the church of Jerusalem, or some of the churches of Judea, as Acts 15:1, Certain men came down from Judea, and taught the brethren, etc. The purest churches may have their apostates and revolters; the apostolic doctrine did not convert all whom it convinced of its truth. 2. More privately. "They were not inwardly such as we are: But they were not of us; they had not from the heart obeyed the form of sound doctrine delivered to them; they were not of our union with Christ the head." Then here is, (1.) The reason upon which it is concluded that they were not of us, were not what they pretended, or what we are, and that is their actual defection: "For, if they had been of us, they would no doubt have continued with us (v. 19); had the sacred truth been rooted in their hearts it would have held them with us; had they had the anointing from above, by which they had been made true and real Christians, they would not have turned antichrists." Those that apostatize from religion sufficiently indicate that, before, they were hypocrites in religion: those who have imbibed the spirit of gospel truth have a good preservative against destructive error. (2.) The reason why they are permitted thus to depart from apostolical doctrine and communion—that their insincerity may be detected: But this was done (or they went out) that they might be made manifest that they were not all of us, v. 19. The church knows not well who are its vital members and who are not; and therefore the church, considered as internally sanctified, may well be styled invisible. Some of the hypocritical must be manifested here, and that for their own shame and benefit too, in their reduction to the truth, if they have not sinned unto death, and for the terror and caution of others. You therefore, beloved, seeing you know these things before, beware lest you also, being led away with the error of the wicked, fall from your own stedfastness. But grow in grace, etc., 2 Pt. 3:17, 18.