Paul didn't shun unbelievers. This is about shunning
babblings, a particular kind of talk. But there's more, this is not
about unbelievers, but about professing believers! The advice here is
given to Timothy, and the contents is that he should avoid a
particular kind of preaching. As John
Gill comments:

The ministry of false teachers is mere babbling; a voice, and
nothing else, as the man said of his nightingale; a sound of words,
but no solid matter in them; great swelling words of vanity, like
large bubbles of water, look big, and make a great noise, but have
nothing in them; contain nothing but vain, empty, idle, and trifling
stuff; what is unprofitable and unedifying, yea, what is profane,
contrary to the nature and perfections of God, and not agreeable to
the doctrine which is according to godliness; and being palmed upon
the Holy Scriptures, is a profanation of them. And all such wicked and
empty prate, and babbling, is to be shunned, avoided, and discouraged,
refused, and rejected; and, as much as can be, a stop should be put to
it, both by ministers and hearers of the word.

17
And their word will eat as doth a canker: of
whom is Hymenaeus and Philetus;

(2:17-18)
Hymenaeus and Philetus are condemned for disagreeing with Paul
about the timing of the resurrection. (See 1 Tim.1:20 and 2
Tim.4:14-15)

18
Who concerning the truth have erred,
saying that the resurrection is past already; and overthrow the faith of
some.

19
Nevertheless the foundation of God standeth sure, having this
seal, The Lord knoweth them that are his. And, Let every one that nameth the
name of Christ depart from iniquity.

"The Lord
knoweth them that are his."
This verse was used by the Catholic Church during the inquisition
to justify killing those suspected of heresy. ("Kill them all, for
`the Lord knows them that are His'.")

The church of Rome has always opposed Christ. Killing
those who follow Christ is a sure sign of being the
anti-Christ.

20
But in a great house there are not
only vessels of gold and of silver, but also of wood and of earth; and some
to honour, and some to dishonour.

21
If a man therefore purge himself from
these, he shall be a vessel unto honour, sanctified, and meet for the
master’s use, and prepared unto every good work.

22
Flee also
youthful lusts: but follow righteousness, faith, charity, peace, with them
that call on the Lord out of a pure heart.

23
But foolish and unlearned
questions avoid, knowing that they do gender strifes.

24
And the servant
of the Lord must not strive; but be gentle unto all men, apt to
teach, patient,

"Be gentle unto
all men, apt to teach, patient."

25
In meekness instructing those that oppose themselves;
if God peradventure will give them repentance to the acknowledging of the
truth;

26
And that they may recover themselves out of the snare of
the devil, who are taken captive by him at his will.

The ones who are taken captive are the ones who oppose God's ministers (verse 25). They think
they strife for the truth, but the reality is they oppose it. They
think they are doing their own or perhaps God's will, but in fact they
do what the devil would like them to do. So Paul instructs that also
the more than difficult members in the Church should be taught in
meekness, in order that they might be saved too.
So this verse does not teach that the devil can capture anyone at
will, but it teaches that those who sin do the devil's will. That is
what he wants them to do.