In theory, sure. Lots of communes have been organized on this basis, for
example. Larger socialist groups could also be formed on a voluntary basis,
if you could find enough volunteers. The problem arises when you start
talking about how to run an existing society.

Then you have a circumstance where the group is not composed of volunteers.
Instead you round up everyone who happens to live in a geographic area, and
tell them they all have to live by your rules. No matter what those rules
are, a significant fraction of the population won't like them - so you have
to force them to go along.

Thus, any nation-state is ultimately based on force. The question is how
best to minimize the amount of force that must be used in order to get a
functioning society. If this is a central issue for you, then socialism is
a bad deal - it requires lots and lots of rules, most of which are against
the best interests of those who follow them, so they all have to be backed
up with force.

For socialists, this isn't a problem. The naive ones convince themselves
they won't ever need to actually shoot anyone, and the more insightful ones
simply believe that other goals are more important than minimizing violence.
If, however, you believe that minimizing the initiation of the use of force
should be one of our top goals, it is very difficult to justify any of the
more expansive types of government.