I'm aware that you pronounce "plus" with an s sound when it's positive and you don't pronounce it when it's negative, but if you wanted to ask for more soup, wouldn't this be ambiguous in writing? For example "je voudrais plus de la soupe" could mean either you want more soup or you want no more, depending on whether you pronounce the s or not? Is there a difference between "plus de soupe" and "plus de la soupe"? I almost always hear French people say "un peu plus" when they want more of something, is this an effort to avoid this ambiguity?

“*Plus de la soupe” is not correct. The partitive article in French is a bit complicated. It is normally du/de la/de l'/des, but when the existence is negated, it is de instead. This is the case in “je ne veux plus de soupe”: since the soup is negated, the partitive article is de. In “je veux plus de soupe”, the word de is a preposition introducing a noun complement, and when it is used to indicate the composition or purpose of an object, there is no article after de. This is the case in particular when expressing a quantity, thus “plus [plys] de soupe” (more soup), “moins de soupe” (less soup), “un peu de soupe” (a little soup), “un bol de soupe” (a bowl of soup), “un litre de soupe” (a liter of soup), etc.

1) It's a plus [noun] to be able to speak a foreign language.
2) I no longer know what to do. [ne...plus around the verb; an adverb for no longer]

To ask for more soup, plus de soupe functions like an adjective:
Je voudrais un peu plus de soupe. Plus de, moins dedo not take an article:

plus de soupe, moins de soupe, plus d'argent, moins d'argent, they are adjectives used with the partitive articles for uncountable nouns. More soup, less money, etc.

Following your logic: "But if you wanted to ask for more soup, wouldn't this be ambiguous in writing? For example "je voudrais plus de la soupe" could mean either you want more soup or you want no more, depending on whether you pronounce the s or not?"

Answer: Not exactly. It cannot mean no more. See the end of this comment.

Grammatically, it is: Je voudrais plus de soupe. Or: Je voudrais plus d'argent. I would like more soup or more money [adjectives]. However, to avoid this quasi-rude form, the French tend to say: Je voudrais encore un peu de soupe or un peu de soupe encore or even un petit peu plus de soupe. [adjectives]
It is also to avoid the confusion of ne plus vouloir de and ne vouloir plus de.

Grammatically, for: I do not want more soup, it would be: Je ne voudrais ou veux pas plus de soupe. However, bear in mind, the question will usually be: Est-ce que tu veux ou vous voudrez encore de la soupe? And the answer is usually: Non, merci, je n'en veux plus ou Je n'en voudrais plus. It is not ungrammatical to say: Je ne veux pas plus de soupe. But it is not usual.

Your question again:
Is there a difference between "plus de soupe" and "plus de la soupe"?
But it all depends on what question.

There is some cases of ambiguity with plus, but your example of « je voudrais plus de la soupe » seems a bit broken, it would sound strange and ill-formed in any context anyway. You can have either :

Je voudrais plus de soupe. (more of it, no ambiguity)

Je ne voudrais plus de soupe. (no more soup, no ambiguity)

(bold s where it would be pronounced if spoken out loud)

But yes, ambiguity can occur when the first part of the negation is dropped in casual forms of speech :

Je [ne] veux plus de soupe. >>> Je veux plus de soupe.

Here I switched tense to simple present to fit the casual context because "voudrais" sounds slightly more polite and formal, but the point is the meaning depends on how it's pronounced, so in conclusion, yes, « Je veux plus de soupe. » has an ambiguous meaning in written form.

Regarding the part about un peu plus, I admit I never noticed this usage of the expression to avoid the ambiguity.