I'm pretty sure this is essentially how it works on all platforms, not actually closing, but instead staying running in the background. In any case, Steam > Exit in the dropdown menu should completely close it.