The dilemma of the technical cofounder

I’ve been working in my company for a bit more than a year. We’re a team of just two.
We’re still validating our business idea, which is fine. We’re making sure that whatever we do actually works, rather than just doing something for the sake of doing it.

A problem I’ve been facing lately is finding the sweet spot between making code that scales and it’s sustainable in time versus having a feature done as quickly as possible.

As we’re still validating our business idea, pretty much everything we do starts as a small experiment, but when we realize it could work we start to put more effort into it.

There’s a moment when I realize that the code that started as a doodle is actually a dozen classes now and that it has become a pain to work on that feature, because it was never properly designed.

I’ve heard comments along the lines of “release as fast as you can” or “code that does what it’s intended to it’s all that matters”, but I disagree.
Having code that’s not properly designed stops you and your company from moving on to more complex features in a timely (and flexible) manner. The funny thing is that I would just “waste” as much time if I were to design everything properly, because a big portion of the ideas we try never reach users.

I guess the trick is learning how to identify the moment when an idea stops being a small hack and turns into a proper feature.