I guess it's all about balance isn't it? Too much of anything (other than fun) is bad. Well, even too much fun can be bad, it can dilute the feeling ;)
Doing 'not enough' scared with code complexity can be as bad as doing 'too much' scared about inflexibility...

I tend to find that erring on the side of doing too little is a safer bet. If you constantly reflect on your design, you'll be able to recognize what is slowing you down and correct your course. I think Sandi Metz has a great perspective on this. Here's a quote from her book:

Every decision you make includes two costs: one to implement it and another to change it when you discover that you were wrong. Taking both costs into account when choosing among alternaties motivates you to make conservative choices that minimize the cost of change.