Yours is a good question. The fact is that you define a potential [itex]\phi[/itex] if the field Eis conservative. But a field is conservative if its rotor is zero (irrotational).

This means that considering Maxwell's equation [itex]\nabla \cross \vec{E}=-\frac{\partial \vec{B}}{\partial t}[/itex], you can argue that IF [itex]-\frac{\partial \vec{B}}{\partial t}=0[/itex] then you can define [itex]\phi\backepsilon' \vec{E}=-\nabla\phi[/itex].

However, if you define [itex]\vec{B}=\nabla\cross\vec{A}[/itex], you could write [itex]\nabla \cross \vec{E}=-\frac{\partial \vec{B}}{\partial t}=-\frac{\partial}{\partial t}\nabla\cross\vec{A}\Rightarrow \nabla\cross\{\vec{E}+\frac{\partial\vec{A}}{\partial t}\}=0[/itex] so that the field [itex]\vec{E}+\frac{\partial\vec{A}}{\partial t}[/itex] is irrotational, then conservative, so that [itex]\vec{E}+\frac{\partial\vec{A}}{\partial t}=-\nabla\phi\Rightarrow\vec{E}=-\nabla\phi-\frac{\partial\vec{A}}{\partial t}[/itex]

This is a more general definition of the potential of the electric field that fits with Maxwell's equations. However, there is a term missing in my last expression related to a gauge transformation, but since it is a term related to the frame of reference, setting proper condition it can be put to zero.