There are several reasons why this would probably be a not-so-good idea:

An object should have full control over its data. With direct field access, it doesn't know when a value gets changed and can't react accordingly. You might later want to cache some data, lazily initialize it, calculate it on the fly instead of putting it into a field, notify someone of a change or a myriad of other things. Without accessor methods, you don't have a place to put that behaviour.

(Some languages make it possible to switch from direct access to access through methods without the clients even noticing. Java doesn't, so it's often quite costly to do that switch later.)

Fields aren't polymorphic. If you later want to make use of polymorphic behaviour - for example getting a value from a different source in a subclass, or applying the DecoratorPattern - you can't do so with direct field access.

As someone else might expand on, good ObjectOriented design and experienced ObjectOriented developers suggest that developing message- and behavior-based systems is a good thing.