I tried several things:
1) I leave my xorg.conf unchanged, and set the variable machdep.allowaperture to 0. However, after reboot, X11 complains, so I suspect the current driver ("vesa" in xorg.conf) does need aperture

2) I left the variable machdep.allowaperture unchanged (at 2), and tried the driver "vesafb" but it doesn't work with the message: "module does not exist"

3) After a "man Xorg" it seems that there's a module called "wsfb" (which is also talked about in the link above). Still leaving machdep.allowaperture to 2, I changed the driver, but the message is still "module does not exist".

Unfortunately, avoiding this aperture is difficult.. but once it's opened by a running X server no other process has direct access.

In the future when KMS is available for more vendors/chipsets, I believe the X aperture will no longer be required.. only drm(4).

If you're using a workstation, you can have xdm(1) run at boot to securely open the aperture device.. non-root users have no directly permissions to read/write to it either, if a user had root they could do more harm using a plethora of other methods.

For non-vesa, it's sometimes possible to set machdep.allowaperture to 1 instead of 2, this does work for some drivers/chipsets but not all, please read xf86(4) to appreciate complexity of the problem.