The Xorg Drivers page contains the instructions for building Xorg drivers that are necesary in order for Xorg Server to take the advantage of the hardware that it is running on. At least one input and one video driver is required for Xorg Server to start.
The Xorg Evdev Driver package contains Generic Linux input driver for the Xorg X server. It handles keyboard, mouse, touchpads and wacom devices, even tough for touchpad and wacom advanced handling, additional drives are required.