I have to agree with tsallgood. A draw bar or bolt through the column is absolutely required to hold the chuck and bit in place. I also speak from experience with trying to use a drill press for a mill. Such use is very dangerous. Perhaps using a drill press stand for strapping a reversing drill motor in would be better than an actual drill press. A reversing drill chuck is threaded to the shaft with a reverse thread screw internally to hold it in place. Milling machines always have a draw bar to retain the collets or chucks in place. A drill press with a taper alone will not be enough. Someone may know of a modification for taper shanks to use some sort of retaining pin or bolt in the quill. Also, some quill bearings are better than others for radial loads.

Googles will work against a small chip of wood or metal but not against a chuck and/or bit planted in the middle of your forehead or that of a child across the room. The bit, while turning, is not only applying a side load but is actively trying to pull the bit downward out of the press due to the angles at the cutting edges. When the bit and holder come loose, metal will bend and break, wood will splinter, and the rotating parts will be thrown with incredible force and velocity in some unknowable direction. Using a X-Y table for positioning holes in a drill press is a good idea. For milling, use a router with a router table if required, or a milling machine.

First thing to remember with old equipment not commonly in use anymore, is to remove the original part for a visual inspection and measurement. Make a sketch with dimensions and take pictures. This helps avoid ordering incorrect parts from old obsolete and incomplete parts listings. In the original parts listings there were probably multiple pages of part numbers. Over the years the parts books frequently condense those listings to one or two lines. Information has been lost. Examine the old part and the mating parts to determine how and why they function, -and why they failed. This will help with installation and adjustments.