HyVolution – Super-Resolution Imaging with a Confocal Microscope

Since the invention of the microscope, there has been continual discussion about the possibility of showing more detailed features of specimens as compared to just magnifying them [1].

In this article, we describe the HyVolution concept and how the combination of confocal multiparameter fluorescence imaging at the confocal super-resolution regime with psf-based real deconvolution allows high-speed multicolor imaging with a resolution down to 140 nm.

The limit of differentiation … will not substantially surpass half the wavelength of blue light in case of most oblique illumination. The "most oblique illumination" would be the use of a condenser with an NA that at least matches the NA of the objective lens. Keep in mind that around 1850 AD, a "microscope" typically employed transmitted light brightfield in the visible range.

Of course, this statement began a race to invent instruments that could do better. Ultra-violet microscopes and novel immersion systems just utilized the formula by varying the parameters λ and NA without transgressing the postulated borders. An extreme case of such variation is the electron microscope, where λ is controlled by the voltage that accelerates the electrons.

To actually cross Abbe’s limit, one could take two different paths: either develop and design instruments with optics that inherently generate an improved resolution, or apply mathematics and computation to the raw images to increase the separability of features hidden in the recorded data. Any system that is capable of resolving better than Abbe’s limit is tagged "super-resolution" [2]. It must be clear, though, that these solutions do not prove Abbe wrong. His considerations and conclusions on visibility of line-shaped objects in classical microscopes are correct. The same is true for the analysis of the separability of point-like emitters by several authors [3, 4, 5], which led to a series of criteria close to Abbe’s limit.

Optical super-resolution

One optical device that has the capability to resolve better than dAbbe is the confocal microscope [6]. In most cases, the confocal microscope is regarded as the gold standard for performing optical sectioning. This is correct, and seems to have a continuing role as the indispensable workhorse for fluorescence imaging in modern biological and medical research. In addition, it offers a series of applications for material science, semiconductor research, quality management and other fields.

Besides the optical sectioning capability, which addresses contrast in axial direction, a confocal device also can improve the lateral resolution. Like the thickness of the optical sections, the lateral resolution depends on the size of the pinhole [7]. If the pinhole size is larger than the central feature of the diffraction pattern, called the "Airy disk”, then the resolution is more or less constantly equal to the diffraction-limited resolution in an ordinary microscope. Upon closing the pinhole to diameters below that "Airy unit", the lateral resolution increases – corresponding to a smaller FWHM of detectable features (see Fig. 1). The limit is reached for the theoretical case of a completely closed pinhole, where the theory predicts an improvement of some 30%.

Fig. 1: Optical performance of a confocal microscope as a function of the diameter of the pinhole (x-axis) in multiples of the diameter of the Airy disk. Green curve: Integrated intensity in the focal plane relative to an infinite pinhole (y-axis on the right). About 90% of the focal intensity is collected if the pinhole diameter is 1 AU (grey vertical). Dotted black line: lateral resolution (in optical units, y-axis on the left). The lateral resolution is increased substantially when closing the pinhole to 0.6 AU (red vertical). Solid black line: optical sectioning performance as full width half maximum of the intensity profile (y-axis on the left) The optical sectioning is significantly deteriorated when opening the pinhole, e.g. to 1.25 AU (blue vertical). Black curves adapted from Wilson 1990.

At a moderately closed pinhole of e.g. 0.6 AU, a good portion of the potential resolution improvement is exploited without sacrificing too much of the precious emission. The focal intensity is shown by the green line in Figure 1. It is calculated by the integration of the intensity distribution that is described by the diffraction integral. At pinhole diameters around 1 AU, the total intensity does not change much due to the fact that 1 AU is just the position of the first zero of the diffraction pattern. You can also read in Figure 1 that ca 90% of the focal light is collected at 1 AU already. Further widening of the pinhole has little effect on the efficiency, which is the reason for recommending 1 AU as a standard. At a diameter of 1.25, for example, the increase is merely 2%. If the pinhole is opened above 1 AU, the measured intensity increase is due to collection of unwanted extra-focal light, the portion which was meant to be removed by investing in a confocal microscope!

As mentioned above, the standard pinhole diameter setting in true confocal imaging is 1 AU. Low-performing systems might need to open the pinhole in order to receive sufficient photons to generate a decent image. To ensure maximum performance, the confocal microscope must have maximum transmission. This is aimed for in the Leica confocal series by implementing a spectrometer-type detector unit (SP-detector) and an acousto-optical crystal-based primary beam splitter (AOBS) [8]. In addition, the implementation of hybrid sensors (HyD) [9] with the highest photon collection efficiency and lowest dark signal ensures brilliant imaging, even at pinhole sizes below 1 AU.

Computational super-resolution

"Seeing is believing [10]" seems to be an obvious imperative for impartial observation in scientific experiments. However, the opposite is true. When we see something, objectivity ends behind the eye-lens. The retina, which is part of the complex and unpredictable computer called "brain", already performs spatial and temporal computation of the incoming optical signal. The picture formed in our conscious mind is something that we have to believe in. If we want to know, we need to measure.

For this reason, old engravings in antique books on microscopy look very nice and artistic, but are severely modified by the perception of the microscopist and the ability of the artist. More realistic pictures were obtained when photography was invented. But here too, many techniques have been developed in classical wet photographic art to improve visibility of some details and reduce visibility of others.

With electronic imaging by video tubes, it was possible for the first time to directly modify the recorded picture. Video microscopy visualized details which could not be seen before, albeit not by increasing the resolution, but the contrast. If objects are very small and not resolved, they create only a minute change in an otherwise bright background. By increasing the amplification in the video system and setting the baseline to an appropriate value, these minute disturbances can be made visible. Nevertheless, objects below the diffraction limit will show up as diffraction patterns – their size and shape determined by diffraction.

Does this mean there is no chance to improve the resolution of image data after they have been recorded? Simple contrast and brightness manipulation will not do so. But there are more elaborate operations that can. The idea behind this "image restoration" [11] is straightforward. The light emanating from the sample is somehow "convolved" by the optical elements in the microscope. The convolution can be described by a mathematical operation. Hence, if we apply the inverse of that operation to the recorded image data, we can hope to receive a result that is closer to the original intensity distribution in the object plane.

As a matter of fact, by following this approach it is possible to improve both axial and lateral resolution by roughly a factor of two [12]. And deconvolution does not only improve resolution, but also reduces noise and thus increases the signal-to-noise ratio (SNR). As calculations always take some time, it is necessary to implement high-speed GPU arrays driven with appropriate software, e.g. CUDA, for top-end mathematical processing even of large datasets.

The resulting image is not even an intensity picture, but a spatial map of emitters. Techniques that offer limited super-resolution, for example structured illumination or optical pixel rearrangement (image scanning), use sets of images which by computation result in an improved image. Thus, a combination of optical methods with mathematical applications [14] is not the exception, but the rule.

The Leica TCS SP8 with HyVolution combines all the options of a highly efficient confocal microscope and professional well-proven deconvolution software with high performance computation hardware. Resolution benefits for such a combination have been shown for scattering objects [14] and fluorescent objects [15]. The combination of confocal multiparameter fluorescence imaging at the confocal super-resolution regime with psf-based real deconvolution allows high-speed multicolor imaging with a resolution down to 140 nm (Fig. 2).