Texture filtering clarification

Hello All,

I needed some clarification regarding where exactly the filtering happens in OpenGL.
So, as per what i understand, we interpolate tex-coords from vertex to fragment shader. These vertices are used to sample a color(texel) from the texture.
When we sample using texture2D() in the fragment shader, this is where the filtering technique is applied.

So,
1) GL_NEAREST: texture2D() returns the color at the appropriate tex-coord
2) GL_LINEAR : texture2D() averages color(texel) values from neighboring texels and returns the value.
And so on, with other filtering techniques.

Is this understanding correct? Or is there some other way the driver does the sampling of colors from the texture and returns the value?
Can anyone please clarify this for me?

So,
1) GL_NEAREST: texture2D() returns the color at the appropriate tex-coord
2) GL_LINEAR : texture2D() averages color(texel) values from neighboring texels and returns the value.
And so on, with other filtering techniques.

Is this understanding correct? Or is there some other way the driver does the sampling of colors from the texture and returns the value?

You have the jist of it. There are a few additional subtleties though that you might want to be aware of.

First, with the MIPmap filtering modes there are some details on how "neighboring" is determined, and even with LINEAR_MIPMAP_LINEAR neighboring texels may not be sampled and filtered in in some cases due to perf cheats in the driver.

Second, with anisotropic texture filtering enabled, the definition of "neighboring" is further stretched (potentially into a very long and thin region of the texture), as the driver may take many "small" samples from the texture to try to sample/filter an avg result for the (approximate) region of it subtended by the fragment projected onto it.

Finally, all of this "neighboring texels" determination including MIPmap level selection is based on derivatives of the texcoords you pass into the texture2D function which are computed behind-the-scenes by the hardware. This can be really important to know about when you start conditionally doing things in your shader related to texturing.

Or is there some other way the driver does the sampling of colors from the texture and returns the value?

There are a number of other texture access functions, including one which bypasses the texture filtering altogether, but I'm assuming that's not what you're talking about.

@Dark Photon:>> you pass into the texture2D function which are computed by-the-scenes by the hardware

Could you please explain what you meant by "by-the-scenes"? I don't understand the phrase in the context.
>> There are a number of other texture access functions, including one which bypasses the texture filtering altogether, but I'm assuming that's not what you're talking about.

Yes, that was not what i was referring to. But it would be great if you could please mention these techniques. It would make an interesting read.

@Dark Photon:>> you pass into the texture2D function which are computed by-the-scenes by the hardware

Could you please explain what you meant by "by-the-scenes"? I don't understand the phrase in the context.

Doh! That should have been "behind-the-scenes" -- sorry about that.

>> There are a number of other texture access functions, including one which bypasses the texture filtering altogether, but I'm assuming that's not what you're talking about.

Yes, that was not what i was referring to. But it would be great if you could please mention these techniques. It would make an interesting read.

Just pull up the GLSL spec and look in the back for details. There are functions that will let you bypass the configured texture filtering mode and sample a specific texel directly (texelFetch), specify specific LOD or gradiant parameters to override the inputs to the MIPmap selection process that I mentioned occurs behind-the-scenes with texture2D (Lod and Grad variants), lookup into the texture with an offset applied to the texcoords so you can look at neighboring areas yourself (Offset variants), etc.

>> There are a number of other texture access functions, including one which bypasses the texture filtering altogether, but I'm assuming that's not what you're talking about.

Yes, that was not what i was referring to. But it would be great if you could please mention these techniques. It would make an interesting read.

If you look at the GLSL reference manual under "T", most of the functions whose names begin with "tex" perform texture lookups. The large number of functions allows you to control how which parts of the process you want to do yourself and which parts you want to let the implementation do for you.

E.g. the functions with "Proj" in their name do the projection from homogeneous coordinates for you, dividing by the last component in the coordinate vector, while those without "Proj" will require this to have been done already if you're using homogeneous texture coordinates.

Conversely, the functions with "Lod" in their name require that you pass in the level-of-detail (LoD) value explicitly, while those with "Grad" in their name require that you pass in values for the derivatives, from which the LoD will be computed. Those with neither "Lod" nor "Grad" will calculate the LoD by applying the fwidth() pseudo-function (or something similar) to the expression for the texture coordinates.

The textureGather function (and offset versions) return a single component from the 4 texels used for bilinear interpolation, allowing you to perform your own interpolation (this is required for non-linear encodings and/or gamma correction).

texelFetch() and texelFetchOffset() bypass all of the filtering operations and require that you specify integer texel coordinates and an integer mipmap level directly. These two are the lowest-level functions; all of the ones which begin with "texture" can be implemented in terms of these.