Arthur's dev blog

Adding auto-zoom feature to Android-Image-Cropper

In this post I will describe the work I did to add auto-zoom functionality to Android-Image-Cropper library.
The goal is a smooth zoom-in/out experience affected by the size of the cropping window. When the user adjusts the crop window to cover less than 50% of the image current total visible sub-area, auto-zoom-in is triggered to zoom-in and center on the relevant cropping image zoomed sub-area so the crop window will cover 64% of it. Similarly, when the crop rectangle is above 65% auto-zoom-out is triggered so zoomed sub-area will be 51% covered.

Result:

Step 1 – Use matrix scale type

Originally the library used either CENTER_INSIDE or FIT_CENTERImageView.ScaleType values to fit the cropping image in the image view, internally image view calculated the required matrix transformation for the image and scale type.
The core of auto-zoom functionality is the manual manipulation of the matrix transformation that in-effect is the zooming and moving of the cropping image area, therefore the first step is to use matrix scale type and set the initial transformation to fit the cropping image in the view (figure 1). Two initial matrix transformations are required: scale and translate (figure 2) .Scale: Fit in the image inside the image view widget by scaling it relative by the size of the image to the size of the image view. Note, to preserve the aspect ratio of the image the scale value is the minimum between the width and height scale.Translate: Center the image inside the image view widget by translating it by half of the empty margin left for either width or height, due to scaling to fit at least one of the dimensions matches exactly.

mImageView.setScaleType(ImageView.ScaleType.MATRIX);

Figure 1: Image View widget must be set to MATRIX scale type as we manipulate it manually.

Step 2 – Handle rotation

In the original solution rotation was done by modifying the image displayed in the image view, creating a new bitmap for each rotation. A wasteful process that can be corrected now using matrix transformation:

Figure 3: Image matrix rotation transformation pivoted on the center of the image.

Adding rotation to the transformations process required changing the sequence and pivot point for all applied transformations as not to cause drift of the image, and update the image rectangle to know its size and location after transformations (figure 4) .

Move the image to the center of the image view so the center of the image aligns exactly to the center of the image view, the rotation and scale transformation are applied on the center of the image so the cropped image will always remain in the center.

Rotation transformation is done before scaling because the width/height of the image may change by rotation so it will affect the amount of scaling required.

Update the final rectangle of the image so we can overlay the cropping UI right on top of it, matching the transformations.

Step 3 – Adding auto-zoom-in/out

Zoom is essentially scaling beyond the size of the image view resulting in image rectangle that is larger than the image view, and translating so the relevant part of the image is visible. Two parts are required; trigger check to change the current zoom level and update of the image transformation matrix to zoom on specific sub-area of the image.

Zoom trigger check is done when cropping window has been changed (figure 5 ), if its size is less than 50% of the current visible sub-area view size, for both dimensions, the zoom level is increased so the (unchanged) cropping rectangle size is exactly 64% of the new visible sub-area, for the larger dimension. If the size is more than 65% of the current visible sub-area view size, for any of the dimensions, the zoom level is decreased so the cropping rectangle size is exactly 51% of the new visible sub-area, for the smaller dimension.

Two transformation are added to update the image matrix (figure 6 ); scaling by the updated zoom level and translation of the image so the zoomed sub-area is in the center of the image view, within image bounds. Additionally cropping window is updated so it covers the same area that it covered before the zoom-in/out.

This way every time the cropping window is too small we zoom into the sub-area and if the cropping window is too large we zoom out of the zoomed-in sub-area, keeping the cropping window in the center. Thus achieving the required auto-zoom experience.

Figure 6: Zoom scale and translation transformations to focus on zoomed sub-area of the cropping image.

Sliding crop area by crop window move

In addition to handling zoom-in/out trigger that occurs when crop window change is complete (user lifting his finger from the touch area), we need to handle scenario where during the cropping window manipulation the edges of the cropping window exceed the edges of the zoomed sub-area in the image view, in this case we need to adjust the zoom offset so the cropping window will be fully in view. There is no need to change the zoom level as we prevent the cropping window size to exceed the visible sub-area during user manipulation.

Figure 8: update zoom offset (x, y) calculations to only move the crop window into view when centering is not required.

Step 4 – Smoothing transitions

To smooth the view transition during zoom-in/out we need to gradually change the image matrix from the starting state, before the zoom-in/out was triggered, to the end state of the new zoomed sub-area view.
First we save the starting state of the image matrix during crop window change event (figure 9 ).
Second, at the final stage of calculating the resulting image matrix, instead of setting it directly on the image view, we use an animation instance that contains the start and end image matrix states (figure 10 ).
Finally, the animation can execute smooth transformation of image matrix (figure 11). The intermediate matrix, represented as 9 cells vector, can be calculated using a simple linear calculation: f(i,t) = start[i] + (end[i] – start[i])*t where i:[0-8] and t:[0-1].

Step 5 – Cropping

Finally when it's time to crop the image, the cropping window needs to be translated from the image view coordinates, affected by all the image matrix transformation, back to source image coordinates. Fortunately, math is beautiful, having the image matrix used to transform the source image to the sub-area used in image view we can invert it and get the matrix that transforms the cropping window coordinates back to source image (figure 12 ). We are using 4 points coordinates and not a rectangle due to non-straight angle rotation, so the cropping rectangle is calculated from the 4 points that either is the required area or contains it within if non-straight rotation was used (figure 13).

Crop rectangle from source image

Due to Android API restrictions we need to handle differently if the source image is available as bitmap object or loaded (with sampling) from URI. If we crop a bitmap object directly we can crop and rotate it using a single operation (figure 14), otherwise , we first load (decode) the cropped area and then rotate the loaded bitmap (figure 15). We could have just fully loaded the source image to bitmap and use the same approach for both methods, but that would have required more bitmap memory allocation and the possibly reaching device limits, loading only the required crop area reduces the allocated bitmap memory thus improving performance and although reaching limits is still a possibility it should be less common.

Figure 15: Load image cropped area and then rotate in separate operation, then execute second cropping if required for non-straight angle rotation.

Handle non-straight angle rotation

In case the image was rotated by a non-straight angle (i.e. not 0, 90, 180 or 270) the cropping rectangle that was used in cropping the source image (figure 14/15) is larger than the actual required rectangle (figure 16) because it is impossible to crop non-rectangular area using Android API. So the result of 'cropBitmap' methods, in this case, is the red rectangle in figure 16 and we need to add conditional second cropping on the result bitmap before it is returned (figure 17). We could have first rotated the fully loaded source image and then cropped the required rectangle, but this would, again, require more bitmap memory allocation, first cropping is more complicated but may have significant performance improvement.

Final words

I have omitted some boilerplate code from the snippets for clarity. Some possible configurations, initializations and state change scenarios requires a little bit more edge case handling.

Android bitmap API limitation is a real problem as it forces more than required bitmap memory allocation and contain bugs (BitmapRegionDecoder) that may require the loading of source image in full. So I may be checking an option of going native for bitmap loading and cropping, may be interesting.

Handling of non-straight angle rotation is not fully complete as the cropping window can exceed the bitmap boundaries, it’s a complicated scenario to handle, especially for circular cropping shape, so I left it for next version.