Resizing user-provided images is a very common yet exceedingly difficult task to get right. Over the dozen or so companies I've worked at, I've probably implemented it at least ten different times. Of all those different implementations, it's likely that exactly zero were technically correct.

My new venture, Roam, is an activity sharing app for vehicular exploration. As a product, it is centrally focused around the viewing and sharing of activities, which are bundles of various data like GPS tracks, weather info, and yes, photos and videos. Since photos and videos are so important to the experience, I have invested significant time ensuring the fidelity and speed of our media content. Let me tell you, this is one rabbit hole I did not expect to climb down, and had no idea it would be as deep as it truly is. Let's explore.

Freshman Image Resizing

We'll call this the "YOLO" approach. It's great for proof-of-concepts and for when you don't care about image quality.

from PIL import Image
from pathlib import Path

def resize_image(input_path: str, target_height: int = 1024, target_width: int = 1024):
    """
    Resizes images to fit within a target height and width ("fit" to dimensions)
    Accepts a path to an image and outputs an image file next to it with the target
    height and width appended to the filename. Uses the same format.
    """
    img = Image.open(input_path)
    input_ = Path(input_path)
    output = Path(input_.parent) / f'{input_.stem}_{target_height}_{target_width}.{input_.suffix}'
    resized = img.copy()
    resized.thumbnail((target_width, target_height), Image.Resampling.LANCZOS)
    resized.save(str(output))

This works great until you encounter any kind of diversity in your images:

  1. What about images with non-standard rotation metadata? Many phones capture images in a sensor-native orientation and store device orientation in the metadata, leaving final dimension/rotation decisions up to the presentation layer.
  2. What image format do you ultimately want to display in? If you're resizing images, chances are you are optimizing for performance, and therefore want to choose an output format with suitable compression characteristics.
  3. What image formats do you accept? Out of the box, most image libraries can only handle a limited number of image types.

Sophomore Image Resizing

Now that we've run our proof of concept and realized this requires much more depth of understanding than we originally planned for, we're going to answer the above questions and layer in logic to our image resizing code. It's no longer just a function. Let's build a pipeline.

Answering some of the above questions:

  1. Because you do not know the capabilities of your presentation layers, it's prudent to reorient the image such that the pixel layout is "top up." This way, we won't end up with upside down, inverted, or sideways images once resized.
  2. We likely want a JPEG output for the highest compatibility, but others are also suitable depending on your use case: WebP had its day and now AVIF is becoming more common.
  3. This is where the true rabbit hole begins - there are an effectively infinite number of different transcoding situations. For now, let's assume we're just accepting photos from modern mobile phones and cameras, and outputting to modern browsers and phones.
import pillow_avif # support avif images (some Android phones)
import pillow_heif # support heif images (iPhone, some Android phones)
pillow_heif.register_heif_opener()
from PIL import Image, ExifTags
from pathlib import Path

def apply_exif_orientation(image: Image.Image) -> Image.Image:
    """
    Applies the EXIF orientation data directly to the image.
    """
    exif = image.getexif()
    if not exif:
        return image
    orientation_tag = None
    for tag, name in ExifTags.TAGS.items():
        if name == "Orientation":
            orientation_tag = tag
            break

    if orientation_tag is None or orientation_tag not in exif:
        return image
    orientation = exif[orientation_tag]
    if orientation == 1:  # Normal
        return image
    elif orientation == 2:  # Mirrored horizontal
        return image.transpose(Image.Transpose.FLIP_LEFT_RIGHT)
    elif orientation == 3:  # Rotated 180
        return image.transpose(Image.Transpose.ROTATE_180)
    elif orientation == 4:  # Mirrored vertical
        return image.transpose(Image.Transpose.FLIP_TOP_BOTTOM)
    elif orientation == 5:  # Mirrored horizontal then rotated 90 CCW
        return image.transpose(Image.Transpose.FLIP_LEFT_RIGHT).transpose(
            Image.Transpose.ROTATE_90
        )
    elif orientation == 6:  # Rotated 90 CW (usually from right-held portrait)
        return image.transpose(Image.Transpose.ROTATE_270)
    elif orientation == 7:  # Mirrored horizontal then rotated 90 CW
        return image.transpose(Image.Transpose.FLIP_LEFT_RIGHT).transpose(
            Image.Transpose.ROTATE_270
        )
    elif orientation == 8:  # Rotated 90 CCW (usually from left-held portrait)
        return image.transpose(Image.Transpose.ROTATE_90)
    else:
        return image  # Unknown orientation

def resize_image(input_path: str, target_height: int = 1024, target_width: int = 1024):
    """
    Resizes images to fit within a target height and width ("fit" to dimensions)
    Accepts a path to an image and outputs an image file next to it with the target
    height and width appended to the filename. Uses the same format.
    """
    img = apply_exif_orientation(Image.open(input_path))
    input_ = Path(input_path)
    # output is now a .jpg
    output = Path(input_.parent) / f'{input_.stem}_{target_height}_{target_width}.jpg'
    resized = img.copy()
    resized.thumbnail((target_width, target_height), Image.Resampling.LANCZOS)
    resized.save(str(output), format='JPEG', quality=85)

Great! We've addressed three of our major concerns with this resizing code:

  1. Orientation is handled gracefully. Now the presentation layer will receive an image always in the "top is up" orientation.
  2. We accept a wider array of input formats - importantly AVIF and HEIF files which are common with mobile phones.
  3. We now output to a JPEG file which means that our presentation layer can simply throw the image reference (i.e. url, or file directly) into whatever container it wants to and it should display properly.

However, we've only managed to open up the door to another wave of new concerns:

  1. What about HDR? Users have become accustomed to their images "popping" off their screens when viewed in their photo gallery - some of this is pure theater by phone manufacturers but in 2025 most phones are authoring image files in HDR formats with use of HDR transfer functions (HLG, PQ).
  2. At Roam, we want to display the images on a map of the activity, so they are geographically and chronologically linked to specific events within the activity. We'll need to extract GPS information as precisely as possible.
  3. Live photos support - iPhone users have become accustomed to live photos, that is, photos which have a small snippet of video associated with them. These videos add life and motion to an otherwise still photo.

Junior Photo Resizing

Let's specifically tackle the problem of HDR and color profiles, since Location and Live photos are largely an issue of infrastructure and data structures. Location and live photos can be topics for another blog post, especially since resizing video is its own whole rabbit hole.

What we have to realize with HDR photos is that though we've had HDR cameras and formats for a while now, the infrastructure and libraries supporting them are still somewhat new. Within closed ecosystems (cough: Apple) this is a somewhat solved system - Apple has adopted (and extended) standards which suit their use case and as long as you don't stray outside their ecosystem, things are relatively easy.

However for us app developers, we must accept file inputs from a variety of manufacturers. We must then adapt our file outputs so they work on the widest range of devices within reason. When it comes to modern image formats, there are many choices to make:

  1. JPEG doesn't support HDR. There are different unrelated file formats with "JPEG" in the name which do, but they are not commonly used. Therefore, we must adopt a file format with wide HDR support. Currently, this is AVIF.
  2. There are many different HDR standards. The most common
  3. We STILL have to be able to generate non-HDR (SDR) images - this will involve converting HDR images to SDR formats, without destroying the color.

Wait, weren't we just resizing images, now we have to convert formats while maintaining proper color information? Yes, welcome to resizing images in 2025.

Further, we must also come to grips with the fact that HEIC and AVIF file formats are image containers - that is they can contain multiple images. Single images represented within those containers can be tiled - and those images can contain rich metadata which are applied to other images like depth maps.

At some point, we have to draw the line and realize that not all possible permutations of (file format, hdr format, color profile, display capabilities) can be supported and arrive at a set of features which we'll minimally support. From that, we can decide how sophisticated or unsophisticated our image processing pipeline will become.

For now, let's assume the following.

  1. We will support images from most modern phones and cameras (2016+) for inputs
  2. This implies supporting many different input formats
  3. We will support modern browsers, iOS and Android for presentation
  4. This implies multiple format support - JPEG and AVIF
import pillow_avif # support avif images (some Android phones)
import pillow_heif # support heif images (iPhone, some Android phones)
pillow_heif.register_heif_opener()
from PIL import Image, ExifTags
from pathlib import Path

# omitted for brevity
def apply_exif_orientation(img: Image.Image) -> Image.Image: ...

def resize_image(
        input_path: str, 
        output_format: str,
        target_height: int = 1024, 
        target_width: int = 1024
):
    """
    Resizes images to fit within a target height and width ("fit" to dimensions)
    Accepts a path to an image and outputs an image file next to it with the target
    height and width appended to the filename. Uses the same format.
    """
    if output_format not in {'jpg', 'avif'}:
        raise ValueError(f'Unsupported output format: {output_format}.')
    img = apply_exif_orientation(Image.open(input_path))

    # retrieve the color profile - some include one or more color profiles
    icc_profile = img.info.get('icc_profile')
    nclx_profile = img.info.get('nclx_profile') 

    # for some reason, some devices won't include an icc_profile all the time, 
    # so we have to bundle our own. fortunately most newer phones use Apple's 
    # Display P3 color profile. this is a dumb heuristic but seems to work
    exif = img.getexif()
    if exif and img.format in {'HEIF', 'AVIF'}: 
        for tag, value in exif.items():
            if ExifTags.TAGS.get(tag) == 'Make' and value in {'Apple', 'Samsung', 'Google'}:
                # assume this is stored as 'DisplayP3.icc' next to this file.
                color_profile = Path(__file__).parent / 'DisplayP3.icc'
                with open(color_profile, 'rb') as icc_file:
                    icc_profile = icc_file.read()

    input_ = Path(input_path)
    # output is now a ~*~*dynamic*~*~
    output = Path(input_.parent) / f'{input_.stem}_{target_height}_{target_width}.{output_format}'
    resized = img.copy()
    resized.thumbnail((target_width, target_height), Image.Resampling.LANCZOS)
    pil_kwargs = {
        'format': 'JPEG' if output_format == 'jpg' else 'AVIF',
    }
    # assume the jpg output will be used for scenarios where
    # speed is more important, and configure as such
    if output_format == 'jpg':
        pil_kwargs['quality'] = 75
    # assume that avif will be used for scenarios where 
    # fidelity is more important and configure as such
    if output_format == 'avif':
        pil_kwargs['quality'] = 90
        pil_kwargs['speed'] = 2
        # pass through the nclx profile if present, as avif supports them
        if nclx_profile:
            pil_kwargs['nclx_profile'] = nclx_profile
    # pass through the display profile
    if icc_profile:
        pil_kwargs['icc_profile'] = icc_profile
    resized.save(str(output), **pil_kwargs)

So what have we accomplished with this updated script?

  1. We now support high fidelity image resizing with accurate color representation across many different source and output formats.
  2. In some situations, we've retained HDR support (when the source photo is HDR with DisplayP3 and/or includes NCLX profiles).
  3. Our output photos when "converting down" from avif/heic to jpeg should still look decent and retain high quality color information.

The outcome of our Junior year is a script that has made a lot of progress toward our goals, but also adopts some sacrifices and shortcuts. Notice that the further along in our journey, less visual progress is represented by an ever deeper understanding of our subject and more time spent.

As was the case in our Sophomore year, we will collide yet again with a new set of problems:

  1. This is SLOW. Not because we're doing it in Python, but because image processing on the CPU with broadly compatible libraries like PIL (and its delegates, libheif, libavif) are just inherently not speedy. This presents problems when 1000s of photos are uploaded per minute.
  2. We've spent so much time resizing images that we've not even yet begun to discuss how our applications will display images, especially complicated by HDR. And what about videos? What about HDR videos?!

Senior Photo Resizing - Time to scale!

We've come a long way since our naive "YOLO" approach our freshman year of photo resizing, and yet, we've only just begun to create a truly production-ready photo resizing backend. What was at first a single-shot "just resize" script which seemingly could do anything has now become a sophisticated, if somewhat convoluted processing pipeline which can actually do less. It handles a narrower (but more specific) set of scenarios well, and gracefully falls back to less optimal defaults when necessary.

However, perhaps it's time to simply surrender, and do nothing. This may seem like a cop out, but trust me, the logic is sound. As a startup founder, you have to prioritize your core job which is finding product market fit and acquiring customers. This is hard to do if you are spending days or weeks pixel–peeping your resized photos. Everything we've learned here likely has almost nothing to do with our product.

So, with the realization that your core job isn't to write a photo resizing service, it may be time to look for services which do make that their core concern. I can not recommend any here, because personally I have not found any that meet our specific needs yet at Roam. I am however shopping for one now that I better understand our requirements. If you have recommendations let me know.

I Have a Photo Resizing Hangover

If you're just starting out on your photo resizing journey, let this post be your spiritual guide, if not your bedrock technical guide. If you can, decide early what your requirements will be and how sophisticated your pipeline for processing user generated content should become. Don't let the overwhelming number of decisions dissuade you from tackling the problem head-on. Also, don't be afraid of dropping out after your freshman or sophomore year. The laws of diminishing returns are definitely in effect in this domain, and it may simply not be worth your time. Keep your eyes on the bigger picture.

Join me next time for more posts about solo foundership as I continue to develop Roam.