\documentclass[../../book.tex]{subfiles}
\graphicspath{{\subfix{.}}}
\begin{document}
\chapter{Handling images with Pillow}
\label{ch:tech:pillow}

It is not uncommon to encounter images in science. You might have microscope images (optical or electron), photographs of equipment, or perhaps even photographs of products you have synthesized. No matter what you might find that you need to accomplish certain tasks, such as resizing them, cropping them, changing formats, and incorporating them into figures. 

While you can accomplish these tasks by hand, learning to do so in Python can help perform them quicker and more accurately.  For instance, say you had 20 microscope images, and you needed crop them all the same. Using Python, you could do so quickly and identically for all of them---a task that would be slow and prone to error if you did it by hand. 

Perhaps more importantly, if you want to incorporate images into plots using \texttt{Plotly}, then you will need to understand how to get images into a format that Plotly recognizes, and you will need to use a image processing library to do so. 

Python provides several libraries that allow you to accomplish all of the above tasks (and so much more).  Here we are going to use what is probably the most simple one: Pillow. Pillow is a nice choice if you want to do fairly basic tasks, as the syntax is straightforward. However, we want to note that Pillow is neither the most powerful nor the fastest imaging processing libraries.  If you find that you are wanting more, then consider looking into OpenCV. OpenCV will allow you do to image recognition, more more detailed processing, and is generally faster. But for this book, and the workflows we demonstrate in it, Pillow will be more than sufficient. 

\section{Opening and displaying images using Pillow}

Before you can do anything with an image, you need to be able to open it. And it is likely that, once you are done manipulating your image, you might want to display it, so you can see what you have done. Finally, you might want to save these changes to a file.  These are handled using three functions of the Pillow library:
\begin{itemize}
    \item \texttt{Image.open()}  This function has one positional argument and one keyword argument.  The positional argument is the address of the image you want to open.  Though there are many options, the most likely use case is that you simply provide the path to the file on your computer, either as a string or a Python Path object. The keyword is \texttt{mode} and accepts one of three values: \textbf{r}, \textbf{w}, or \textbf{a}, which stand for read, write, and append, respectively.  The default is to open in read mode, and we suggest using this mode most often. Of course, the image you are pointing towards must be openable by the Pillow library. There are a very large number of image files that exist, but most common bitmap image files will be openable by Pillow.  It is worth noting here that Pillow cannot open vector graphics images. If you need to alter such images, we are going to recommend using a dedicated program, such as Abobe Illustrator, or Inkscape. While it is possible to alter vector graphics programmatically, it is much more difficult, and is beyond the scope of this book.

    \item \texttt{Image.show()} This function accepts two keyword arguments, \texttt{title} and \texttt{command}.  The value for \texttt{title} is a string, and will be displayed as a title above the image. If no title is supplied, then no title will be shown.  The value for command is a string that specifies an executable (program) you wish to display the image in (i.e., Microsoft Paint, or Apple Preview).  If you do not specify anything for command, then the default will be used.  If you are running the script in the Spyder IDE, then the image will be shown in the Spyder image pane. 

    \item \texttt{Image.save()} This function accepts a positional argument, a keyword argument, and arbitrary keyword arguments. The positional argument is a string or Python \texttt{Path} object specifying a directory and filename for the image file to be saved.  When you are specifying the filename, you can use the extension to specify the format.  If you use "my\_image.png" as the filename, then a png file will be saved.  However, you can also use the keyword \texttt{format} to directly specify the format. We are going to suggest that you specify the format in the filename. Finally, once a format is selected, then there are a series of keywords associated with that format that can be used to fine-tune how the image is saved. We suggest accepting the defaults, but if you wish to learn how to use these, then you can read the Pillow documentation.  
\end{itemize}

\section{Manipulating image appearance using Pillow}
Once you have your image in opened and in memory, then you can manipulate it.  Pillow does include the ability to blur, sharpen, and perform other manipulations.  However, we are not going to explore these sort of manipulations.  There are two reasons for this.  First, we think the ability to alter the image is fraught with pitfalls in terms of how you accurately represent data, and we think it is best for such considerations to be carefully discussed and considered before attempting any manipulation of the image. Second, if you really want to process images in this way, then other libraries are probably a better choice. 

However, there are manipulations that are worth knowing how to accomplish. These include simple operations such as resizing, cropping, rotating, and flipping. These are accomplished using four different functions:
\begin{itemize}
    \item \texttt{Image.resize()} Accepts a positional argument that is either a single integer or a tuple with two integers. If an integer is used, then the image is scaled by that amount. If a tuple is used, then the first value is the desired width in pixels, and the second is the desired height in pixels. Note that, if you want to scale by a fractional value (i.e., a float), then you have to use a tuple. You can obtain the original height and width using the .height() and .width() methods, and then multiply them by the float of your choosing. Also, note that the final values need to be integers (since we are dealing with pixels) so you will need to round accordingly.  The function returns an image that has been appropriately re-scaled. See the example at the end of this chapter for usage. 

    \item \texttt{Image.crop()} This accepts a tuple with four integers. The integers specify the bound of a box, in units of pixels from the top left of the image.  They are specified in the order: left of box, top of box, right of box, bottom of box). The function will return the pixels that are within this box. See the example at the end of this chapter for usage. 


    \item \texttt{Image.transpose()} This function can be used to flip the image as a mirror might, either vertically or horizontally. It can also be used to rotate by 90 degree intervals. You use this by supplying one of the following values:
    \begin{itemize}
        \item \texttt{Image.FLIP\_LEFT\_RIGHT}: Flip the image horizontally (left to right).

        \item \texttt{Image.FLIP\_TOP\_BOTTOM}: Flip the image vertically (top to bottom).

        \item \texttt{Image.ROTATE\_90}: Rotate the image 90 degrees counterclockwise.

        \item \texttt{Image.ROTATE\_180}:Rotate the image 180 degrees.

        \item \texttt{Image.ROTATE\_270}: Rotate the image 270 degrees counterclockwise.
    \end{itemize}

    \item \texttt{Image.rotate()} This function accepts a positional argument, and several keyword arguments. It is used to rotate by an arbitrary number of degrees. The position argument specifies the angle, \textbf{in degrees}, by which the image is rotated. Two keywords are of interest: 
    \begin{itemize}
        \item \texttt{expand} is given boolean values.  When set to True, then the image is rotated and the full image is shown. When set to False, then the original, unrotated, edge of the image is preserved, and any pixels falling outside of this edge after rotation are cutoff. The default value is \texttt{False}, so please be aware of that.

        \item \texttt{center} accepts a tuple with two integers, and these specify the origin of rotation (the point about which rotation occurs) using units of pixels.  If no values are supplied, then the center of the image is used. 
    \end{itemize}
    This function returns the rotated image. 

\end{itemize}



\section{Manipulating image files using Pillow}
Finally, we think there are cases where you are not attempting to directly control how the image appears, but how the image is stored.  This would include changing the color encoding from something like RGB (preferred for screens) to CMYK (preferred for printing).  We note that changing these settings can (but not always will) lead to changes in image appearance.  So if you are dissatisfied with the outcome, perhaps seek the guidance of someone who is an expert in this area.  We suggest not attempting such changes, unless you are directly asked.  For instance, maybe a journal wishes you to submit images in tiff format in CMYK encoding. If you are asked to do this, then the Pillow library provides the ability and you will be working with someone who understands the format. 

All changes are handled with \texttt{Image.convert()}.  We think the only use case you might have is converting color modes, and so you can do this by simply providing a positional argument that is a string with values like 'L' for grayscale, 'RGB' for RGB color, 'CMYK' for CMYK color, and so on. 

\section{Example}
What follows is an example where we open an image in Pillow, resize, rotate, and crop it, before displaying it and saving it to the current working directory. 

\begin{lstlisting}
from PIL import Image

# Open an image file
image = Image.open("example.png")

# Resize the image to a specific size 
# in this example, the width is set to a number of pixels = 80% its original size
# the height is directly specified in terms of pixels
resized_image = image.resize((round(image.width()*0.8), 400))

# Rotate the resized image by 45 degrees counterclockwise
# since expand is set to \texttt{True}, the image is not cropped at all
rotated_image = resized_image.rotate(45, expand = True)

# Crop a region from the rotated image 
# this case is a box that:
# has its upper left corner 150 pixels from the left of the image and 100 pixels from the top
# has its lower right coner, 200 pixels from the left and 150 pixels from the top
# i.e., the box is 50 pixels wide and 150 pixels tall
cropped_image = rotated_image.crop((150, 100, 200, 250))

# Display the cropped image
cropped_image.show()

# Save the cropped image to a file
cropped_image.save("output.png")

\end{lstlisting}

\end{document}