TDM 40100: Project 6 — 2022

Looking sharp for fall break?

Motivation: Images are everywhere, and images are data! We will take some time to dig more into working with images as data in this next series of projects.

Context: We are about to dive straight into a series of projects that emphasize working with images (with other fun things mixed in). We will start out with a straightforward task, with testable results.

Scope: Python, images

Learning Objectives

Use numpy and skimage to process images.

Make sure to read about, and use the template found here, and the important information about projects submissions here.

Dataset(s)

The following questions will use the following dataset(s):

/anvil/projects/tdm/data/images/apple.jpg

Questions

Question 1

We are going to use scikit-image to load up our image.

from skimage import io, color
from matplotlib.pyplot import imshow
import numpy as np
import hashlib
from typing import Callable

img = io.imread("/anvil/projects/tdm/data/images/apple.jpg")
imshow(img)

If you take a look at img, you will find it is simply a numpy.ndarrary. If you were to take a look, you would find that it is currently represented by a 100x100x3 array. The first matrix represents the pixels red values, the send the pixels green values, and the third the blue values. For example, we could make our apple greener as follows.

test = img.copy()
test = test + [0,100,0]
imshow(test)

In this project, we are going to sharpen this image using a technique called unsharp masking. In order to do this, we will need to first change our image representation from RGB (red, gree, blue) to LAB. The L in LAB stands for perceptual lightness. The a and b are used to represent the 4 unique colors of human vision: red, green, blue, and yellow. If we don’t convert to this representation, our sharpening can distort the colors of our image, which is not what we want. By using LAB, we can apply our transformation to just the lightness (the L), and our colors won’t appear distorted or changed.

The following is an example of converting to LAB.

img = color.rgb2lab(img)

To convert back, you can use the following.

img = color.lab2rgb(img)*255

The reason for the 255 is because during the conversions the values are rescaled to between 0 and 1, and we will want that to be between 0 and 255 so we can export properly later on.

The first step in creating our unsharp mask, and the task for question (1), is to create a filter for our image. The filter should be represented as a function, my_filter. my_filter should accept a single argument, img, which is a numpy ndarray, similar to img in our first provided snippet of code. my_filter should return a numpy ndarray that has been processed.

def my_filter(img: np.ndarray) -> np.ndarray:
    """
    Given an ndarray representation of an image,
    apply a median blur to the image.
    """
    pass

Implement a median filter, that takes a target pixel, gets all of the immediate neighboring pixels, and sets the target pixel to the median value of the neighbors, and itself (total of 9 pixels). Make sure that the pixels you are getting the median of are the original pixels, not the already filtered pixels. To simplify things, completely ignore and copy over the border pixels so we don’t have to worry about all of those edge cases. The sharpened image will have a 1 pixel border that is equivalent to the original.

The following is an example of finding a median of multiple pixels.

img = io.imread('/anvil/projects/tdm/data/images/apple.jpg')
np.median(np.stack((img[0, 0, :], img[0, 50, :])), axis=0)

This is the most difficult question for this project, the rest should be quicker.
It may take a minute or so to run for images larger than our apple.jpg — we aren’t using any special prebuilt functions or optimizations, so it is a lot of looping.

To verify your filter is working properly, you can run the following code and make sure the hash is the same.

img = io.imread("/anvil/projects/tdm/data/images/apple.jpg")
img = color.rgb2lab(img)
filtered = my_filter(img)
filtered = color.lab2rgb(filtered)
filtered = (filtered*255).astype('uint8')
io.imsave("filtered.jpg", filtered)
with open("filtered.jpg", "rb") as f:
    my_bytes = f.read()

m = hashlib.sha256()
m.update(my_bytes)
m.hexdigest()

output

9a5d9f62d52bcb96ea68a86dc1e3a6ae3a9715ff86476c4ccec3b11e4e7dde8e

To see the blur:

imshow(filtered)

To see the blurred image normal scaled:

from IPython import display
display.Image("filtered.jpg")

Items to submit

Code used to solve this problem.
Output from running the code.

Question 2

The next step in the process is to create a mask. To create the mask, write a function called create_mask that accepts the image, a filter function (my_filter from question (1)), and strength. create_mask should return the image (as an ndarray).

def create_mask(img: np.ndarray, filt: Callable, strength: float = 0.8) -> np.ndarray:
    """
    Given the original image, a filter function,
    and a strength value. Return a mask.
    """
    pass

The mask is simple. Take the given image, apply the filter to the image, and subtract the resulting image from the original. Take that result, and multiple by strength. strength is a value typically between .2 and 2 that effects how strongly to sharpen the image.

Test to make sure your result is correct by running the following.

img = io.imread("/anvil/projects/tdm/data/images/apple.jpg")
img = color.rgb2lab(img)
mask = create_mask(img, my_filter, 2)
mask = color.lab2rgb(mask)
mask = (mask*255).astype('uint8')
io.imsave("mask.jpg", mask)
with open("mask.jpg", "rb") as f:
    my_bytes = f.read()

m = hashlib.sha256()
m.update(my_bytes)
m.hexdigest()

output

e6cd9badbcb779615834e734d65730e42ded4db2030e0377d5c85ea6399d191a

Take a look at the mask itself! This will help you understand what the mask actually is.

imshow(mask)

To see the properly scaled mask:

from IPython import display
display.Image("mask.jpg")

Items to submit

Code used to solve this problem.
Output from running the code.

Question 3

The final step is to apply your mask to the original image! Write a function called unsharp that accepts an image (as an ndarry, like usual), and a strength and applies the algorithm!

def unsharp(img: np.ndarray, strength: float = 0.8) -> np.ndarray:
    """
    Given the original image, and a strength value,
    return the sharpened image in numeric format.
    """

    def _create_mask(img: np.ndarray, filt: Callable, strength: float = 0.8) -> np.ndarray:
        """
        Given the original image, a filter function,
        and a strength value. Return a mask.
        """
        return (img - filt(img))*strength


    def _filter(img: np.ndarray) -> np.ndarray:
        """
        Given an ndarray representation of an image,
        apply a median blur to the image.
        """

How do you apply the full algorithm?

Create the mask using the create_mask function.
Add the result to the numeric form of the original image.

That is pretty straightforward! Of course, you’ll need to convert back to RGB before exporting, like normal, but it really isn’t that bad!

You can verify things are working as follows.

img = io.imread("/anvil/projects/tdm/data/images/apple.jpg")
sharpened = color.rgb2lab(img)
sharpened = unsharp(sharpened, 2)
sharpened = color.lab2rgb(sharpened)
sharpened = (sharpened*255).astype('uint8')
io.imsave("sharpened.jpg", sharpened)
with open("sharpened.jpg", "rb") as f:
    my_bytes = f.read()

m = hashlib.sha256()
m.update(my_bytes)
m.hexdigest()

output

e6cd9badbcb779615834e734d65730e42ded4db2030e0377d5c85ea6399d191a

You can test to see what the sharpened image looks like as follows.

imshow(sharpened)

Or the normally scaled image:

from IPython import display
display.Image("sharpened.jpg")

There are quite a few ways you could change this algorithm to get better or slightly different results.

There is quite a bit of magic that happens during the color.lab2rgb conversion.

Items to submit

Code used to solve this problem.
Output from running the code.

Question 4

Find another image (it could be anything) and use your function to sharpen it. Mess with the strength parameter to see how it effects things. Show at least 1 before and after image.

Items to submit

Code used to solve this problem.
Output from running the code.

Question 5 (optional)

Instead of using the median blur effect, you could use a different filter, like a Gaussian blur. If you Google a bit, you will find that there are premade (and probably much faster) functions to perform a Gaussian blur. Use the Gaussian blur in place of the median blur, and perform the unsharp mask. Are the results better or worse in your opinion?

Items to submit

Code used to solve this problem.
Output from running the code.

Please make sure to double check that your submission is complete, and contains all of your code and output before submitting. If you are on a spotty internet connection, it is recommended to download your submission after submitting it to make sure what you think you submitted, was what you actually submitted.

In addition, please review our submission guidelines before submitting your project.