Lode's Computer Graphics Tutorial

Image Arithmetic

Introduction
The Code
Add
Subtract
Difference
Multiply
Average
Cross Fading
Min and Max
Amplitude
AND, OR and XOR

Introduction

Image Arithmetic is where you take two images, do calculations on their color channels, and get a new resulting image that is a combination of the two. These operations can be done with the Arithmetic tool in Paint Shop Pro, or "Calculations" in Photoshop.

This should be a pretty easy article with simple code and more looking at images.

NOTE: As mentioned at the end of the Light and Color tutorial, it is best to do these operations in linear RGB, so gamma correction before and after the operations below is reccommended for real world applications.

The Code

Here follows the code that loads in .png images and allows you to do the arithmetic and display them.

The .png images are in the /pics folder and called photo1.png, photo2.png and photo3.png. You can download them here.

These are the source images in lower quality. They're taken while hiking in the Belgian Ardennes.

Here's the code, it should be very straightforward. In the double loop that performs the image arithmetic, currently the formulas for "average" are filled in, replace this part of the code by the code given in the next sections.

int main(int argc, char *argv[])
{
  unsigned long w = 0, h = 0;
  //declare image buffers
  std::vector<ColorRGB> image1, image2, image3, result;

  //load the images into the buffers. This assumes all have the same size.
  loadImage(image1, w, h, "pics/photo1.png");
  loadImage(image2, w, h, "pics/photo2.png");
  loadImage(image3, w, h, "pics/photo3.png");
  result.resize(w * h);

  //set up the screen
  screen(w,h,0, "Image Arithmetic");

  //do the image arithmetic (here: 'average')
  for(int y = 0; y < h; y++)
  for(int x = 0; x < w; x++)
  {
    result[y * w + x].r = (image1[y * w + x].r + image2[y * w + x].r) / 2;
    result[y * w + x].g = (image1[y * w + x].g + image2[y * w + x].g) / 2;
    result[y * w + x].b = (image1[y * w + x].b + image2[y * w + x].b) / 2;
  }

  //draw the result buffer to the screen
  for(int y = 0; y < h; y++)
  for(int x = 0; x < w; x++)
  {
    pset(x, y, result[y * w + x]);
  }

  //redraw & sleep
  redraw();
  sleep();
}

Note that you could also use "result[y * w + x] = (image1[y * w + x] + image2[y * w + x]) / 2" instead of doing it for every color component separately, but this notation doesn't work for all examples.

Add

Addition is when you add the corresponding color channels of the images to each other. Each color component is a number between 0 and 255, so if the sum of the two colors becomes higher than 255, it has to be truncated to 255 again, by taking the minimum of the result and 255. Copypaste this into the arithmetic loop and run it to see the result of the sum of the two photos.

    result[y * w + x].r = min(image2[y * w + x].r + image3[y * w + x].r, 255);
    result[y * w + x].g = min(image2[y * w + x].g + image3[y * w + x].g, 255);
    result[y * w + x].b = min(image2[y * w + x].b + image3[y * w + x].b, 255);

The sky of the second photo is so bright, that adding the color components of the other image makes it so bright that the colors are truncated to 255 and thus extremely white. You can still recognize the cows in the bottom part of the image though.

Instead of doing this for all three the channels, you can also do it on only one color channel, but that would generate an excessive amount of screenshots here ;)

Subtract

Subtraction works in a similar way, but now you have to truncate negative results to 0.

    result[y * w + x].r = max(image2[y * w + x].r - image1[y * w + x].r, 0);
    result[y * w + x].g = max(image2[y * w + x].g - image1[y * w + x].g, 0);
    result[y * w + x].b = max(image2[y * w + x].b - image1[y * w + x].b, 0);

The cow picture is the one subtracted from the horses, so the cow picture's colors become negative. That explains the purplish colors. The bright sky subtracted from the flower makes the whole upper part pitch black.

The order of subtraction is important, you get this if you subtract the second from the third instead:

Difference

Difference is almost the same as subtract, only now instead of truncating negative values, you take their absolute value. So you get the difference between the colors of both images.

    result[y * w + x].r = abs(image1[y * w + x].r - image2[y * w + x].r);
    result[y * w + x].g = abs(image1[y * w + x].g - image2[y * w + x].g);
    result[y * w + x].b = abs(image1[y * w + x].b - image2[y * w + x].b);

Now the places that were black in one image are filled with the color of the other image.

Multiply

To multiply, don't multiply the color component values from 0-255 with each other, then the maximum value would be 255 * 255 = 65025, and with such a big color value, you can't do much. Instead, convert the values to floating point numbers between 0 and 1, and multiply those. The result will then also always be between 0 and 1. After multiplication, multiply it with 255 again.

    result[y * w + x].r = int(255 * (image2[y * w + x].r / 255.0 * image1[y * w + x].r / 255.0));
    result[y * w + x].g = int(255 * (image2[y * w + x].g / 255.0 * image1[y * w + x].g / 255.0));
    result[y * w + x].b = int(255 * (image2[y * w + x].b / 255.0 * image1[y * w + x].b / 255.0));

Here's the cow image multiplied with the horse image:

The bright sky of the cow image can be seen as value 1, and multiplying 1 with the horses keeps the horses, so the top part looks almost the same as the original horse image. The bottom part is the product of two darker parts, which becomes even darker.

Average

The average is gotten by adding the two images, and dividing the result through two.

    result[y * w + x].r = (image1[y * w + x].r + image2[y * w + x].r) / 2;
    result[y * w + x].g = (image1[y * w + x].g + image2[y * w + x].g) / 2;
    result[y * w + x].b = (image1[y * w + x].b + image2[y * w + x].b) / 2;

Cross Fading

Cross Fading can be achieved by using the Weighed Average, first the first image has a high weight and the second a low weight. But as time goes on, the weight of the second image is increased and the one of the first image decreased, resulting in a nice fade. Here's an example where the weight factor of the first image is 0.75, and the one of the second 0.25:

    result[y * w + x].r = int(image1[y * w + x].r * 0.75 + image2[y * w + x].r * 0.25);
    result[y * w + x].g = int(image1[y * w + x].g * 0.75 + image2[y * w + x].g * 0.25);
    result[y * w + x].b = int(image1[y * w + x].b * 0.75 + image2[y * w + x].b * 0.25);

The horse image is now more visible than the cow image.

To do an actual cross fade, a modified main function has to be used, because the result has to be redrawn every frame and the time is taken into account. For that a new while loop is added, and the weight is the value weight that goes from 0 to 1 for the first image, and from 1 to 0 for the second by using "1 - weight".

Weight itself normally goes linearly from 0 to 1, but here something special is done: it's calculated as the cosine of the time, this means that the image will constantly fade from the first to the second, back to the first, back the second, and so on... Since the cosine gives a value between -1 and +1, while we want a value between 0 and 1, add 1 to it and divide the result through 2.

int main(int argc, char *argv[])
{
  unsigned long w = 0, h = 0;
  //declare image buffers
  std::vector<ColorRGB> image1, image2, image3, result;

  //load the images into the buffers. This assumes all have the same size.
  loadImage(image1, w, h, "pics/photo1.png");
  loadImage(image2, w, h, "pics/photo2.png");
  loadImage(image3, w, h, "pics/photo3.png");
  result.resize(w * h);

  //set up the screen
  screen(w,h,0, "Image Arithmetic");

  float weight;

  while(!done())
  {
    weight = (1.0 + cos(getTicks() / 1000.0)) / 2.0;

    //do the image arithmetic
    for(int y = 0; y < h; y++)
    for(int x = 0; x < w; x++)
    {
      result[y * w + x].r = int(image1[y * w + x].r * weight + image2[y * w + x].r * (1 - weight));
      result[y * w + x].g = int(image1[y * w + x].g * weight + image2[y * w + x].g * (1 - weight));
      result[y * w + x].b = int(image1[y * w + x].b * weight + image2[y * w + x].b * (1 - weight));
    }

    //draw the result buffer to the screen
    for(int y = 0; y < h; y++)
    for(int x = 0; x < w; x++)
    {
      pset(x, y, result[y * w + x]);
    }

    //redraw
    redraw();
  }
}

Here are a few frames:

Min and Max

This involves taking only value of the pixel with the lowest or highest value, for example taking the minimum of both:

    result[y * w + x].r = min(image1[y * w + x].r, image2[y * w + x].r);
    result[y * w + x].g = min(image1[y * w + x].g, image2[y * w + x].g);
    result[y * w + x].b = min(image1[y * w + x].b, image2[y * w + x].b);

The result here is very obvious:

Because the cow image has such a bright sky, the darker horse image wins there. But in the bottom part, the horse image is brighter than the cows, so there the cows win. So this operation has filtered out exactly the sky of the cows and replaces that with the horses.

Taking the max instead does of course the opposite:

    result[y * w + x].r = max(image1[y * w + x].r, image2[y * w + x].r);
    result[y * w + x].g = max(image1[y * w + x].g, image2[y * w + x].g);
    result[y * w + x].b = max(image1[y * w + x].b, image2[y * w + x].b);

Amplitude

The amplitude is calculated by using the formula of the amplitude on the two corresponding color channels. This formula is sqrt(x² + y²). Because the result of this can be 1.41 times larger than 255, it's divided through 1.41 = sqrt(2.0) at the end. All sorts of conversions to double and back to int have to be typed as well because the standard sqrt function of the current gcc compiler gives an ambiguity error on integers instead of converting it.

    result[y * w + x].r = int(sqrt(double(image1[y * w + x].r * image1[y * w + x].r + image2[y * w + x].r * image2[y * w + x].r)) / sqrt(2.0));
    result[y * w + x].g = int(sqrt(double(image1[y * w + x].g * image1[y * w + x].g + image2[y * w + x].g * image2[y * w + x].g)) / sqrt(2.0));
    result[y * w + x].b = int(sqrt(double(image1[y * w + x].b * image1[y * w + x].b + image2[y * w + x].b * image2[y * w + x].b)) / sqrt(2.0));

AND, OR and XOR

What we're now going to do is apply the "&", "|" and "^" operators on the binary values of the color values. The results on normal photos are like, very ugly.

Here's the AND operator:

    result[y * w + x].r = image1[y * w + x].r & image2[y * w + x].r;
    result[y * w + x].g = image1[y * w + x].g & image2[y * w + x].g;
    result[y * w + x].b = image1[y * w + x].b & image2[y * w + x].b;

Or would you prefer OR?

    result[y * w + x].r = image1[y * w + x].r | image2[y * w + x].r;
    result[y * w + x].g = image1[y * w + x].g | image2[y * w + x].g;
    result[y * w + x].b = image1[y * w + x].b | image2[y * w + x].b;

The result of the XOR operator also isn't immediately beautiful:

    result[y * w + x].r = image1[y * w + x].r ^ image2[y * w + x].r;
    result[y * w + x].g = image1[y * w + x].g ^ image2[y * w + x].g;
    result[y * w + x].b = image1[y * w + x].b ^ image2[y * w + x].b;

You never know when it comes in handy, maybe for encryption of images ;)