Take the 2-minute tour ×

I've got a problem with image processing, but I don't know how to solve it.

I have 2 pictures: - Pic 1: http://goo.gl/BBxVl - Pic 2: http://goo.gl/X0VFW

The pic 2 actually express the pic 1 but it was covered by the object. I am writing a program using matlab code to define that: If the picture is a the fulfilled (perfect) one, I will "imshow" it. But if there are any errors with the picture, the program will show the notice board for the user.

Until this time, I still can not solve it because I don't know where to begin; I also don't have any definition about the shape and the color of the object that cover the picture.

share|improve this question
add comment

3 Answers

So you don't know which one of them is the correct one, and you want to choose the correct one? Or you have a ground truth image that you can compare the input with? Because if you know how the correct image looks like, you can just do a simple subtraction between the images and find the erroneous one. So I don't think that's the question.

So, in the case you have two images and you have to determine which one is correct, do you know the type of the error? It is always a "patch" added on the image? Is it always rectangle or it can also be blended with the background image?

One idea would be to cut the image in parts and do histogram analysis for every one of them, but it will only work if the original image is quite constant and the patched object differs a lot from the background.

Maybe you can do edge detection on the image and then try to detect straight lines (Hough line detection can do that) so that you can search for a rectangular shape. (If you have the lines as a result then the algorithm for finding a rectangle is independent of the shape and size of the rectangle. You only check the line direction)

If you can describe with more details what are the restrictions of your error objects, maybe we can think of a better idea.

share|improve this answer
When I got the picture, I don't know which is the right one, and I also don't have any input to compare. I think you was right cos' we have many kind of picture. If we don't have the standard, we can not find any things. I would like to ask you one more question: If the picture is covered by the object with rectangle shape, how can we detect it? I can detect the straight line with Hough Transform. And about the object with rectangle shape, if I don't know the its dimension, can I do it? Thanks for answering me. –  Soul Nov 29 '12 at 15:13
You can have a rough estimation for example if you have the line equation after Hough, you can check if you have two pairs of almost parallel lines. You can check their direction and if it's lower than a threshold they are parallel. You can also check their position, if (after finding the two parallel pairs) the positions of them are away enough to form a rectangle. –  George Aprilis Nov 29 '12 at 15:25
Actually the answer from bjoernz is very good, if you have errors that are "patches" on the original image, you can check the edges of the area that is different and compare (with a similarity estimator, gradient, mean value etc) to see which of the two "insides" matches the "outside" better. And it is more shape independent than the other one. –  George Aprilis Nov 29 '12 at 15:30
Using Hough in this case is a very bad idea. How are you going to distinguish between rectangles in the input image from the supposed added rectangles patches (in case they are present) ? Thresholding is not an option, there might be actual rectangles giving a higher response by Hough than the one given by the patch, and vice versa. –  mmgp Dec 2 '12 at 1:05
I meant that Hough will run on the difference image (and maybe be compared with the rectangles found in both images to check where it comes from. But it's a clumsy idea of course. –  George Aprilis Dec 3 '12 at 10:02
add comment

If the images you have posted are good examples of the problem, I would suggest the following algorithm:

  1. substract both images from each other
  2. determine the area of difference, this will give you the shape of the object that covers the "perfect" image
  3. look along the edges of the distortion which of the two images is a better fit.
share|improve this answer
add comment

My idea also starts with the absolute difference between both images. The issue here is that you might get a lot of regions that weren't modified at all, due to compression and the inner working of some file formats (like jpg). For instance, here are two sample images and the binary difference between them to highlight every modified point, although I only manually modified the visible rectangles. In the non-binary difference you hardly notice all these points, but they are there. A threshold solves the problem here, I played with the value 20:


Now, to determine which of the images is the "good" one, I used an inpainting algorithm (you can find one such implementation at http://www.cc.gatech.edu/~sooraj/inpainting/). The reasoning for this is that the final in-painted image is more likely to resemble the image before inpainting if it does not contain the artificial patches. So, the "bad" image is the one with greater difference after the inpainting process. For this, you calculate again the absolute difference, now between the inpainted image and the original one for both cases. Then use some measure in the regions with absolute difference > 0. A dummy sum of the intensities in grayscale, in this example, gives 424454 for the image without the rectangles and 758366 for the other one.

Here is the masked image for in-painting, the "good" image inpainted, the "bad" image inpainted, and the respective absolute differences in grayscale:


share|improve this answer
add comment

Your Answer


