What I should have said was: “So if someone has a picture of herself and her ex-spouse standing in front of their old house in Wilford, Idaho…or better yet in the kitchen of that aforesaid house, Google is telling me that I need millions of pictures, because only after amassing that many pictures will I have a chance of finding those one or two pictures taken at exactly the correct angle, with the correct exposure, and at the correct time of day (afternoon sunlight vs. nighttime tungsten) so that I can fill in the correct background. What I do not believe, though, is that even after collecting billions of pictures, you will have exactly the right one to fill in the background. So instead, I think it would be better to have an algorithm that let you explicitly specify and upload 2-3 slightly-wrong pictures of the scene and use fancy algorithms to distort, skew, alter white balance, and otherwise reconstruct occluded objects, so as to produce the one ‘correct’ background.”

So I was wrong to have stated it the first way. Apologies.

The point that I was trying to make, however, remains the same: Unless you are talking about generic or popular backgrounds, such as the ocean or the Eiffel tower, it doesn’t matter how many millions or billions of photographs you have assembled. If you don’t have a picture of Fred and Ina’s kitchen taken from the right angle at the right time of day, you never will.

I made a similar comment here. In case the link goes down, here is a reproduction of that comment:

——————————–

Here’s the problem I have with the method: It works really well if your task is to fill in image spaces with generic, through semantically consistent, backgrounds.

What if, however, the background should be filled with something that you know really is supposed to be there. For example, suppose you have a picture of Fred and Ina, taken in their kitchen in Wilford, Idaho. You want to remove Fred from the picture. But behind Fred is that vase that they picked up on their trip to France in the late 1970s. And just next to the face is the old clock handed down through the generations from Ina’s family, from ancestors who used to be clockmakers in Switzerland. And below the clock is the old WWI photo of Ina’s Great Uncle.

Now, you want to remove Fred from the picture, but not replace him with any old generic kitchen kitsch shelving. You want to replace it with what is really there in the picture.

How does big data help? I don’t really think it does.

With smart algorithms, on the other hand, you could get the user to provide the algorithm with just 2-3 pictures of the same scene, taken from different angles, and then have the algorithm reconstruct what really is behind Fred.

So that’s my only point. Simple-method-big-data works great if you simply want a generic, albeit semantically meaningful, fill-in. But if you’re really trying to replace what is behind the removed element, I don’t see how big data helps you. As I often argue, it comes down to the task you are trying to solve.

——————————–

Does this clarify things a bit?