I'm never sure if anyone is serious around here, but if you are legitimately looking for a solution Tuco, here's a shot.
First, unwind your loops. That usually can save you time. From my experiences it's faster to calc x and y each iteration in a single loop than multi looping.
I'm not C/C++ wiz, so I can't give you much more than that.
As for filling your holes, you're going to have to do some checks (and average neighbors) since you don't want edges to wrap around to the other side. You can even do diagonal neighbors if you want to get crazy. Someone else can probably give you better advice how to speed this up.
Output: