The question as it was asked is quite legitimate, although there is a more forthright version. Incidentally, the latter is even more general: Is it possible that after a series of such moves the four pegs form a square of size other than the original? Is that question simpler? In part it is. We may think of the original square as being the unit - the smallest - square of the grid. If so, we may claim outright that it is impossible that a series of moves will result in a smaller square. But, if that's the case, and taking into account that the moves are reversible, it is as much impossible to get a larger square as it is to get a smaller one.

The question, or perhaps its solution, begs for a generalization. What about a triangular or hexagonal grids? Clearly, each sustains a problem with exactly same solution. E.g, three pegs on a triangular grid form an equilateral triangle. They may jump over each other and land on the other side of and at the same distance from, the peg jumped over. Is it possible that after a series of moves they form an equilateral triangle of the size greater than the original?

Nathan Bowler offered a formalization of the foregoing argument:

Define a set S of vectors to be a lattice iff a, b, c (not necessarily distinct) in S implies a + b - c in S. That is, if any 3 corners of a parallelogram are in a lattice then so is the fourth. Since the intersection of any set of lattices is a lattice, for any set of vecotrs there is a minimal lattice containing all of them. (Thus, the minimal lattice of a set of vectors is a subset of any lattice that contains the given set of vectors.)

Lemma

Jumping one point over another in the manner of a peg move does not affect the minimal lattice.

Proof

Let A be the minimal lattice of a set of points. Then A contains all of those points and also (since it is a lattice) contains the new point obtained through the jump. Thus it contains the minimal lattice B of the new set of points. But since peg jumping is reversible, B also contains A. So they are the same lattice.

Now since the minimal lattice of a square is the standard square lattice with that square as one of the cells, it is a trivial consequence of the above lemma that it is impossible to obtain a square of a different size or orientation from the original by peg-jumping moves.

Further generalizations may start with a less regular grid or a number of pegs not necessarily matching the grid's shape. Say, three or five pegs on a "brick wall" grid? Is it possible to get a polygon similar to the original one but of a different size?

References