Grid MEX

You are given an integer $N$. Construct an $N \times N$ grid (i.e. $N$ rows and $N$ columns) filled with $N^2$ integers. Let $A_{i, j}$ represent the element in $i$-th row and $j$-th column. Define $V_{i, j}$ be the set of all elements $A_{x, y}$ such that **exactly one** of $x = i$ and $y = j$ holds. In other words, $V_{i, j}$ is the set of all elements in the same row or column, excluding $(i