Partition into Permutations

You are given an array of integers $[A_1, A_2, \ldots, A_N]$. Let's call adding an element to this array at any position (including the beginning and the end) or removing an arbitrary element from it a *modification*. It is not allowed to remove an element from the array if it is empty. Find the minimum number of modifications which must be performed so that the resulting array can be partitioned