Similar Heights

Chef is teaching his class of $N$ students at Hogwarts. He groups students with the same height together for an activity. Some of the students end up in a groups with only themselves and are saddened by this. With the help of his magic wand, Chef can **increase** the height of any student to any value he likes. Now Chef wonders, what is the minimum number of students whose height needs to be incr