OK, well you've now given us more information about the relative height differences, and I'm afraid it doesn't help your case.
Under common law, in a situation where the legal boundary between two parcels of land also coincides with a height difference in the land, it is the responsibility of the owner of the higher piece of land to retain his soil and hence (in the absence of any statement in historic deeds / clarification on the Land Registry title) it is his job to build or maintain a wall to retain his soil and stop it falling into your plot. His it is normally his retaining wall on the common boundary
However that does NOT entitle him to build piers beyond outer line of the existing ancient wall into your land, to provide structural support. To keep his soil in place, he has to cut back into his land if renewing a wall that is being pushed into your land by the weight of soil behind.
If he has then extended the top of this wall vertically, he is allowed to do that subject to planning restrictions, which, yes, does mean up to two metres above the ground level on his side.
I'm afraid I can't now picture the layout arrangement that you have. You previously said your garage (side wall I assume) sits on the boundary and the new ugly wall extends in line from this side wall. If that is true, and his plot level is higher than yours, then his soil presumably also presses against the side wall of the garage, on his side? How much difference in level are we talking?