A number of people above have struggled to understand the logic of your employer's actions. I am very sure your employer is incorrect in its interpretation.
Not sure if I count here as an expert, but I probably have more direct experience working with HR then many people here.
We seem to have established that you work days of equal length. Had you not, the ONLY correct way to calculate holiday is in hours. That does not need to apply here.
People on zero hours contracts have no guarantee of work on a particular day. Nor indeed does the employer have a right to demand work on a specified day. That does not apply here.
You seem to have a contract that specifies a number of days work per year or per month (which is it), but with local agreement as to when you will work these days. Can we confirm this is right, and how far in advance this is fixed.
If one works a five day week, that is 260 days per year. The minimum holiday entitlement is 28 days, so employee works 232 days. I suggest you work out how many working days you work. Or tell us here and we can work it out.
That is the physical number of days you need to attend, and the employer cannot have it both ways by turning around late in the day and say, I want you to work the after next, you say, but I wanted holiday, so the employer says, that counts as your holiday then, pal.
He could refuse you that as time off (though it could be pretty unreasonable) BUT YOUR HOLIDAY IS NOT LOST. It is there for another time. Back to my original assertion that you should work out how many days per year you are contracted to work.
The way payroll are trying to calculate this currently is ridiculous and wrong.
At the end of the holiday year, if you have not had all your holiday shifts, you and your employer could agree to transfer some shifts into the next year, or to pay you for some unused ones, but there are EU WTD regulations compelling employers to ensure employees get at least 20 shifts/days off (for a full timer). Pro rata for a part timer. This very sensible piece of legislation is to ensure employees don't feel they have to give up holiday entitlement in exchange for money.
Lastly, one person suggested contacting HMRC for advice. HMRC deal with many things related to tax and NI regulations, but holiday entitlement is not something they will deal with.