Your issue is not really AB fodder
and is certainly NOT common sense
As soon as you mention self-employed drivers
I am reminded of
Ready mixed concrete v MOPNI ( min of pensions and nat insurance)
here
http://www.hmrc.gov.uk/manuals/esmmanual/esm7030.htm
It is obvious you have been trying lawfully of course to avoid VAT charges and not surprisingly the VAT man is now after you
good luck
I think but I do not have all the facts that you will find you have been badly advised int he past