If it is Badgers, they are protected so you would have to get professional advice, it should be fine to leave it alone, we have 3 Badgers setts where I live, the only thing you need to be careful of if it is Badgers, is that as the sett gets larger the ground underfoot becomes unstable and you will need to fence it off. If you are concerned phone your local pest control and they can advise you more, I personally think it is more likely foxes, not quite such a problem.