Let me rephrase my answer. Anybody can collect their mail from the local Delivery Office, but this may not be convenient, unless its very close. For instance, for me it would mean a 16 mile round trip every day.
The people that are sending you letters have paid for a delivery-to-the-door service, and this is what you should expect. If your mail is being beggered about with, then its safe to assume that others on your "Walk" are suffering as well.
You should still complain.