If you find out the culprit, make a deposit of your own into a paper bag and drop it in through their letter box.
My grandmother did this with a friend when she was a girl. With a mate, they filled a brown envelope with horse-turd, sealed it and wrapped it in string, leaving it on the pavement, a man picking it up and taking it home, the two girls following to see where he lived.....