You could do...after all, people have been saying they don't "give a monkey's" since the late 1950s. They often used to add another word after 'monkey's', too!
Perhaps I should have added to my earlier answer the fact that the word 'monkey' was often used in contemptuous phrases, such as 'make a monkey out of', 'I'm a monkey's uncle' etc. Hence, 'not to give a monkey's whatever' was the lowest of criticisms.