This expression of surprise is still in use in Yorkshire and other parts of northern England. I have always thought it arose from the habit of using words with similar beginnings to rude or religious words to avoid giving offence. For example barsteward or sugar. In this case, the original expression would be "By God".
The 'ee' part at the beginning is just a more or less meaningless interjection such as 'eh' or 'oh'. The 'by gum' element is exactly as Grunty says, a euphemism for 'by God', in much the same way as 'Cor blimey!' is a euphemism for 'God blind me!'