Alan Clark used it once. John Major complained. Clark said it was a reference to President Bongo of Gabon. He was a lad, wasn't he!
It may be a reference to bongo drums or to the frequency of the sound bongo in some African languages. The former sounds more plausible: "I hear the sound of distant drums, Carstairs! My God, what are they saying?"
We know what he meant, don't we? He means any supposedly Third World country that gets our money. By chance, they are full of black or brown people whose leaders spend it on Paris apartments, Ferraris and Ray Bans (Ray Bans? How naff! Not that the others are much better. At least he didn't say Rolexes. Not clear whether he thought these things suitably aspirational or in bad taste)
He was, according to him, speaking in the language of people at the rugby or golf club (inter alia) the language of the plain- spoken people of Yorkshire, who he represents. I can just imagine Fred Trueman saying that down the golf club! Ah, sweet! Didn't know we still had an Empire full of inferior races