I don't personally regard Johnson going as a "short term bauble"
If you heard Wes Streeting last night, he avowed that it would help Labour perhaps if Johnson stayed but he was still calling for him to go.
The problem isn't just Boris Johnson, and that's one reason why I am not sure the parliamentary party has the nerve to try to unseat him, and I don't think he will ever resign unless he's forced to.
When Johnson took over as PM at the height of the Brexit crisis, he effectively - almost certainly guided by Dominic Cummings - effected a mini-coup within that party by purging the cabinet of anyone who was not a pretty hardline supporter of Brexit. Many MPs lost the whip, some left the party by the time of the election etc.
This had the added advantage for him that he could claim that he was leading a "new" government.
If he goes, it's likely that half the cabinet at least would go too, depending on who took over. It's also possible that a civil war would break out between the different factions. Unless the party agrees a single candidate to take over (which could happen) the whole process could take up to three months. If it isn't done quickly that could take us quite close to an election. Then again, new leaders often get a poll bounce, as happened to Gordon Brown in 2007.
there are too many uncertainties to be able to say that Johnson going would only be good for Labour in the short term.