//Perhaps it depends on sunset, rather than sunrise//
It depends on the length of the day (the time between sunrise and sunset). Although the sunrise is still getting (very slightly) later each day, sunset is getting later by a greater amount, so the length of the day increases. As well as the latest sunrise being well after the shortest day, the earliest sunset (which you would also expect to be on the 21st December) is well before that date, occurring on 12th December.
The reasons for this are a bit complex but, basically, the length of a "solar day" (i.e. the time it takes for the sun to appear to return to the longitude you happen to be on) is not exactly 24 hours. It's sometimes more and sometimes less. 24 hours is used as a mean average length of a day but the actual length of a solar day can differ from 24 hours by as much as 16 and a half minutes. It varies due to effects caused by the tilt of the earth's axis and the fact that the orbit of the earth around the sun is not perfectly circular.