If you hold it for too long, the apple logo will appear on the devices screen. If this happens, let go of the buttons immediately. This is called a "soft reset" and will reset your phone (not to factory, don't worry) and then turn it back on, so you will then have to try to turn it off again.