When met with defiance, a targeted threat often strikes directly at the source of a community's false security. Gideon responds to the men of Penuel with a warning aimed exactly at their pride and safety. He promises that once he returns from battle safely and unharmed, he will bring a severe punishment upon them [ביאור שטיינזלץ]. This penalty will involve completely breaking and tearing down their tower [מצודת ציון, ביאור שטיינזלץ].
The choice to target the tower is highly intentional. The people of Penuel trusted in their strong fortress to protect them. With such robust protection, they could not even use the excuse that they refused to help Gideon out of fear of the enemy. Because their entire sense of confidence rested on this fortified structure, destroying it serves as a precise, measure-for-measure punishment [מצודת דוד, מלבי״ם].