The growing pressure of the famine eventually overcame Jacob's resistance. When he finally told his sons to return and buy a little food, he acted strictly out of a lack of choice [ביאור שטיינזלץ]. Judah explains to Joseph that the sheer necessity of the famine was the only thing that forced their father to send Benjamin with them.
Yet, even when he agreed to this plan, Jacob warned the brothers and shared his deep fear. He made it clear that if they did not bring Benjamin back to him, they would be the direct cause of his death from sorrow [ספורנו].