The president said in his weekly address that the inaction and silence from Congress is "inexcusable"