Go back, but bring a cell phone or make sure you can borow one. He threatens to "take care of it himself" tell him you will call the authorities on an animal cruelty charge. At the very least it will inconvenience him, or just take care of it yourself.
Just dont punish Rocky by not going back.