a person's back: the side turned away from view, and so the part that takes the blows or hides whatever is thrown behind it
Hebrew has more than one word for the back. The broader one (גַּב) can just as easily mean a wheel's rim or an eyebrow, because it is really about a curved surface. This word is narrower and almost always human. It names the back as the side a person turns away, which is why it shows up wherever someone is being struck, walked over, or finished with something.
Three of its six appearances are the same proverb wearing different clothes. Wisdom is found on the lips of the discerning, and a rod is waiting for the back of the fool. The saying is less a recommendation of violence than an observation about consequence: words reach the person who will listen, and what will not go in through the ears eventually arrives lower down.
Isaiah puts the same body part at the center of three very different scenes. The servant of the Lord hands his back over to the men who will beat it and refuses to hide his face. A conquered Jerusalem is told to lie flat so her captors can walk across her, and her back becomes the road. And Hezekiah, pulled back from the edge of death, says that God has thrown all his sins behind God's own back.
That last image is the one worth holding on to. Something behind your back has not stopped existing. It has simply stopped being looked at, and a person who will not look at a thing will not act on it either. Elsewhere the gesture runs the other way: Israel is accused of flinging God and God's law behind its own back, in verses that use a closely spelled word (גַּו).