This is one of those times that being a King James reader only can cause a person problems. Our English language has changed dramatically over the last several hundred years. Consequently, words used at that time may not convey in any accurate way to us what was meant or understood by those words hundreds of years ago.
The rebuke is to the kings, governors, judges or those exercising political or ruling authority.
Read it from the Amplified and you will catch the idea. The Lord is speaking to those in authority (rulers, governors, judges etc.) that should be ruling and deciding justly in His name.
PSALM 82 - Amplified Version
1 GOD STANDS in the assembly [of the representatives] of God; in the midst of the magistrates or judges He gives judgment [as] among the gods.
2 How long will you [magistrates or judges] judge unjustly and show partiality to the wicked? Selah [pause, and calmly think of that]!
3 Do justice to the weak (poor) and fatherless; maintain the rights of the afflicted and needy.
4 Deliver the poor and needy; rescue them out of the hand of the wicked.
5 [The magistrates and judges] know not, neither will they understand; they walk on in the darkness [of complacent satisfaction]; all the foundations of the earth [the fundamental principles upon which rests the administration of justice] are shaking.
6 I said, You are gods [since you judge on My behalf, as My representatives]; indeed, all of you are children of the Most High.
7 But you shall die as men and fall as one of the princes.
8 Arise, O God, judge the earth! For to You belong all the nations.
The Psalmist is appealing for those who administer justice and govern on the earth to do so in defense of the poor, needy, and afflicted. Apparently their judicial system had become corrupted by those who would take bribes, etc.
In doing so they were perverting justice for the most needy among them. This is a common theme brought up in the writings of the prophets as well. Isaiah is one that devotes a good bit of his writing to it in particular.
Hope that is helpful in making better sense of it. I love the KJV, but if it is all a person reads in English it will cause lots of misunderstanding because our language has changed so much since it was written.