Why does Caliban hate Prospero so much in The Tempest? In his view, what has Prospero done wrong?

Caliban believes that Prospero treats him badly and stole his island from him.

When Prospero came to the island, Caliban was already there. Prospero made him into his servant, or slave, and took control of the island. Caliban believes that Prospero treats him unnecessarily cruelly and accuses him of stealing the island from him. He thinks the island is his birthright because his mother was there first.

According to Caliban, Prospero tricked him by treating...

Caliban believes that Prospero treats him badly and stole his island from him.

When Prospero came to the island, Caliban was already there. Prospero made him into his servant, or slave, and took control of the island. Caliban believes that Prospero treats him unnecessarily cruelly and accuses him of stealing the island from him. He thinks the island is his birthright because his mother was there first.

According to Caliban, Prospero tricked him by treating him kindly in the beginning so that he would show Prospero the secrets of the island.

This island's mine, by Sycorax my mother,
Which thou takest from me. When thou camest first,
Thou strokedst me and madest much of me, wouldst give me
Water with berries in't, and teach me how
To name the bigger light, and how the less,
That burn by day and night: and then I loved thee
And show'd thee all the qualities o' the isle … (Act 1, Scene 2)

It is Prospero’s contention that Caliban tried to assault Miranda, and therefore their treatment of him is justified.  Caliban doesn’t deny it, using the accusation to tease Prospero and Miranda. They both hate Caliban, and Prospero continuously threatens him.

When Caliban meets Trinculo and Stephano, he tries to convince them to kill Prospero and take the island from him.

Remember
First to possess his books; for without them
He's but a sot, as I am, nor hath not
One spirit to command: they all do hate him
As rootedly as I. (Act 3, Scene 2)

Caliban claims that everyone on the island hates Prospero and he has enslaved all of its magical creatures and inhabitants. Prospero definitely seems to use Ariel to do his bidding. Although Caliban says that once they take Prospero’s books from him he will be helpless, Prospero is playing a much better game. Caliban doesn't have a chance against him.

However, at the end of the play, Prospero decides to let go of all of his grievances. He could have punished Caliban and his confederates for their attempted attack on him, but he chooses not to. He lets go of his magic and leaves the island to Caliban.