What was Young Goodman Brown supposed to learn from his experience?

Quite honestly, Young Goodman Brown isn't "supposed" to learn anything.  That is the point.  Nathaniel Hawthorne presents a character that, from the very first page of the short story, knows that he is doing something wrong.  His wife, Faith, had a bad dream about his upcoming journey into the woods, and begs him to stay home.  He responds by insisting that he must go this particular night, and then asks if she doubts him already, even though they are "but three months married."  She relents and he begins his journey, but shortly after Hawthorne writes,

"Poor little Faith!" thought he, for his heart smote him. "What a wretch am I, to leave her on such an errand! She talks of dreams, too. Methought, as she spoke, there was trouble in her face, as if a dream had warned her what work is to be done to-night. But, no, no! 'twould kill her to think it. Well; she's a blessed angel on earth; and after this one night, I'll cling to her skirts and follow her to Heaven."

Even before he is out of Salem village, he feels guilt for what he is about to do, but he continues on his way nonetheless.  He knows that his wife would be horrified, even killed, to know what he is going to do.  Yet he is content to proceed, promising himself that it will be just this one time.

Further, once Young Goodman Brown is in the woods and meets up with the strange traveler, he continues to insist that he should not be in the woods, nor walking with the man.  The traveler does not insist that Brown continue, yet Brown does.  When the two come upon a figure that appears to be Goody Cloyse, a revered elder of the town, Brown hides because he doesn't want her to see him with the traveler.  Perhaps the most telling event in the woods occurs when Goodman Brown finally says he will go no further.  The traveler says that Brown "will think better of this by-and-by," but leaves Brown behind and goes on his way.  Shortly after he leaves,

The young man sat a few moments by the road-side, applauding himself greatly, and thinking with how clear a conscience he should meet the minister, in his morning-walk, nor shrink from the eye of good old Deacon Gookin. And what calm sleep would be his, that very night, which was to have been spent so wickedly, but purely and sweetly now, in the arms of Faith!

Young Goodman Brown believes that he has overcome temptation, but immediately begins to feel very proud, and starts to pat himself on the back.  In short, he has stopped walking, but because he is now giving into pride, he is moving into further and further into sin.

Goodman Brown eventually continues further into the woods and eventually discovers a strange witch-meeting, which appears to include all of the town folk believed to be "good" (including his wife, Faith).  Brown then attempts to stand up to the evil that he believes he sees.  However, everything disappears.  As Brown comes back into town the next morning, Hawthorne speculates that perhaps the entire event was a dream.  He details how Brown can no longer trust the people in town, believing he encountered them in the woods at an evil witch-meeting.  The rest of Brown's life was lived in distrust of those around him, and "his dying hour was gloom."

From the first moment Young Goodman Brown started his journey, he knew that what he was doing was wrong.  He ignored his wife's feelings, entered into the woods, and, despite having a number of clear indications that he should stop, he kept going all under the premise that it was just "this one night."  Whether or not the witch-meeting was real is irrelevant.  What matters is that Young Goodman Brown knew he was doing something wrong, and he did it regardless.  He returns so preoccupied with his perceived sins of everyone else in Salem that he is never able to see his own actions as sinful.  He develops a "holier than thou" attitude toward everyone else in town, despite the fact that the events in the woods may have been a dream.  In the story, the only person who definitely did something wrong is Young Goodman Brown himself, and he is never able to understand that.

Hawthorne presents this character as a case study in human nature.  The troubled writer spent his entire life trying to reconcile himself with his family's history.  His great great grandfather John Hathorne was the head magistrate during the Salem Witch Trials, an event that included a number of self-righteous individuals so incapable of seeing their own capacity for sin that they brought harm upon innocent people.  Like Young Goodman Brown, many of the self-righteous never came to understand their actions as sinful.  Hawthorne spent his career studying the nature of sin, including the most ignorant varieties of it.  This is why Young Goodman Brown isn't supposed to learn anything; sometimes people don't, even after their actions cause others to suffer.