The scaffold, in Puritan society, was a place of public penance. This is where many punishments were carried out and this place was chosen for that task because it was very public. Since it stood up above the crowd, most people could easily see the person being punished. Punishments varied from a simple public announcement of a person's sin/crime to some very severe physical punishments. Thus, the scaffold becomes the symbol of humiliation and punishment.

Additionally, since you included the book title in your question, the scaffold also is the only place where all four main characters are together, aside from the Governor's Hall. In that sense, the scaffold also serves as an effective plot device to further the story.