The first American Labor Day was held in 1882. Its origins stem from the desire of the Central Labor Union to create a holiday for workers. It became a federal holiday in 1894.
It was originally intended that the day would be filled with a street parade to allow the public to appreciate the work of the trade and labor organizations. After the parade, a festival was to be held to amuse local workers and their families. In later years, prominent men and women gave speeches. This is less common now but is sometimes seen in election years.
One of the reasons for choosing to celebrate this holiday on the first Monday in September and not on May 1, as is common in the rest of the world, was to add a holiday in the long gap between Independence Day in July and Thanksgiving in November.