Frederick Graue emigrated from Germany to the U.S. in the 1840s, eventually settling in Fullersburg, Illinois, part of today's Oak Brook.

In 1849 he and William Asche purchased land along Salt Creek where they would build a gristmill.

Limestone for the mill's basement walls came from a quarry near Lemont. Bricks for the other walls were made from clay from the Graue farm and fired in kilns near the mill. Flooring, beams, and posts were from white oak timbers cut along the I & M Canal. The four 1-ton buhrstones used for grinding were imported from France.

The mill opened in April 1852 and became a major center of economic life during the 19th century. Local farmers routinely brought their wheat, corn, and other grains to the mill for grinding. The nearby Frederick Graue House was the family's 1850s Victorian home. The Graue family operated the mill for 70 years, but modern milling methods eventually rendered the mill obsolete, and it was abandoned.

When the Forest Preserve District acquired the mill in 1933, boards covered the windows and door. Exterior bricks were crumbling, and the mill’s machinery no longer worked.
In 1934 the Forest Preserve District decided to restore the mill to its mid-1800s operations. Men of Civilian Conservation Corps Camp V-1668, which was based at Fullersburg Woods Forest Preserve, replaced the windows and door, cleaned out the interior, and built a new millrace. Since the 1940s, periodic restoration work has kept the mill in working order.

In May 1975 Graue Mill joined the National Register of Historic Places, and in 1981  the American Society of Mechanical Engineers recognized it as an Illinois Historic Mechanical Engineering Landmark.