Born in Germany, Frederick Graue came to the United States and settled in Fullersburg, Illinois, in 1842. In 1849, he and a partner, William Asche, purchased the site of a burned-down sawmill and built a gristmill there. Limestone for the basement walls was quarried near Lemont; bricks for the rest of the walls were made from clay from the Graue farm and fired in kilns near the mill site. Flooring, beams and posts were from white oak timbers cut along the I & M Canal. The four one-ton buhrstones used for grinding were imported from France. After the gristmill opened in April 1852, it ground wheat, corn, and other grains produced by local farmers.

The mill was a major center of economic life during the 19th century. Frederick Graue and his third son, F.W. (William) Graue operated the mill for 70 years until modern milling methods rendered it obsolete and the building was abandoned.

Graue Mill was listed in the National Register of Historic Places in May 1975. In 1981 it was recognized as an Illinois Historic Mechanical Engineering Landmark by the American Society of Mechanical Engineers — the only gristmill so designated on a national or local level, representative of an important technology and era in the history of America.