The palace (Schloss) was originally constructed as a cloister priory of the Benedictine monks of Weingarten in the year 1654 by Michael Beer from the Vorarlberg region. In 1802, it lost its status as cloister in the process of secularisation and in 1806, was taken over by the then kingdom of Württemberg. During the years 1823 to 1830, the chateau was converted to a royal Summer residence for King Wilhelm 1st of Württemberg, in accordance with the plans of the then royal architect Giovanni Salucci. The rather monotone South wing of the cloister was adapted and a balcony, two stories high with a superimposed half-storey, was added to the centre, endowing the building with the aura of a palace. The royal private chambers were located in the West wing, the community chambers and guest rooms in the East wing.

During the air raids in 1944, the chateau was badly damaged by fire bombs. Three of the wings burnt down to the vaults of the ground floor and the exterior stone wall was destroyed, even the neighbouring buildings were reduced to rubble. The construction of an emergency roof in the Winter of 1948/49 prevented the further collapse of the ruin. Over the period between 1951 and 1965, the chateau was reconstructed in accordance with historical architectural regulations. Only the South wing of the farm / inn was reconstructed, the three remaining wings were removed in 1958.

Today, the West wing and the South wing of the former residence are inhabited by the ducal family, the ducal administration is lodged in the East wing. The palace is the residence of Friedrich, Duke of Württemberg and is not open for visitors.