The very first reference about this castle is from 1391 when the Berka from Duba divided a part of his possession among his three elder sons. He kept Houska, Chudy Hradek, Milcany and Frydlant for himself and his younger sons. The castle is located over the junction of streams running through the rock valleys to the villages Drchlavy and Drevcice. The position of the castle is very strategic. The access was possible only from the south side, where it used to be protected by s 150 meters long and 4 meters high rampart.