In 1606, the Spaniards occupied Ternate. Sultan Said became a prisoner but his kin asked the help of the VOC. In 1607, Admiral Cornelis Matelieff tried in vain to expel the Spaniards. The Dutch built fort Melayu on the East coast, carving out their own sphere of influence on the island to rival that of the Spanish. In 1609 the name was changed to Castle Orange.
Between from 1611 and the move to Batavia (Jakarta) in 1619 Castle Orange was administrative center of the Dutch East India Company in Asia and the seat of the Governor General, After 1619 it was the seat of the Governor of the Moluccas.
Until the peace of 1648, the two areas into which Ternate was divided existed in constant hostility. After the Spaniards left the Moluccas in 1663, plans were made to reduce the fort, but in practice only small were made. The new building in 1757 was necessary due to the poor state of the buildings. In 1791 the Military State Commission inspected the fort, but the designs of engineer C.F. Reimer were never executed.
In 1795, war broke out between the Netherlands (which at that time became a satellite state of revolutionary France) and England. In 1801 the English laid siege to fort Orange After five months Governor Willem Jacob Cransen capitulated. After the Peace of Amiens in 1802 the Moluccas were returned to the Dutch, but the following year war broke again. In 1810 the fort capitulated after a short shelling and the Moluccas came under English administration. In 1817 the Dutch returned.
The 19th century was a time of decline for the Moluccas. In 1840 the city of Ternate and Castle Oranje were damaged by an earthquake. damage. In the years 1870-1874 several building works were executed. In 1916 the land wall with its bastions was demolished. Till the end of the 20th century Castle Orange was used by the army and the police. Between 1980 and 2015 several restoration projects were established by the Indonesian Government. Castle Orange now serves as public space and a Spice Museum.