FURIOUS MP Barry Sheerman today demanded urgent action to develop Castle Hill.
He called on Kirklees Council to consider buying parts of the site owned by the Thandi Partnership developers to enable work to go ahead.
He added that the Almondbury site in its current state was an appalling advert for the whole of Huddersfield.
The council last week admitted it could do little work on the site until it was known what plans the Thandi partnership had.
The Thandis were forced to tear down the new hotel they were building on the site after it broke planning rules.
Mr Sheerman said the site was a valuable asset for Huddersfield and needed work now.
He said: "I took a visitor from the United States up to Castle Hill to show Huddersfield at its best - and it was appalling.
"The whole place looks awful. I was humiliated to be showing it to a guest.
"It is one of the most important heritage sites in the whole of Yorkshire and a symbol for the town. It is just ghastly.
"There is an exciting opportunity to create an education and visitor centre up on Castle Hill and there is money available in the form of grants for such projects.
"The council should consider issuing a compulsory purchase order for the site."
Improvements costing £220,000 at Castle Hill are due to get the green light next week.
Better parking, footpaths and visitor information are planned for the historic landmark.
But future phases depend on legal wranglings over the former pub and restaurant being settled, the council said.
Clr John Smithson, the council deputy leader, said: "The conservation management plan was not intended to resolve issues relating to the former pub and restaurant.
"But the plan does include proposed policies that will provide a framework for considering whether the former pub and restaurant could be replaced if a planning application is put in."
The work likely to be approved by a Cabinet committee at next Wednesday's meeting could begin this year.
Clr Smithson added: "The enhancement proposals are a significant step in the wider plans to restore this very important and historic site."