It houses treasures by Lowry, Hockney and Lucian Freud.
Yet despite its central location, it seems some are still completely oblivious of the town’s answer to the Tate.
So much so, Huddersfield Art Gallery’s status as a hidden gem has even been the subject for artists housing their work there.
Sitting on the top floor of the library on the Piazza, the gallery boasts hundreds of spectacular works by household names.
Lowry’s depictions of industrial Huddersfield scenes sit in a quiet corner, while a few feet away Henry Moore’s naked ‘Falling Warrior’ sculpture sprawls across the floor.
The naked figure’s presence has stirred up a lot of debate in times past.
“So controversial was the twisted, distorted figure in 1958, the council had to vote on whether to enter it into the collection,” explained Kirklees’ senior curator Grant Scanlan.
“Thankfully they opted to take it on, but it caused a huge frenzy. People were very upset about it.”
The collection, which most famously includes Francis Bacon’s Figure Study II, has been in the building since 1940.
The abstract Bacon painting depicts a woman stooped over with an umbrella, and is currently on loan to the Liverpool Tate.
Grant said: “Often things are loaned to other galleries, and they will loan things back to us. It’s like a big swap shop.”
Other famous works housed in the treasure trove include two by Lowry, depicting mill workers at Chapel Hill, as well as a sketch.
A pontillist painting by Camille Pissarro’s brother Lucian also adorns the wall.
And alongside a fleet of impressionist-style works, a life-painted portrait of none other than Sir Paul McCartney hangs.
The work, by the musician’s friend John Bratby, is far from a traditional portrait, depicting the Beatles legend with green skin sat among a sea of flowers.
Grant said: “Bratby was known for pushing for his paintings to be hung in galleries, so it wasn’t unknown for him to simply turn up, painting under his arm, asking if it could be put up.”
Sketches by Hockney and Freud feature in the Moore to More exhibition, while portraits of Joseph Kaye, “builder of Huddersfield”, and the town’s first mayor Charles Henry Jones hang in the Victorian room.
But despite the huge names, lesser-known local artists aren’t left out of the mix either.
An exhibition by print-maker David Armes currently on display features dictated recordings of him asking directions on his journey to the gallery.
Words to the effect of “Art Gallery? There isn’t one...” are printed on paper rolls around the room.
But there most certainly is one, and it pulls in 20,000 visitors a year.
These visitors are essential to the gallery’s future, as Kirklees plan to close or cut funding for several museums in the future, with the Tolson Museum first to go.
Work has been done to attract more to the gallery, with a gift shop and an area to drink coffee now on the same floor.
The library building is Grade II-listed and built in the turmoil of WWII.
Its fine edifice and statues outside representing literature and the arts survived the war - but may not survive further council cuts.
Grant added: “We really encourage people to come and visit the gallery.
“Many are often really surprised when they come expecting to see a few bland watercolours but are met by some of the big names, and there is a lot which tells you all about the town’s heritage.”