The Huddersfield Temperance Society was formed on 2 March 1832 with around 30 members. Meetings were initially held in a room at the infants school on Spring Street, with subsequent venues including school rooms on George Street and the Philosophical Hall on Ramsden Street.
In 1870, the society purchased the Cambridge Arms Music Hall (formerly the Black Swan Inn) on Upperhead Row after its licence was not renewed at the Brewster Sessions of 1869. It was then converted it into a "public house without the drink" at a reported cost of £2,000. This soon proved to be too small a venue and the purpose-built Victoria Temperance Hall was opened on Buxton Road on 1 March 1880 at a cost of about £10,000.
The foundation stone of a new meeting hall between Princess Street and Page Street was laid on Saturday 25 August 1900, by which time the society had sold the Victoria Temperance Hall to the Huddersfield Industrial Society.