Serle's House in Southgate Street was built around 1730 as a private residence. The building has been in military use since 1781 when it was bought by James Serle who's son was a captain in the South Hants Militia who turned the family home into a command centre for the regiment. The house was later sold to the government in 1796. The building now houses the Museum of The Hampshire Regiment and the Offices of the Lord Lieutenant of Hampshire.
The large garden to the front of the house is dedicated and maintained as a Garden of Remembrance to all Members of the Regiment who have died in action or whilst in Service with the Regiment.