The Historic England website refers to the walled gardens and 'pleasure grounds' north of the Castle (https://historicengland.org.uk/listing/the-list/list-entry/1000231?section=official-list-entry):

"GARDENS AND PLEASURE GROUNDS The area of formal gardens immediately around the Castle include... the extended walled garden to the north...

...

"The rectangular walled garden (listed grade II) stretches some 200m from the north face of the Castle, over rising ground. A strong formal axis is maintained northwards through the three main compartments. The first, closest to the Castle, contains a central flagged path, laid in 1995 to replace the previous grass surface, which is flanked each side by five free-standing sections of clipped yew hedge. There are rectangular lawns to either side with a flagged perimeter path and herbaceous borders along the east and west walls. A central set of steps leads up to the second, 50m long compartment."

"The walled garden appears to have been laid out soon after the Castle was built, but was extended before the C17. In the C18 it was in use as a kitchen garden and orchard and was redesigned by Col Lowther with a central axis flanked by roses and with herbaceous borders along the east and west walls. The present (1990s) formal treatment and the addition of the northernmost compartment and loggia were the work of Walter Hines Godfrey in 1933-5."