Three days in England - Overwhelmed by options - Looking for ideas

Looking for ideas - we're two adults in our mid-30's, no kids. We'll be leaving a wedding on a Monday in Ipswich but our flight home from London isn't until Friday. We'll be spending a week in London beforehand so we'd like to explore the countryside.

The only thing we've (sort of) landed on is seeing/staying in the Cotswolds, and touring some castles (holy crap there's a ton to chose from). We're not big drinkers/partiers but I'd like to be able to go hang out in a real English pub and stay at a Lord-of-the-Rings kind of inn (I already found the inn that the Prancing Pony is based on - thought not sure if we want to stay there).

Other random thoughts. Unfortunately our schedule doesn't seem like it'll line up with any Premier League games. Not planning on renting a car but can if we absolutely must. Would kind of like to stay in one spot since it's only 3 nights.