I'd also make William and Sons, as well as William Evans. While on St. James St., also visit the Beretta Gallery, just across the street.
While in the neighborhood of Purdey's, eat lunch (or dinner) at the Audley, just across the street. They have the best fish and chips in London.
On Mount Street, not far away, is a great venue for an Italian lunch, Serafino's. You can go upstairs, or downstairs...go downstairs, same good, cheaper. The seafood pasta bowl is to die for.
Be sure to take in Rule's Restaurant (oldest restaurant in London). Make a reservation for dinner, they are famous for game dishes, ask a manager for the tour (and stories of royal illicit liasons in a private room).
