It will depend on the class of the adhesive, and may be worth experimenting with some scrap materials before risking damaging the box and felt.
PVA might respond to hot water – the question with water or any water-based solvent is what it might do to the woodwork, which is unlikely to be protected under the felt.
An impact adhesive like Evo-stick, or a rubber-based one, might dissolve in meths, isopropyl alcohol or white spirit, though I've not tried these. There are some solvents made for removing adhesive labels safely – I do not know if they will touch cured impact-adhesive, but again, might be worth a try.
'
At work I used to use a very effective but remarkably safe, citrus-oil based, solvent intended primarily intended for cleaning electrical equipment. I'm sorry, I forget its name but I am fairly sure it was an Ambersil product, made for cleaning electrical equipment. The present Ambersil site lists what seems the nearest equivalent, Ambersolv SB1. The soiling it claims to remove include include rubber-based adhesives.
I don't know if Ambersil sell directly. Possibly not, especially to single-use buyers, but the material ought be available through a tools or automotive factor. The solvent we bought came in 5l bottles via RadioSpares I think, but Ambersil seems now to list only aerosol cans, and if it works, like all the solvents, you might need a fair bit thanks to the shielding given by the felt.
'
Synthetic resins are well-nigh insoluble. I have encountered professionally one or two nasty solvents that do attack them, but very slowly. However you'd probably find them very hard to obtain, they might not reach the adhesive without evaporating from the felt, and they could do as much damage to the wood and felt as to the glue. And to you if don't handle them with the care they demand.
Whatever you try, could end up doing more harm than good, unfortunately, hence my suggestion that you use test-pieces first, from scrap materials.
May be worth contacting a professional furniture-restorer for advice.
'
Personally, faced with the same problem, I'd revise why I want to expose the wood, unless the felt is so badly damaged and filthy it is neither use nor ornament. (In which case perhaps to renew it, especially if restoring an antique.) After all, it was put there for a purpose!