Probably an ignorant question - never studied the issue - but, did anyone question the involvement of the Howards or Arundels in the Gunpowder Plot? Shortly thereafter, many of them left England for Virginia and later Maryland in particular where they established an aristocratic Catholic refuge for awhile.
At this point in time though many of the "royal" families had suffered greatly in the years since the death of Henry VIII. Several key plots against Elizabeth had resulted in many of these leading Catholics exiling themselves, Arundell being one of them.
There are quite likely a lot of skeletons in the closets of some of these characters, although it is unlikely that we will ever know for sure. Many of these characters were influential in the exiled council that resided in the Low Countries, and although never directly implicated in the plot, there is certainly the belief that many were aware of an impending action against the monarchy.