This is really not the ideal set of interactions between the extension manager and its related components. Unfortunately changing it is going to be hard.
Unfortunately adding any new features is also pretty hard until at least some of these dependencies are broken.