we are dealing with hierarchy of objects: text object is a child of panel. Generally some objects properties are inherited from it’s parent: if you change position or orientation of parent object, child objects also change thier positions.

However in general opacity property is not inherited from parent object, because otherwise it would be hard to make parent object more transparent than child. Opacity is inherited only from groups, because group doesn’t have it’s own body and it’s opacity can be used to simultaneous control of children opacities.

So, in your case you should create a group above the panel. Group panel with some temporary object (by selecting two objects together and pressing G) and than remove temporary object from result group using hierarchy tree.