When changing selection from object to text frame swatches should change to the text itself intead of the text box.
I always accidentially fill the text box with colour first instead of the text. I think, users want to change the text colour a lot more often than the colour of the text box.