Fruit of the poisonous tree is a legal metaphor used to describe evidence that is obtained illegally. The logic of the terminology is that if the source (the "tree") of the evidence or evidence itself is tainted, then anything gained (the "fruit") from it is tainted as well. Therefore, one may not need to refute the evidence, just prove that the evidence was gained illegally.
Similarly, at least in my mind, if you get money or fame in the wrong way, it doesn't count. The success becomes tainted and seizes to be success.