This phrase was used routinely by my parents, grandparents, their friends & relations ( n. California WASP, 1940's & 50's) in relation to any child of ambiguous paternity. It was evidently a favorite, for it was also sometimes employed to describe other situations in which behind-the-scenes activity of a disreputable kind was suspected. Ray Ott