After all, Freiza and his father thought of the saiyans as literal monkeys. The saiyans were born powerful, but lacked the drive to train and make themselves stronger. Essentially, Goku, by finding Gohan and training with Earthlings became exempt. Correct me if I'm wrong, no other warrior aliens lived on Earth in the early days. Vegeta and Nappa, both sportsng futuristic wartime tech could not adequately detect what powers were present in Goku. I think it is safe to say that Frieza, by operationg a transgalactic or possibly universe size ring of conquerers had the tech all the time to stop altruistic planet protectors and saviors.
But Frieza inside feared the saiyans. The prophecy of a super saiyan made Frieza think he could be toppled by brute force. When it was revealed in Goku, Frieza's fear met him halfway.