This is an English idiom which is frequently used. It is used in a metaphorical sense and does not relate either to haystacks or to needles.
The expression is used to denote that something is extremely difficult to find. In fact, it suggests that something is nearly, if not completely, impossible to find.
It is suggested that this phrase was first used in a more literal sense among farmers when they lost things in their haystacks. The first recorded usage dates to the year 1834 and can be found in The Complete Works of Washington Irving. The citation reads “if I want to find any particular article, it is, in the language of an humble but expressive saying, — looking for a needle in a haystack.”
Although this is the first recorded usage it is widely accepted that the phrase was used in spoken language prior to this and was fairly well-established by 1834. Many people attest to the fact that the expression actually dates to the early 1600’s although there is no concrete evidence to support this.
An example of the usage of this expression can be found in a quote from Jared Sandberg. He is quoted as having said “one of the Internet’s strengths is its ability to help consumers find the right needle in a digital haystack of data.”