How do I know if I can trust the survey results to be accurate?

Because the 2006 RAW Survey was conducted via open invitation to all interested parties, rather than based on random sampling from a known population, it is not legitimate to calculate estimates of “sampling error” that would indicate how close our results are to the statistics for the entire world population of photographers and imaging practitioners.

Moreover, because this is the first survey of its kind, there is no existing “external standard” against which the results for the 2006 RAW Survey can be compared to check “accuracy.” The results presented in our report were reviewed and checked repeatedly, and are an accurate representation of the experiences, perceptions, and beliefs of the individuals that completed the questionnaire.

Having a very large number of respondents also provides some protection against the survey results being “biased” in a dramatic way. For example, we examined the distribution of responses to all survey questions at the end of the second day after the launch date, when over 8,000 persons had responded. We compared those initial results to the statistics generated when there were about 11,000 respondents, later when there were 15,000 respondents, and finally, with over 19,000 respondents. The statistical distributions for each answer category for all survey items were always within 1 or 2 percentage points at each point in the data collection process. Thus, having large numbers of respondents minimizes the chance that the survey results have been slanted or biased in a particular direction by respondents with unusual views.

Calvin Jones – Mon, 2006/04/24 – 10:01pm