Limitations to study

Since I have chosen a highly complex research question that relies on interpreting data from conflicting and often partisan sources, there are many limitations to my study.

The first limitation is the context in which the research question was developed.  Perhaps it would have been better to ask a quantitative question; “Was there a difference in the way that COINTELPRO programs were conducted against NEW LEFT and WHITE HATE targets?”   This is a question that possibly should be resolved before investigating the relationship between Hoover's use of bureaucratic politics and the efficiency of COINTELPRO operations in the two programs.

A second contextual issue is that bureaucratic politics is just one factor in several that may have affected COINTELPRO success; there is the possibility that the internal culture of the FBI interfered with the ability of agents to infiltrate the New Left, the possibility that Sullivan's attempt to replace Hoover as director and the resultant fall-out from that situation affected field operations, and the possibility that political liberals who had no problems in countering the right-wing terror of the Klan suddenly discovered “Constitutional” concerns when the terrorists of the Left were targeted, and thus abandoned the alliance with Hoover.

A limitation involving the sampling design is that the sample I used to generate my content analysis is relatively small compared to the amount of literature that exists;  I could have used additional material from the Church Committee (Intelligence activities,1976), from Theoharis (2004), from Jeffreys-Jones (2007), from Elliff (1979), and from Cunningham (2003).  I chose specifically to avoid using material from Churchill (1990) due to the extreme level of bias in that material.

The research instrument I used was a content analysis.   I was unsure of both my use of this method (my first attempt), and the thoroughness of my attempt. There is software that can aid in this approach, but I did it manually.  It is worth the effort to redo this analysis with the addition of additional source material and a more efficient approach. 

There is also the issue in data collection of estimating the effect that COINTELPRO had on the New Left.  DeLoach (1995) notes this difficulty and briefly mentions possible factors for the subsequent decline in New Left activity which do include COINTELPRO. 

Another data collection limitation is the difficulty in separating Hoover's personal identification with the agency from his direction of the agency for the Bureau's sake.  For instance, Gentry (1991) notes an example in which Raymond Chandler, the writer, insulted Hoover personally, and was subjected to the collection of 250 pages of personal data in an FBI file. 

The last data collection issue is in using data related to Assistant Director Sullivan's statements.  The literature demonstrates that Sullivan had a pattern of telling his audience what they wanted to hear at any given time.  This applies particularity to data regarding New Left activities;  Sullivan enthusiastically took part in organizing the Huston plan directed at the New Left, yet in his autobiography claims that the New Left was not a threat...this, at a time Mark Felt was being prosecuted for covert FBI operations.

Finally, the theoretical underpinnings of the study could use some fleshing out.  Bureaucratic politics is a good model to explain the basics of how Hoover used these particular tactics and why they succeeded on an organizational advancement basis, but a model I referred to as the “politics of the personal” could have more fully explained why Hoover made these tactical choices.  “Politics of the personal” was a theory I was exposed to in undergraduate political science some 25 years ago, but I was unable to locate a source explaining it, and the only examples I was able to find of it in use were out of the context I was familiar in understanding; it seems that the term had been hijacked by 3^rd wave feminism.

References

Churchill, W. (1990). The COINTELPRO papers: documents from the FBI’s secret wars against domestic dissent. Boston, MA: South End Press.

Cunningham, D. (2003). Understanding State Responses to Left-versus Right-Wing Threats The FBI’s Repression of the New Left and the Ku Klux Klan. Social Science History, 27(3), 327–370.

DeLoach, C. (1995). Hoover’s FBI: the inside story by Hoover’s trusted lieutenant. Washington, D.C. : Lanham, MD: Regnery Pub.

Elliff, J. (1979). The reform of FBI intelligence operations. Princeton: Princeton University Press.

Gentry, C. (1991). J. Edgar Hoover: the man and the secrets. New York: Norton.

Intelligence activities and the rights of Americans. 94^th Cong 1. (1976).

Jeffreys-Jones, R. (2007). The FBI: a history. New Haven: Yale University Press.

Theoharis, A. G. (2004). The FBI & American democracy: a brief critical history. Lawrence: University Press of Kansas.

State the consequences of the limitations.  For example, what is the consequence of a small sample?  You mentioned content analysis - how can the results be validated?

All studies rely on validity and reliability to be of use; validity refers to the accuracy of measurement and reliability refers to the stability or the consistency of the measuring instrument (Hagan, 2012), in this case, my content analysis.  If my content analysis valid, then it accurately measures what I mean it to, the percentage of supporting literature that had discussed the specific use of the tactic of bureaucratic politics that I had identified as a variable.  If my analysis is reliable, then I should get the same results every time I repeat the analysis with the same data.  If my study is not both valid and reliable, then the study is of no use and presents no useful knowledge.

Sample size is important in the validity of a study. Triola (2014) warns that a sample size can be too small in statistical sampling.  On the other hand, Krippendorff (2013) explains that the content analysis method deviates from the representational mode of sampling theory in that the researcher is seeking to sample texts that can accurately answer the research question, not to seek to represent the textual population.

Krippendorff discusses validity in terms of face validity, social validity, and empirical validity.  Empirical validity is what we are discussing in relation to research technique.  Krippendorf specifies three obstacles to validation; substantive, conceptual, and methodological obstacles.  Krippendorf suggests that validating evidence used in the content analysis can be achieved by two methods.  The first is by measuring the correlative validity, which involves checking the findings obtained through the content analysis with other findings from methods considered more valid.  The second is predictive validity, in which the findings of the content analysis accurately anticipate knowledge which was not in the analysis.

Hagan, F. E. (2012). Essentials of research methods in criminal justice and criminology (3rd ed). Boston: Prentice Hall.

Krippendorff, K. (2013). Content analysis: an introduction to its methodology (3rd ed). Los Angeles, Calif.: Sage.

Triola, M. F. (2014). Elementary statistics (12th edition). Boston: Pearson.