Sampling Issues

Kirk sent me an e-mail asking about my sampling issues. I thought I might start writing about them here. To see the problem, you’ll need to know a bit about my argument.

What The Hell Are You Trying to Prove?

Er, test. We never prove. We only test hypotheses. My dissertation is a theory about ethnic cleansing during civil war. I argue that places that recognize sub-group identity and lend it legitimacy by granting it representation are more likely to wind up with ethnic cleansing during a civil war than are places where sub-group identity is officially ignored and the state pretends that society is composed of atomized individuals. The logic behind my thinking is that human moral sensibilities are inherently context-driven. Very few people believe that any specific act is universally wrong or right across time and space. We look to context to determine what actions are appropriate in given circumstances.

Killing someone is always a difficult act for individuals who are not psychopaths. To kill someone requires objectification. For a normal person to kill someone, you can’t stop and think, “This guy is just some other guy like me. He’s a human being who had a mother, too.” You can’t pull the trigger. You have to objectify them. You can’t view them as the same as you.

In countries where sub-group identities are a daily part of the deal, individuals spend most of their day “slotting” people. They are always conscious of out-group identity. When you spend all day slotting people as a matter of business, then you get used to it. This makes objectification easier. Moreover, your moral obligations within civil society are constructed as being different toward different people based on group identity. In countries where the government doesn’t operate under the assumption that everyone belongs to a group and that is the natural way to do business, there exists a polite public fiction that everyone is the same and differentiating between groups is rude. That fiction serves as a barrier of sorts for ethnic cleansing. If you’re already divvying everyone up into in-group and out-group all day long, you’re a step closer to the sort of objectification needed to kill. It’s an easier sell for politicians who need your support.

The Jordanian Civil War didn’t have ethnic cleansing. The Lebanese Civil War did. Both civil wars had similar causes—the destabilizing presence of armed Palestinian guerrillas. Levantine Arabs are very culturally similar. Jordanians and Palestinians are for all intents and purposes the same people. Same dialect, same food, there was no distinction between them at all prior to the British drawing a line between then in 1921. There’s a little more cultural distinction between Lebanese and Palestinians/Jordanians. But not much. So virtually everything is the same except the state and its system for recognition. Lebanon distinguishes between groups. The whole system is built on it. Jordan doesn’t formally distinguish between groups. The whole system is built on pretending there's no difference. So we have a natural experiment of sorts.

How Are You Going to Prove (er,Test) It?

Well, the lack of ethnic cleansing in the Jordanian civil war is a matter of record. Yes, civilians got killed, no one claims it wasn’t bloody, but these were mostly collateral damage in an attempt to root out the guerrillas. Jordan did not become Rwanda. Ethnic cleansing begins in Lebanon almost immediately after Ayn Rumaniyya. We can draw on secondary historical sources for evidence.

The question is finding a way to trace the causality. There’s a lot of psych stuff I can draw on. Moreover, there’s media stuff I can use to show how the press shapes public discourse and how the state creates the framework of basic assumptions the media run with. But what I need to prove is that public discourse in Jordan and Lebanon are shaped the way that I say that they are. So for this, I am gathering a sample for what is called a content analysis, a statistical study of the properties of given texts.

On the basis of my argument, I make the following predictions about Jordan and Lebanon newspapers from the period:

1. At all times discourse about domestic politics in Lebanon will make more references to sub-groups and fewer references to Lebanese as a whole than in Jordan.
2. In both countries, we should see a rise on objectifying language used to describe sub-groups the closer we get to civil war, sustained objectifying language during the civil war and a slow decline in objectifying language after civil war.
3. Nonetheless, the extent of objectifying language will remain lower in Jordan during all three periods

The Sampling Problem

I have selected two major newspapers for each country that were in publication prior to, during and after the civil war. For Jordan, I selected ad-Difa` (The Defense) and ad-Dastur (The Constitution). For Lebanon, I selected an-Nahar (The Day), al-Hayat (Life) and as-Safir (The Ambassador). I spent some time with an online calculator that suggested to me that with a sample of 300 articles, I could probably get the coveted p=<0.05. So for each period (before, during and after the civil war), I will sample 300 articles, for a total of 2400 coded articles for the damned project. Half of each sample will be domestic politics news articles (150) and half will be domestic politics editorials (150). This way, there is a chance that I can differentiate between the two genres, as I imagine the more objectifying language will crop up in the editorials.

I’ve just surveyed the Arab Press Archive’s holdings for ad-Difa` and ad-Dastur from the dates June 21, 1967 (the day after the loss of the 1967 War) to August 31, 1970 (the day before the start of the Jordanian Civil War). This period contains a total of 1,168 days. Both dailies publish every day of the week including Friday, so we should expect both papers to have that many issues. The Arab Press Archvie has 804 issues of ad-Difa` from this period, 68.84 percent of the estimated population, and 722 issues of ad-Dastur, 61.82 percent of the estimated population.

My first problem is whether I should take 150 articles and 150 editorials from a combined pool of 1,526 issues of both papers combined, or if I should sample 75 articles and 75 editorials from ad-Difa` and 75 articles and 75 editorials from ad-Dastur. Ad-Difa` is a privately held paper that had its origins in the Palestine Mandate, but was published from Amman after 1948. Ad-Dastur is a government-run Jordanian newspaper. These two papers had the highest circulation in Jordan at the time. As ad-Dastur is a state newspaper and the Hashemites were avid state builders, you would expect it to virtually never distinguish between Transjordanians and Palestinians. Ad-Difa` would be more likely to not toe the line, but it’s important to note that the papers survival in the period we are studying rests on it not rocking the boat too much.

So should I split the sample, or take from a pool? Right now, I’m splitting the sample. Kirk said he’d look at what I’m doing and give advice, as his work often rests on getting random samples. Anyone who wants to pipe in is more than welcome.

Reading through al-Difa`

One of the things I’ve noticed as I read through al-Difa` is it’s genuine lack of domestic political coverage. Granted, I’m reading September 1967 right now, so you would expect foreign policy to predominate. Jordan has just lost a war and the whole West Bank in the process. Domestically, you would expect little participation. Jordan is under martial law. But still, I’m floored by just how few articles deal with domestic issues. I expected sub-group references to be muted. But to be honest, reference to even a general public is muted. The press in Jordan so far seems to represent to the public what the state is doing. The public itself isn’t really covered. It reminds me of Habermas' description of the Château de Versailles, with the whole point of the edifice being to represent the king's power to the people, not the people's opinion to the king. I think of Gretchen working at King Abdullah II’s press office and rather wonder about her take on this. It also makes me think of Ellis' fun class studying Arab monarchy by reading Shakespeare, as those plays are a good English source for understanding monarchy. I wonder if Jordan being a monarchy might undermine the comparison I'm drawing. Still, Jordan isn't Syria.

Lebanese politicians, spewers of endless bullshit that they are, nonetheless address constituencies. I’m not seeing much of this in the Jordanian press as I read.