Linguistics 300, F13, Undergraduate tutorial

Linguistics 300, F13, Assignment 3

As discussed in Coded corpora, CorpusSearch coding queries encode the results of many ordinary CorpusSearch queries as a single string of characters. For this assignment, you will download a file containing coding strings from coded versions of the Penn Parsed Corpora of Historical English and the Parsed Corpus of Early English Correspondence. The meaning of the various symbols is described in the coding conventions. In order to analyze the coding strings, you will be using a Linux/Unix command called grep (an acronym based on 'global regular expression print'), and not surprisingly, you will need to learn a bit about regular expressions in order to use it. The tutorial on grep provides a summary of the regular expressions that you will use.

As you'll soon see, the grep searches that you need for this and following assignments are quite repetitive. In order to save time and to avoid making typos in your command-line input, it is very convenient to run the searches in batches rather than on a one-by-one basis. The tutorial on shell scripts provides you with a sample script called batchGrep that you can edit for the first part of the assignment below. In order to complete the second part, you will need to edit the sample script called spreadsheetGenerator.

Assignment

An important concept in dealing with large corpora is the sanity check. In large corpora, it isn't humanly possible to reliably check every single last piece of data for errors "by hand", but it is possible to identify types of data that are logically impossible (or even just very unlikely) and then to search for them (hoping that we don't find them and reporting or correcting them when we do). For example, no clause in the corpus can be both a negative declarative and a question.
Based on the information in the coding conventions, come up with sanity checks of this sort and implement them in a shell script, with appropriate comments for each line, as in the examples below.
```
echo "negative declarative can't be a question and vice versa"
grep -c "^[^_]:.:.:[^_]" ling300.cod.ooo

echo "main verb BE in column 1 implies the same in column 2"
grep -c "^B:[^b]" ling300.cod.ooo
```
Submit the shell script, together with the results you obtain from running it. Your shell script should contain your last name (e.g., batchGrep-chomsky).
An even more important concept, and one that is central to all scientific enterprise, is replication. The availability of the historical corpora allows us to replicate the pioneering work carried out by Elleg$aring;rd 1953 and reported in Kroch 1989. For this part of the assignment, replicate Table 3 from Kroch 1989:223 (p. 22 of the version on Kroch's website). From your results, produce a figure analogous to Kroch's Figure 6.
In replicating the results, make sure that your script is consistent with the following criteria:
1. Since be and have exhibit no change with regard to do support in his data, Ellegård omitted them from consideration.
2. Ellegård 1953:199 identifies a class of verbs that he calls the know class. Sentences with verbs from the know class strongly disfavor do support, and Ellegård therefore omits them from his study (see the coding conventions for details).
3. Sentences containing main verb do are also omitted from Ellegård's study on the grounds that they strongly disfavor do support.
4. Ellegård reports a disfavoring effect of intransitivity on the rate of do support. More recent work (notably by Aaron Ecay, an alum of this class) shows that this effect is actually due to unaccusativity. Since Ellegård was unaware of unaccusativity, you should ignore it for the purposes of replicating his results and simply divide all relevant verbs (in other words, the ones that aren't be, have, main verb do, or members of the know class) into transitive and intransitive.
5. Exclude the "weird" genres Y and Z.

Troubleshooting

The preferences on your text editor should be set as "primitively" as possible.
- For instance, in TextEdit, under "New Document", set the "Format" option to "Plain text" (not to "Rich text").
- Under "Open and Save", deselect all button options.
- Under "Plain Text File Encoding", select UTF-8 or UTF-16.
The names of your shell scripts should contain no extensions (no .txt, .rtf, etc.).
The commands in the assignment assume that you are using the tcsh shell. The default shell on most systems seems to be a slightly different shell, the bash shell. You can find out which shell you are running by typing
```
    echo $SHELL
    
```
If typing the simple filenames to run the shell scripts gives an error message, chances are that the bash shell doesn't know how or where to find your command. Help it out by prefacing the command you are trying to run with . (which explicitly tells the shell to look in the current directory) or with bash.
```
    batchGrep
    
```
try one of the following two options:
```
    ./batchGrep

    bash batchGrep
    
```
There is no space in the first option, but there is a space after bash in the second option.