How to compare tagged training files to gold standard

$UT=/home/migration/other/MIDENG/PPCMBE/ut

copy gold standard file into training directory, taking care not to overwrite training files
```
mv $file.lex $file-training.lex

cp $SOMEDIR/$file.lex $file-gold.lex
```
deal with EOS in training file (and possibly also in gold standard)
normalize the gold standard file (delete comments and incorrect tags, put one word/tag pair per line, omit blank lines)
```
cat $file-gold.lex | $UT/clean-up-lex | tr -s ' ' '\012' | grep . > LHS
```

repeat for training file

cat $file-training.lex | $UT/clean-up-lex | tr -s ' ' '\012' | grep . > RHS

using diff --ignore-all-space --left-column --side-by-side (already aliased to diffils), compare the two versions
```
diffils LHS RHS > DIFF
```
diff uses the symbols < and > to indicate material that is missing from one version or the other. There should be no discrepancies of this type.
The existing discrepancies should be restricted to ones indicated by the pipe symbol ("|").
Any instances where the gold standard is wrong should be marked for future reference (say, by affixing three asterisks to the pipe symbol or somewhere in the line).