Troubleshooting

• CorpusSearch
• Fonts
  • General information
    • Set font encoding
    • Identify characters
    • Replace characters
  • Problems
    • Script doesn't handle accented characters
    • UTF-8 encoded file displays in emacs, but is corrupted in TextEdit
• Parser
  • Error message about job control
  • Parser outputs empty file

• If CorpusSearch breaks on an input file, the error message reports the line number where (or near where) things went wrong in the input file and also what CorpusSearch thinks the problem is.

• Missing wrapper parens. Wrapper parens are the unlabelled pair of parens that enclose each token (following the Penn Treebank convention), as in (1a). If the wrapper parens are missing, the token will look like (1b), and you will have to add the relevant parens. See Step 9.2 of the parser-mode documentation.

  (1) a.  ( (IP-MAT ...
          ))
  
      b.  (IP-MAT ...
          )
  

• Mismatched parens. Mismatched parens are generally the result of editing a parsed file in ordinary emacs rather than in parser-mode emacs or CorpusDraw.
  • In order to find mismatched parens, use the command forward-sexp, which starts from some left paren and jumps to the corresponding right paren. You can invoke the command on the mode line with M-x forward-sexp, or you can bind the command to something like C-x C-f in your .emacs file. In what follows, we assume the latter.

  • You can repeat forward-sexp arbitrarily many times by prefacing it with C-u NUM, where NUM is the desired number of repetitions.

    C-u 400 C-x C-f
    

    A command like the one just above will either go to completion or result in an error message. If the latter, you know that the mismatched paren is somewhere between where you started the command and the expected end.

  • Continued use of C-x C-f will allow you to zero in on the mismatched parens that are the problem.

  • CorpusSearch sometimes erroneously reports mismatched parens in connection with certain terminals (notably, \" (double quote). To further aggravate matters, forward-sexp works differently in ordinary emacs and in parser-mode emacs in these cases, treating the offending character on a par with an open paren in the one case, but not the other. A workaround is to replace the offending character by something like <double-quote>.

• CorpusSearch helpfully breaks on various ill-formed input sequences, including:

  (2) a.  ( (IP-MAT (NP-SBJ (PRO$ My) (N neighbor))
                    (VBD told)
                    me					← bare word (missing preterminal over terminal)
            (. .)))
  
      b.  ( (IP-MAT (NP-SBJ (PRO$ My) (N neighbor))
                    (VBD told)
                    (NP-OB2 (PRO m e))			← terminal contains space
            (. .)))
  
      c.  ( (IP-MAT (NP-SBJ (PRO$ My			← preterminal isn't unary-branching
                                  (N neighbor)))
                      
                    (VBD told)
                    (NP-OB2 (PRO me)))
            (. .))
  

• Set font encoding
  • In emacs
    • Make sure that font encoding in xml header is set to UTF-8.
    • Open new buffer.
    • Using M-x prefer-coding-system, set font encoding to utf-8.
    • Insert desired data in empty file with Control-x i.
  • In Textedit
    • Deprecated. Textedit allows opening and saving files in different font encodings. It is possible to add font encodings to the options in the popup menus with the customized list.
• Identify characters
  Control-x = (or Meta-x what-cursor-position)

• Replace characters
  • Use of TextEdit is deprecated.
  • Open text (UTF-8) file with TextEdit.
  • Convert it to RTF format (cmd-cap-T).
  • Save it as an RTF file.
  • Open the RTF file with Word.
  • Disable automatic corrections.
  • Search and replace.
    • ordinary underline: {U0057}
    • low combining underline: {U0332}
  • Save the results as an RTF file.
  • Open the modified RTF file with TextEdit.
  • Convert it back to a text file (cmd-cap-T).
  • Save it as a text (UTF-8) file.

• Script doesn't handle accented characters

  Some scripts (like the one that restores the subtags in the Cordial corpus) don't handle accented characters (regardless of the font encoding). The workaround is to translate the accented characters into the corresponding html character entities, to run the script on the resulting ASCII text file, and to restore the accented characters in the output. The relevant translation scripts are utf8-html and html-utf8.

• UTF-8 encoded file displays in emacs, but is corrupted in TextEdit

  Use of TextEdit is deprecated. If TextEdit open preferences are set to UTF-8 and it encounters a non-UTF-8 character in the file, TextEdit either silently disregards the non-UTF-8 characters or refuses to open files containing them. Emacs is more helpful because it displays such characters so that they can be identified and then replaced, deleted, or otherwise processed as needed.

  • In emacs, check for spurious characters or character sequences.
  • Delete or replace as desired.
  • If file still refuses to open in TextEdit, open file in emacs.
  • In emacs, save the entire file to the "kill ring". To do so, go to beginning of file and set the mark (generally with Control-[space]). Then go to the end of the file, and "kill" the file with Control-w.
  • Paste the "killed" file into a TextEdit buffer with "Paste" or Apple-v.