Repair/RepairCompiler/JLex/Main.java

   1 /**************************************************************
   2   JLex: A Lexical Analyzer Generator for Java(TM)
   3   Written by Elliot Berk. Copyright 1996.
   4   Contact at ejberk@princeton.edu.
   5
   6   Version 1.2.4, 7/24/99, [C. Scott Ananian]
   7    Correct the parsing of '-' in character classes, closing active
   8      bug #1.  Behaviour follows egrep: leading and trailing dashes in
   9      a character class lose their special meaning, so [-+] and [+-] do
  10      what you would expect them to.
  11    New %ignorecase directive for generating case-insensitive lexers by
  12      expanding matched character classes in a unicode-friendly way.
  13    Handle unmatched braces in quoted strings or comments within
  14      action code blocks.
  15    Fixed input lexer to allow whitespace in character classes, closing
  16      active bug #9.  Whitespace in quotes had been previously fixed.
  17    Made Yylex.YYEOF and %yyeof work like the manual says they should.
  18
  19   Version 1.2.3, 6/26/97, [Raimondas Lencevicius]
  20    Fixed the yy_nxt[][] assignment that has generated huge code
  21    exceeding 64K method size limit. Now the assignment
  22    is handled by unpacking a string encoding of integer array.
  23    To achieve that, added
  24    "private int [][] unpackFromString(int size1, int size2, String st)"
  25    function and coded the yy_nxt[][] values into a string
  26    by printing integers into a string and representing
  27    integer sequences as "value:length" pairs.
  28    Improvement: generated .java file reduced 2 times, .class file
  29      reduced 6 times for sample grammar. No 64K errors.
  30    Possible negatives: Some editors and OSs may not be able to handle
  31      the huge one-line generated string. String unpacking may be slower
  32      than direct array initialization.
  33
  34   Version 1.2.2, 10/24/97, [Martin Dirichs]
  35   Notes:
  36     Changed yy_instream to yy_reader of type BufferedReader. This reflects
  37      the improvements in the JDK 1.1 concerning InputStreams. As a
  38      consequence, changed yy_buffer from byte[] to char[].
  39      The lexer can now be initialized with either an InputStream
  40      or a Reader. A third, private constructor is called by the other
  41      two to execute user specified constructor code.
  42
  43   Version 1.2.1, 9/15/97 [A. Appel]
  44    Fixed bugs 6 (character codes > 127) and 10 (deprecated String constructor).
  45
  46   Version 1.2, 5/5/97, [Elliot Berk]
  47   Notes:
  48     Simply changed the name from JavaLex to JLex.  No other changes.
  49
  50   Version 1.1.5, 2/25/97, [Elliot Berk]
  51   Notes:
  52     Simple optimization to the creation of the source files.
  53      Added a BufferedOutputStream in the creation of the DataOutputStream
  54      field m_outstream of the class CLexGen.  This helps performance by
  55      doing some buffering, and was suggested by Max Hailperin,
  56      Associate Professor of Computer Science, Gustavus Adolphus College.
  57
  58   Version 1.1.4, 12/12/96, [Elliot Berk]
  59   Notes:
  60     Added %public directive to make generated class public.
  61
  62   Version 1.1.3, 12/11/96, [Elliot Berk]
  63   Notes:
  64     Converted assertion failure on invalid character class
  65      when a dash '-' is not preceded with a start-of-range character.
  66      Converted this into parse error E_DASH.
  67
  68   Version 1.1.2, October 30, 1996 [Elliot Berk]
  69     Fixed BitSet bugs by installing a BitSet class of my own,
  70      called JavaLexBitSet.  Fixed support for '\r', non-UNIX
  71      sequences.  Added try/catch block around lexer generation
  72      in main routine to moderate error information presented
  73      to user.  Fixed macro expansion, so that macros following
  74      quotes are expanded correctly in regular expressions.
  75      Fixed dynamic reallocation of accept action buffers.
  76
  77   Version 1.1.1, September 3, 1996 [Andrew Appel]
  78     Made the class "Main" instead of "JavaLex",
  79      improved the installation instructions to reflect this.
  80
  81   Version 1.1, August 15, 1996  [Andrew Appel]
  82     Made yychar, yyline, yytext global to the lexer so that
  83      auxiliary functions can access them.
  84
  85 /***************************************************************
  86   Package Declaration
  87   **************************************************************/
  88 package JLex;
  89
  90 /***************************************************************
  91   Imported Packages
  92   **************************************************************/
  93 import java.io.InputStream;
  94 import java.io.OutputStream;
  95 import java.io.FileInputStream;
  96 import java.io.FileOutputStream;
  97 import java.io.DataOutputStream;
  98 import java.io.BufferedOutputStream;
  99
 100 import java.lang.System;
 101 import java.lang.Integer;
 102 import java.lang.Character;
 103
 104 import java.util.Enumeration;
 105 import java.util.Stack;
 106 import java.util.Hashtable;
 107 import java.util.Vector;
 108
 109 /******************************
 110   Questions:
 111   2) How should I use the Java package system
 112   to make my tool more modularized and
 113   coherent?
 114
 115   Unimplemented:
 116   !) Fix BitSet issues -- expand only when necessary.
 117   2) Repeated accept rules.
 118   6) Clean up the CAlloc class and use buffered
 119   allocation.
 120   7) Reading and writing in Unicode.
 121   Use DataInputStream and DataOutputStream,
 122   along with writeUTF???
 123   8) Fix or at least annotate ^x bug.
 124   9) Add to spec about extending character set.
 125   10) making sure newlines are handled correctly
 126 S  and consistly with the m_unix flag
 127   11) m_verbose -- what should be done with it?
 128   12) turn lexical analyzer into a coherent
 129   Java package
 130   13) turn lexical analyzer generator into a
 131   coherent Java package
 132   16) pretty up generated code
 133   17) make it possible to have white space in
 134   regular expressions
 135   18) clean up all of the class files the lexer
 136   generator produces when it is compiled,
 137   and reduce this number in some way.
 138   24) character format to and from file: writeup
 139   and implementation
 140   25) Debug by testing all arcane regular expression cases.
 141   26) Look for and fix all UNDONE comments below.
 142   27) Fix package system.
 143   28) Clean up unnecessary classes.
 144   *****************************/
 145
 146 /***************************************************************
 147   Class: CSpec
 148  **************************************************************/
 149 class CSpec
 150 {
 151   /***************************************************************
 152     Member Variables
 153     **************************************************************/
 154
 155   /* Lexical States. */
 156   Hashtable m_states; /* Hashtable taking state indices (Integer)
 157                          to state name (String). */
 158
 159   /* Regular Expression Macros. */
 160   Hashtable m_macros; /* Hashtable taking macro name (String)
 161                                 to corresponding char buffer that
 162                                 holds macro definition. */
 163
 164   /* NFA Machine. */
 165   CNfa m_nfa_start; /* Start state of NFA machine. */
 166   Vector m_nfa_states; /* Vector of states, with index
 167                                  corresponding to label. */
 168
 169   Vector m_state_rules[]; /* An array of Vectors of Integers.
 170                                     The ith Vector represents the lexical state
 171                                     with index i.  The contents of the ith
 172                                     Vector are the indices of the NFA start
 173                                     states that can be matched while in
 174                                     the ith lexical state. */
 175
 176
 177   int m_state_dtrans[];
 178
 179   /* DFA Machine. */
 180   Vector m_dfa_states; /* Vector of states, with index
 181                                  corresponding to label. */
 182   Hashtable m_dfa_sets; /* Hashtable taking set of NFA states
 183                                   to corresponding DFA state,
 184                                   if the latter exists. */
 185
 186   /* Accept States and Corresponding Anchors. */
 187   Vector m_accept_vector;
 188   int m_anchor_array[];
 189
 190   /* Transition Table. */
 191   Vector m_dtrans_vector;
 192   int m_dtrans_ncols;
 193   int m_row_map[];
 194   int m_col_map[];
 195
 196   /* Regular expression token variables. */
 197   int m_current_token;
 198   char m_lexeme;
 199   boolean m_in_quote;
 200   boolean m_in_ccl;
 201
 202   /* Verbose execution flag. */
 203   boolean m_verbose;
 204
 205   /* JLex directives flags. */
 206   boolean m_integer_type;
 207   boolean m_intwrap_type;
 208   boolean m_yyeof;
 209   boolean m_count_chars;
 210   boolean m_count_lines;
 211   boolean m_cup_compatible;
 212   boolean m_unix;
 213   boolean m_public;
 214   boolean m_ignorecase;
 215
 216   char m_init_code[];
 217   int m_init_read;
 218
 219   char m_init_throw_code[];
 220   int m_init_throw_read;
 221
 222   char m_class_code[];
 223   int m_class_read;
 224
 225   char m_eof_code[];
 226   int m_eof_read;
 227
 228   char m_eof_value_code[];
 229   int m_eof_value_read;
 230
 231   char m_eof_throw_code[];
 232   int m_eof_throw_read;
 233
 234   char m_yylex_throw_code[];
 235   int m_yylex_throw_read;
 236
 237   /* Class, function, type names. */
 238   char m_class_name[] = {
 239     'Y', 'y', 'l',
 240     'e', 'x'
 241     };
 242   char m_implements_name[] = {};
 243   char m_function_name[] = {
 244     'y', 'y', 'l',
 245     'e', 'x'
 246     };
 247   char m_type_name[] = {
 248     'Y', 'y', 't',
 249     'o', 'k', 'e',
 250     'n'
 251     };
 252
 253   /* Lexical Generator. */
 254   private CLexGen m_lexGen;
 255
 256   /***************************************************************
 257     Constants
 258     ***********************************************************/
 259   static final int NONE = 0;
 260   static final int START = 1;
 261   static final int END = 2;
 262
 263   /***************************************************************
 264     Function: CSpec
 265     Description: Constructor.
 266     **************************************************************/
 267   CSpec
 268     (
 269      CLexGen lexGen
 270      )
 271       {
 272         m_lexGen = lexGen;
 273
 274         /* Initialize regular expression token variables. */
 275         m_current_token = m_lexGen.EOS;
 276         m_lexeme = '\0';
 277         m_in_quote = false;
 278         m_in_ccl = false;
 279
 280         /* Initialize hashtable for lexer states. */
 281         m_states = new Hashtable();
 282         m_states.put(new String("YYINITIAL"),new Integer(m_states.size()));
 283
 284         /* Initialize hashtable for lexical macros. */
 285         m_macros = new Hashtable();
 286
 287         /* Initialize variables for lexer options. */
 288         m_integer_type = false;
 289         m_intwrap_type = false;
 290         m_count_lines = false;
 291         m_count_chars = false;
 292         m_cup_compatible = false;
 293         m_unix = true;
 294         m_public = false;
 295         m_yyeof = false;
 296         m_ignorecase = false;
 297
 298         /* Initialize variables for JLex runtime options. */
 299         m_verbose = true;
 300
 301         m_nfa_start = null;
 302         m_nfa_states = new Vector();
 303
 304         m_dfa_states = new Vector();
 305         m_dfa_sets = new Hashtable();
 306
 307         m_dtrans_vector = new Vector();
 308         m_dtrans_ncols = CUtility.MAX_SEVEN_BIT + 1;
 309         m_row_map = null;
 310         m_col_map = null;
 311
 312         m_accept_vector = null;
 313         m_anchor_array = null;
 314
 315         m_init_code = null;
 316         m_init_read = 0;
 317
 318         m_init_throw_code = null;
 319         m_init_throw_read = 0;
 320
 321         m_yylex_throw_code = null;
 322         m_yylex_throw_read = 0;
 323
 324         m_class_code = null;
 325         m_class_read = 0;
 326
 327         m_eof_code = null;
 328         m_eof_read = 0;
 329
 330         m_eof_value_code = null;
 331         m_eof_value_read = 0;
 332
 333         m_eof_throw_code = null;
 334         m_eof_throw_read = 0;
 335
 336         m_state_dtrans = null;
 337
 338         m_state_rules = null;
 339       }
 340 }
 341
 342 /***************************************************************
 343   Class: CEmit
 344   **************************************************************/
 345 class CEmit
 346 {
 347   /***************************************************************
 348     Member Variables
 349     **************************************************************/
 350   private CSpec m_spec;
 351   private DataOutputStream m_outstream;
 352
 353   /***************************************************************
 354     Constants: Anchor Types
 355     **************************************************************/
 356   private final int START = 1;
 357   private final int END = 2;
 358   private final int NONE = 4;
 359
 360   /***************************************************************
 361     Constants
 362     **************************************************************/
 363   private final boolean EDBG = true;
 364   private final boolean NOT_EDBG = false;
 365
 366   /***************************************************************
 367     Function: CEmit
 368     Description: Constructor.
 369     **************************************************************/
 370   CEmit
 371     (
 372      )
 373       {
 374         reset();
 375       }
 376
 377   /***************************************************************
 378     Function: reset
 379     Description: Clears member variables.
 380     **************************************************************/
 381   private void reset
 382     (
 383      )
 384       {
 385         m_spec = null;
 386         m_outstream = null;
 387       }
 388
 389   /***************************************************************
 390     Function: set
 391     Description: Initializes member variables.
 392     **************************************************************/
 393   private void set
 394     (
 395      CSpec spec,
 396      OutputStream outstream
 397      )
 398       {
 399         if (CUtility.DEBUG)
 400           {
 401             CUtility.assert(null != spec);
 402             CUtility.assert(null != outstream);
 403           }
 404
 405         m_spec = spec;
 406         m_outstream = new DataOutputStream(outstream);
 407       }
 408
 409   /***************************************************************
 410     Function: emit_imports
 411     Description: Emits import packages at top of
 412     generated source file.
 413     **************************************************************/
 414   /*void emit_imports
 415     (
 416      CSpec spec,
 417      OutputStream outstream
 418      )
 419       throws java.io.IOException
 420         {
 421           set(spec,outstream);
 422
 423           if (CUtility.DEBUG)
 424             {
 425               CUtility.assert(null != m_spec);
 426               CUtility.assert(null != m_outstream);
 427             }*/
 428
 429           /*m_outstream.writeBytes("import java.lang.String;\n");
 430           m_outstream.writeBytes("import java.lang.System;\n");
 431           m_outstream.writeBytes("import java.io.BufferedReader;\n");
 432           m_outstream.writeBytes("import java.io.InputStream;\n");*/
 433         /*
 434           reset();
 435         }*/
 436
 437   /***************************************************************
 438     Function: print_details
 439     Description: Debugging output.
 440     **************************************************************/
 441   private void print_details
 442     (
 443      )
 444       {
 445         int i;
 446         int j;
 447         int next;
 448         int state;
 449         CDTrans dtrans;
 450         CAccept accept;
 451         boolean tr;
 452
 453         System.out.println("---------------------- Transition Table "
 454                            + "----------------------");
 455
 456         for (i = 0; i < m_spec.m_row_map.length; ++i)
 457           {
 458             System.out.print("State " + i);
 459
 460             accept = (CAccept) m_spec.m_accept_vector.elementAt(i);
 461             if (null == accept)
 462               {
 463                 System.out.println(" [nonaccepting]");
 464               }
 465             else
 466               {
 467                 System.out.println(" [accepting, line "
 468                                  + accept.m_line_number
 469                                  + " <"
 470                                  + (new java.lang.String(accept.m_action,0,
 471                                                accept.m_action_read))
 472                                  + ">]");
 473               }
 474             dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(m_spec.m_row_map[i]);
 475
 476             tr = false;
 477             state = dtrans.m_dtrans[m_spec.m_col_map[0]];
 478             if (CDTrans.F != state)
 479               {
 480                 tr = true;
 481                 System.out.print("\tgoto " + state + " on [" + ((char) 0));
 482               }
 483             for (j = 1; j < m_spec.m_dtrans_ncols; ++j)
 484               {
 485                 next = dtrans.m_dtrans[m_spec.m_col_map[j]];
 486                 if (state == next)
 487                   {
 488                     if (CDTrans.F != state)
 489                       {
 490                         System.out.print((char) j);
 491                       }
 492                   }
 493                 else
 494                   {
 495                     state = next;
 496                     if (tr)
 497                       {
 498                         System.out.println("]");
 499                         tr = false;
 500                       }
 501                     if (CDTrans.F != state)
 502                       {
 503                         tr = true;
 504                         System.out.print("\tgoto " + state + " on [" + ((char) j));
 505                       }
 506                   }
 507               }
 508             if (tr)
 509               {
 510                 System.out.println("]");
 511               }
 512           }
 513
 514         System.out.println("---------------------- Transition Table "
 515                            + "----------------------");
 516       }
 517
 518   /***************************************************************
 519     Function: emit
 520     Description: High-level access function to module.
 521     **************************************************************/
 522   void emit
 523     (
 524      CSpec spec,
 525      OutputStream outstream
 526      )
 527       throws java.io.IOException
 528         {
 529           set(spec,outstream);
 530
 531           if (CUtility.DEBUG)
 532             {
 533               CUtility.assert(null != m_spec);
 534               CUtility.assert(null != m_outstream);
 535             }
 536
 537           if (CUtility.OLD_DEBUG) {
 538             print_details();
 539           }
 540
 541           emit_header();
 542           emit_construct();
 543           emit_helpers();
 544           emit_driver();
 545           emit_footer();
 546
 547           reset();
 548         }
 549
 550   /***************************************************************
 551     Function: emit_construct
 552     Description: Emits constructor, member variables,
 553     and constants.
 554     **************************************************************/
 555   private void emit_construct
 556     (
 557      )
 558       throws java.io.IOException
 559         {
 560           if (CUtility.DEBUG)
 561           {
 562             CUtility.assert(null != m_spec);
 563             CUtility.assert(null != m_outstream);
 564           }
 565
 566           /* Constants */
 567           m_outstream.writeBytes("\tprivate final int YY_BUFFER_SIZE = 512;\n");
 568
 569           m_outstream.writeBytes("\tprivate final int YY_F = -1;\n");
 570           m_outstream.writeBytes("\tprivate final int YY_NO_STATE = -1;\n");
 571
 572           m_outstream.writeBytes("\tprivate final int YY_NOT_ACCEPT = 0;\n");
 573           m_outstream.writeBytes("\tprivate final int YY_START = 1;\n");
 574           m_outstream.writeBytes("\tprivate final int YY_END = 2;\n");
 575           m_outstream.writeBytes("\tprivate final int YY_NO_ANCHOR = 4;\n");
 576
 577           m_outstream.writeBytes("\tprivate final char YY_EOF = \'\\uFFFF\';\n");
 578           if (m_spec.m_integer_type || true == m_spec.m_yyeof)
 579             m_outstream.writeBytes("\tpublic final int YYEOF = -1;\n");
 580
 581           /* User specified class code. */
 582           if (null != m_spec.m_class_code)
 583             {
 584               m_outstream.writeBytes(new String(m_spec.m_class_code,0,
 585                                                 m_spec.m_class_read));
 586             }
 587
 588           /* Member Variables */
 589           m_outstream.writeBytes("\tprivate java.io.BufferedReader yy_reader;\n");
 590           m_outstream.writeBytes("\tprivate int yy_buffer_index;\n");
 591           m_outstream.writeBytes("\tprivate int yy_buffer_read;\n");
 592           m_outstream.writeBytes("\tprivate int yy_buffer_start;\n");
 593           m_outstream.writeBytes("\tprivate int yy_buffer_end;\n");
 594           m_outstream.writeBytes("\tprivate char yy_buffer[];\n");
 595           if (m_spec.m_count_chars)
 596             {
 597               m_outstream.writeBytes("\tprivate int yychar;\n");
 598             }
 599           if (m_spec.m_count_lines)
 600             {
 601               m_outstream.writeBytes("\tprivate int yyline;\n");
 602             }
 603           m_outstream.writeBytes("\tprivate int yy_lexical_state;\n");
 604           /*if (m_spec.m_count_lines || true == m_spec.m_count_chars)
 605             {
 606               m_outstream.writeBytes("\tprivate int yy_buffer_prev_start;\n");
 607             }*/
 608           m_outstream.writeBytes("\n");
 609
 610
 611           /* Function: first constructor (Reader) */
 612           m_outstream.writeBytes("\t");
 613           if (true == m_spec.m_public) {
 614             m_outstream.writeBytes("public ");
 615           }
 616           m_outstream.writeBytes(new String(m_spec.m_class_name));
 617           m_outstream.writeBytes(" (java.io.Reader reader)");
 618
 619           if (null != m_spec.m_init_throw_code)
 620             {
 621               m_outstream.writeBytes("\n");
 622               m_outstream.writeBytes("\t\tthrows ");
 623               m_outstream.writeBytes(new String(m_spec.m_init_throw_code,0,
 624                                                 m_spec.m_init_throw_read));
 625               m_outstream.writeBytes("\n\t\t{\n");
 626             }
 627           else
 628             {
 629               m_outstream.writeBytes(" {\n");
 630             }
 631
 632           m_outstream.writeBytes("\t\tthis ();\n");
 633           m_outstream.writeBytes("\t\tif (null == reader) {\n");
 634           m_outstream.writeBytes("\t\t\tthrow (new Error(\"Error: Bad input "
 635                                  + "stream initializer.\"));\n");
 636           m_outstream.writeBytes("\t\t}\n");
 637           m_outstream.writeBytes("\t\tyy_reader = new java.io.BufferedReader(reader);\n");
 638           m_outstream.writeBytes("\t}\n\n");
 639
 640
 641           /* Function: second constructor (InputStream) */
 642           m_outstream.writeBytes("\t");
 643           if (true == m_spec.m_public) {
 644             m_outstream.writeBytes("public ");
 645           }
 646           m_outstream.writeBytes(new String(m_spec.m_class_name));
 647           m_outstream.writeBytes(" (java.io.InputStream instream)");
 648
 649           if (null != m_spec.m_init_throw_code)
 650             {
 651               m_outstream.writeBytes("\n");
 652               m_outstream.writeBytes("\t\tthrows ");
 653               m_outstream.writeBytes(new String(m_spec.m_init_throw_code,0,
 654                                                 m_spec.m_init_throw_read));
 655               m_outstream.writeBytes("\n\t\t{\n");
 656             }
 657           else
 658             {
 659               m_outstream.writeBytes(" {\n");
 660             }
 661
 662           m_outstream.writeBytes("\t\tthis ();\n");
 663           m_outstream.writeBytes("\t\tif (null == instream) {\n");
 664           m_outstream.writeBytes("\t\t\tthrow (new Error(\"Error: Bad input "
 665                                  + "stream initializer.\"));\n");
 666           m_outstream.writeBytes("\t\t}\n");
 667           m_outstream.writeBytes("\t\tyy_reader = new java.io.BufferedReader(new java.io.InputStreamReader(instream));\n");
 668           m_outstream.writeBytes("\t}\n\n");
 669
 670
 671           /* Function: third, private constructor - only for internal use */
 672           m_outstream.writeBytes("\tprivate ");
 673           m_outstream.writeBytes(new String(m_spec.m_class_name));
 674           m_outstream.writeBytes(" ()");
 675
 676           if (null != m_spec.m_init_throw_code)
 677             {
 678               m_outstream.writeBytes("\n");
 679               m_outstream.writeBytes("\t\tthrows ");
 680               m_outstream.writeBytes(new String(m_spec.m_init_throw_code,0,
 681                                                 m_spec.m_init_throw_read));
 682               m_outstream.writeBytes("\n\t\t{\n");
 683             }
 684           else
 685             {
 686               m_outstream.writeBytes(" {\n");
 687             }
 688
 689           m_outstream.writeBytes("\t\tyy_buffer = new char[YY_BUFFER_SIZE];\n");
 690           m_outstream.writeBytes("\t\tyy_buffer_read = 0;\n");
 691           m_outstream.writeBytes("\t\tyy_buffer_index = 0;\n");
 692           m_outstream.writeBytes("\t\tyy_buffer_start = 0;\n");
 693           m_outstream.writeBytes("\t\tyy_buffer_end = 0;\n");
 694           if (m_spec.m_count_chars)
 695             {
 696               m_outstream.writeBytes("\t\tyychar = 0;\n");
 697             }
 698           if (m_spec.m_count_lines)
 699             {
 700               m_outstream.writeBytes("\t\tyyline = 0;\n");
 701             }
 702           m_outstream.writeBytes("\t\tyy_lexical_state = YYINITIAL;\n");
 703           /*if (m_spec.m_count_lines || true == m_spec.m_count_chars)
 704             {
 705               m_outstream.writeBytes("\t\tyy_buffer_prev_start = 0;\n");
 706             }*/
 707
 708           /* User specified constructor code. */
 709           if (null != m_spec.m_init_code)
 710             {
 711               m_outstream.writeBytes(new String(m_spec.m_init_code,0,
 712                                                 m_spec.m_init_read));
 713             }
 714
 715           m_outstream.writeBytes("\t}\n\n");
 716
 717         }
 718
 719   /***************************************************************
 720     Function: emit_states
 721     Description: Emits constants that serve as lexical states,
 722     including YYINITIAL.
 723     **************************************************************/
 724   private void emit_states
 725     (
 726      )
 727       throws java.io.IOException
 728         {
 729           Enumeration states;
 730           String state;
 731           int index;
 732
 733           states = m_spec.m_states.keys();
 734           /*index = 0;*/
 735           while (states.hasMoreElements())
 736             {
 737               state = (String) states.nextElement();
 738
 739               if (CUtility.DEBUG)
 740                 {
 741                   CUtility.assert(null != state);
 742                 }
 743
 744               m_outstream.writeBytes("\tprivate final int "
 745                                      + state
 746                                      + " = "
 747                                      + (m_spec.m_states.get(state)).toString()
 748                                      + ";\n");
 749               /*++index;*/
 750             }
 751
 752           m_outstream.writeBytes("\tprivate final int yy_state_dtrans[] = {\n");
 753           for (index = 0; index < m_spec.m_state_dtrans.length; ++index)
 754             {
 755               m_outstream.writeBytes("\t\t" + m_spec.m_state_dtrans[index]);
 756               if (index < m_spec.m_state_dtrans.length - 1)
 757                 {
 758                   m_outstream.writeBytes(",\n");
 759                 }
 760               else
 761                 {
 762                   m_outstream.writeBytes("\n");
 763                 }
 764             }
 765           m_outstream.writeBytes("\t};\n");
 766         }
 767
 768   /***************************************************************
 769     Function: emit_helpers
 770     Description: Emits helper functions, particularly
 771     error handling and input buffering.
 772     **************************************************************/
 773   private void emit_helpers
 774     (
 775      )
 776       throws java.io.IOException
 777       {
 778         if (CUtility.DEBUG)
 779           {
 780             CUtility.assert(null != m_spec);
 781             CUtility.assert(null != m_outstream);
 782           }
 783
 784         /* Function: yy_do_eof */
 785         m_outstream.writeBytes("\tprivate boolean yy_eof_done = false;\n");
 786         if (null != m_spec.m_eof_code)
 787           {
 788             m_outstream.writeBytes("\tprivate void yy_do_eof ()");
 789
 790             if (null != m_spec.m_eof_throw_code)
 791               {
 792                 m_outstream.writeBytes("\n");
 793                 m_outstream.writeBytes("\t\tthrows ");
 794                 m_outstream.writeBytes(new String(m_spec.m_eof_throw_code,0,
 795                                                   m_spec.m_eof_throw_read));
 796                 m_outstream.writeBytes("\n\t\t{\n");
 797               }
 798             else
 799               {
 800                 m_outstream.writeBytes(" {\n");
 801               }
 802
 803             m_outstream.writeBytes("\t\tif (false == yy_eof_done) {\n");
 804             m_outstream.writeBytes(new String(m_spec.m_eof_code,0,
 805                                               m_spec.m_eof_read));
 806             m_outstream.writeBytes("\t\t}\n");
 807             m_outstream.writeBytes("\t\tyy_eof_done = true;\n");
 808             m_outstream.writeBytes("\t}\n");
 809           }
 810
 811         emit_states();
 812
 813         /* Function: yybegin */
 814         m_outstream.writeBytes("\tprivate void yybegin (int state) {\n");
 815         m_outstream.writeBytes("\t\tyy_lexical_state = state;\n");
 816         m_outstream.writeBytes("\t}\n");
 817
 818         /* Function: yy_initial_dtrans */
 819         /*m_outstream.writeBytes("\tprivate int yy_initial_dtrans (int state) {\n");
 820         m_outstream.writeBytes("\t\treturn yy_state_dtrans[state];\n");
 821         m_outstream.writeBytes("\t}\n");*/
 822
 823         /* Function: yy_advance */
 824         m_outstream.writeBytes("\tprivate char yy_advance ()\n");
 825         m_outstream.writeBytes("\t\tthrows java.io.IOException {\n");
 826         /*m_outstream.writeBytes("\t\t{\n");*/
 827         m_outstream.writeBytes("\t\tint next_read;\n");
 828         m_outstream.writeBytes("\t\tint i;\n");
 829         m_outstream.writeBytes("\t\tint j;\n");
 830         m_outstream.writeBytes("\n");
 831
 832         m_outstream.writeBytes("\t\tif (yy_buffer_index < yy_buffer_read) {\n");
 833         m_outstream.writeBytes("\t\t\treturn yy_buffer[yy_buffer_index++];\n");
 834         /*m_outstream.writeBytes("\t\t\t++yy_buffer_index;\n");*/
 835         m_outstream.writeBytes("\t\t}\n");
 836         m_outstream.writeBytes("\n");
 837
 838         m_outstream.writeBytes("\t\tif (0 != yy_buffer_start) {\n");
 839         m_outstream.writeBytes("\t\t\ti = yy_buffer_start;\n");
 840         m_outstream.writeBytes("\t\t\tj = 0;\n");
 841         m_outstream.writeBytes("\t\t\twhile (i < yy_buffer_read) {\n");
 842         m_outstream.writeBytes("\t\t\t\tyy_buffer[j] = yy_buffer[i];\n");
 843         m_outstream.writeBytes("\t\t\t\t++i;\n");
 844         m_outstream.writeBytes("\t\t\t\t++j;\n");
 845         m_outstream.writeBytes("\t\t\t}\n");
 846         m_outstream.writeBytes("\t\t\tyy_buffer_end = yy_buffer_end - yy_buffer_start;\n");
 847         m_outstream.writeBytes("\t\t\tyy_buffer_start = 0;\n");
 848         m_outstream.writeBytes("\t\t\tyy_buffer_read = j;\n");
 849         m_outstream.writeBytes("\t\t\tyy_buffer_index = j;\n");
 850         m_outstream.writeBytes("\t\t\tnext_read = yy_reader.read(yy_buffer,\n");
 851         m_outstream.writeBytes("\t\t\t\t\tyy_buffer_read,\n");
 852         m_outstream.writeBytes("\t\t\t\t\tyy_buffer.length - yy_buffer_read);\n");
 853         m_outstream.writeBytes("\t\t\tif (-1 == next_read) {\n");
 854         m_outstream.writeBytes("\t\t\t\treturn YY_EOF;\n");
 855         m_outstream.writeBytes("\t\t\t}\n");
 856         m_outstream.writeBytes("\t\t\tyy_buffer_read = yy_buffer_read + next_read;\n");
 857         m_outstream.writeBytes("\t\t}\n");
 858         m_outstream.writeBytes("\n");
 859
 860         m_outstream.writeBytes("\t\twhile (yy_buffer_index >= yy_buffer_read) {\n");
 861         m_outstream.writeBytes("\t\t\tif (yy_buffer_index >= yy_buffer.length) {\n");
 862         m_outstream.writeBytes("\t\t\t\tyy_buffer = yy_double(yy_buffer);\n");
 863         m_outstream.writeBytes("\t\t\t}\n");
 864         m_outstream.writeBytes("\t\t\tnext_read = yy_reader.read(yy_buffer,\n");
 865         m_outstream.writeBytes("\t\t\t\t\tyy_buffer_read,\n");
 866         m_outstream.writeBytes("\t\t\t\t\tyy_buffer.length - yy_buffer_read);\n");
 867         m_outstream.writeBytes("\t\t\tif (-1 == next_read) {\n");
 868         m_outstream.writeBytes("\t\t\t\treturn YY_EOF;\n");
 869         m_outstream.writeBytes("\t\t\t}\n");
 870         m_outstream.writeBytes("\t\t\tyy_buffer_read = yy_buffer_read + next_read;\n");
 871         m_outstream.writeBytes("\t\t}\n");
 872
 873         m_outstream.writeBytes("\t\treturn yy_buffer[yy_buffer_index++];\n");
 874         m_outstream.writeBytes("\t}\n");
 875
 876         /* Function: yy_move_start */
 877         m_outstream.writeBytes("\tprivate void yy_move_start () {\n");
 878         if (m_spec.m_count_lines)
 879           {
 880             if (m_spec.m_unix)
 881               {
 882                 m_outstream.writeBytes("\t\tif ((byte) '\\n' "
 883                                        + "== yy_buffer[yy_buffer_start]) {\n");
 884               }
 885             else
 886               {
 887                 m_outstream.writeBytes("\t\tif ((byte) '\\n' "
 888                                        + "== yy_buffer[yy_buffer_start]\n");
 889                 m_outstream.writeBytes("\t\t\t|| (byte) '\\r' "
 890                                        + "== yy_buffer[yy_buffer_start]) {\n");
 891               }
 892             m_outstream.writeBytes("\t\t\t++yyline;\n");
 893             m_outstream.writeBytes("\t\t}\n");
 894           }
 895         if (m_spec.m_count_chars)
 896           {
 897             m_outstream.writeBytes("\t\t++yychar;\n");
 898           }
 899         m_outstream.writeBytes("\t\t++yy_buffer_start;\n");
 900         m_outstream.writeBytes("\t}\n");
 901
 902         /* Function: yy_pushback */
 903         m_outstream.writeBytes("\tprivate void yy_pushback () {\n");
 904         m_outstream.writeBytes("\t\t--yy_buffer_end;\n");
 905         m_outstream.writeBytes("\t}\n");
 906
 907         /* Function: yy_mark_start */
 908         m_outstream.writeBytes("\tprivate void yy_mark_start () {\n");
 909         if (m_spec.m_count_lines || true == m_spec.m_count_chars)
 910           {
 911             if (m_spec.m_count_lines)
 912               {
 913                 m_outstream.writeBytes("\t\tint i;\n");
 914                 m_outstream.writeBytes("\t\tfor (i = yy_buffer_start; "
 915                                        + "i < yy_buffer_index; ++i) {\n");
 916                 if (m_spec.m_unix)
 917                   {
 918                     m_outstream.writeBytes("\t\t\tif ((byte) '\\n' == yy_buffer[i]) {\n");
 919                   }
 920                 else
 921                   {
 922                     m_outstream.writeBytes("\t\t\tif ((byte) '\\n' == yy_buffer[i] || "
 923                                            + "(byte) '\\r' == yy_buffer[i]) {\n");
 924                   }
 925                 m_outstream.writeBytes("\t\t\t\t++yyline;\n");
 926                 m_outstream.writeBytes("\t\t\t}\n");
 927                 m_outstream.writeBytes("\t\t}\n");
 928               }
 929             if (m_spec.m_count_chars)
 930               {
 931                 m_outstream.writeBytes("\t\tyychar = yychar\n");
 932                 m_outstream.writeBytes("\t\t\t+ yy_buffer_index - yy_buffer_start;\n");
 933               }
 934           }
 935         m_outstream.writeBytes("\t\tyy_buffer_start = yy_buffer_index;\n");
 936         m_outstream.writeBytes("\t}\n");
 937
 938         /* Function: yy_mark_end */
 939         m_outstream.writeBytes("\tprivate void yy_mark_end () {\n");
 940         m_outstream.writeBytes("\t\tyy_buffer_end = yy_buffer_index;\n");
 941         m_outstream.writeBytes("\t}\n");
 942
 943         /* Function: yy_to_mark */
 944         m_outstream.writeBytes("\tprivate void yy_to_mark () {\n");
 945         m_outstream.writeBytes("\t\tyy_buffer_index = yy_buffer_end;\n");
 946         m_outstream.writeBytes("\t}\n");
 947
 948         /* Function: yytext */
 949         m_outstream.writeBytes("\tprivate java.lang.String yytext () {\n");
 950         m_outstream.writeBytes("\t\treturn (new java.lang.String(yy_buffer,\n");
 951         m_outstream.writeBytes("\t\t\tyy_buffer_start,\n");
 952         m_outstream.writeBytes("\t\t\tyy_buffer_end - yy_buffer_start));\n");
 953         m_outstream.writeBytes("\t}\n");
 954
 955         /* Function: yylength */
 956         m_outstream.writeBytes("\tprivate int yylength () {\n");
 957         m_outstream.writeBytes("\t\treturn yy_buffer_end - yy_buffer_start;\n");
 958         m_outstream.writeBytes("\t}\n");
 959
 960         /* Function: yy_double */
 961         m_outstream.writeBytes("\tprivate char[] yy_double (char buf[]) {\n");
 962         m_outstream.writeBytes("\t\tint i;\n\t\tchar newbuf[];\n");
 963         m_outstream.writeBytes("\t\tnewbuf = new char[2*buf.length];\n");
 964         m_outstream.writeBytes("\t\tfor (i = 0; i < buf.length; ++i) {\n");
 965         m_outstream.writeBytes("\t\t\tnewbuf[i] = buf[i];\n");
 966         m_outstream.writeBytes("\t\t}\n");
 967         m_outstream.writeBytes("\t\treturn newbuf;\n");
 968         m_outstream.writeBytes("\t}\n");
 969
 970         /* Function: yy_error */
 971         m_outstream.writeBytes("\tprivate final int YY_E_INTERNAL = 0;\n");
 972         m_outstream.writeBytes("\tprivate final int YY_E_MATCH = 1;\n");
 973         m_outstream.writeBytes("\tprivate java.lang.String yy_error_string[] = {\n");
 974         m_outstream.writeBytes("\t\t\"Error: Internal error.\\n\",\n");
 975         m_outstream.writeBytes("\t\t\"Error: Unmatched input.\\n\"\n");
 976         m_outstream.writeBytes("\t};\n");
 977         m_outstream.writeBytes("\tprivate void yy_error (int code,boolean fatal) {\n");
 978         m_outstream.writeBytes("\t\tjava.lang.System.out.print(yy_error_string[code]);\n");
 979         m_outstream.writeBytes("\t\tjava.lang.System.out.flush();\n");
 980         m_outstream.writeBytes("\t\tif (fatal) {\n");
 981         m_outstream.writeBytes("\t\t\tthrow new Error(\"Fatal Error.\\n\");\n");
 982         m_outstream.writeBytes("\t\t}\n");
 983         m_outstream.writeBytes("\t}\n");
 984
 985         /* Function: yy_next */
 986         /*m_outstream.writeBytes("\tprivate int yy_next (int current,char lookahead) {\n");
 987         m_outstream.writeBytes("\t\treturn yy_nxt[yy_rmap[current]][yy_cmap[lookahead]];\n");
 988         m_outstream.writeBytes("\t}\n");*/
 989
 990         /* Function: yy_accept */
 991         /*m_outstream.writeBytes("\tprivate int yy_accept (int current) {\n");
 992         m_outstream.writeBytes("\t\treturn yy_acpt[current];\n");
 993         m_outstream.writeBytes("\t}\n");*/
 994
 995
 996         // Function: private int [][] unpackFromString(int size1, int size2, String st)
 997         // Added 6/24/98 Raimondas Lencevicius
 998         // May be made more efficient by replacing String operations
 999         // Assumes correctly formed input String. Performs no error checking
1000 m_outstream.writeBytes("private int [][] unpackFromString(int size1, int size2, String st)\n");
1001 m_outstream.writeBytes("    {\n");
1002 m_outstream.writeBytes("      int colonIndex = -1;\n");
1003 m_outstream.writeBytes("      String lengthString;\n");
1004 m_outstream.writeBytes("      int sequenceLength = 0;\n");
1005 m_outstream.writeBytes("      int sequenceInteger = 0;\n");
1006
1007 m_outstream.writeBytes("      int commaIndex;\n");
1008 m_outstream.writeBytes("      String workString;\n");
1009
1010 m_outstream.writeBytes("      int res[][] = new int[size1][size2];\n");
1011 m_outstream.writeBytes("      for (int i= 0; i < size1; i++)\n");
1012 m_outstream.writeBytes("        for (int j= 0; j < size2; j++)\n");
1013 m_outstream.writeBytes("          {\n");
1014 m_outstream.writeBytes("            if (sequenceLength == 0) \n");
1015 m_outstream.writeBytes("              { \n");
1016 m_outstream.writeBytes("                commaIndex = st.indexOf(',');\n");
1017 m_outstream.writeBytes("                if (commaIndex == -1)\n");
1018 m_outstream.writeBytes("                  workString = st;\n");
1019 m_outstream.writeBytes("                else\n");
1020 m_outstream.writeBytes("                  workString = st.substring(0, commaIndex);\n");
1021 m_outstream.writeBytes("                st = st.substring(commaIndex+1);\n");
1022 m_outstream.writeBytes("                colonIndex = workString.indexOf(':');\n");
1023 m_outstream.writeBytes("                if (colonIndex == -1)\n");
1024 m_outstream.writeBytes("                  {\n");
1025 m_outstream.writeBytes("                    res[i][j] = Integer.parseInt(workString);\n");
1026 m_outstream.writeBytes("                  }\n");
1027 m_outstream.writeBytes("                else \n");
1028 m_outstream.writeBytes("                  {\n");
1029 m_outstream.writeBytes("                    lengthString = workString.substring(colonIndex+1);  \n");
1030 m_outstream.writeBytes("                    sequenceLength = Integer.parseInt(lengthString);\n");
1031 m_outstream.writeBytes("                    workString = workString.substring(0,colonIndex);\n");
1032 m_outstream.writeBytes("                    sequenceInteger = Integer.parseInt(workString);\n");
1033 m_outstream.writeBytes("                    res[i][j] = sequenceInteger;\n");
1034 m_outstream.writeBytes("                    sequenceLength--;\n");
1035 m_outstream.writeBytes("                  }\n");
1036 m_outstream.writeBytes("              }\n");
1037 m_outstream.writeBytes("            else \n");
1038 m_outstream.writeBytes("              {\n");
1039 m_outstream.writeBytes("                res[i][j] = sequenceInteger;\n");
1040 m_outstream.writeBytes("                sequenceLength--;\n");
1041 m_outstream.writeBytes("              }\n");
1042 m_outstream.writeBytes("          }\n");
1043 m_outstream.writeBytes("      return res;\n");
1044 m_outstream.writeBytes("    }\n");
1045
1046
1047
1048       }
1049
1050   /***************************************************************
1051     Function: emit_header
1052     Description: Emits class header.
1053     **************************************************************/
1054   private void emit_header
1055     (
1056      )
1057       throws java.io.IOException
1058       {
1059         if (CUtility.DEBUG)
1060           {
1061             CUtility.assert(null != m_spec);
1062             CUtility.assert(null != m_outstream);
1063           }
1064
1065         m_outstream.writeBytes("\n\n");
1066         if (true == m_spec.m_public) {
1067           m_outstream.writeBytes("public ");
1068         }
1069         m_outstream.writeBytes("class ");
1070         m_outstream.writeBytes(new String(m_spec.m_class_name,0,
1071                                           m_spec.m_class_name.length));
1072         if (m_spec.m_implements_name.length > 0) {
1073            m_outstream.writeBytes(" implements ");
1074            m_outstream.writeBytes(new String(m_spec.m_implements_name,0,
1075                                           m_spec.m_implements_name.length));
1076         }
1077         m_outstream.writeBytes(" {\n");
1078       }
1079
1080   /***************************************************************
1081     Function: emit_table
1082     Description: Emits transition table.
1083     **************************************************************/
1084   private void emit_table
1085     (
1086      )
1087       throws java.io.IOException
1088       {
1089         int i;
1090         int elem;
1091         int size;
1092         CDTrans dtrans;
1093         boolean is_start;
1094         boolean is_end;
1095         CAccept accept;
1096
1097         if (CUtility.DEBUG)
1098           {
1099             CUtility.assert(null != m_spec);
1100             CUtility.assert(null != m_outstream);
1101           }
1102
1103         m_outstream.writeBytes("\tprivate int yy_acpt[] = {\n");
1104         size = m_spec.m_accept_vector.size();
1105         for (elem = 0; elem < size; ++elem)
1106           {
1107             accept = (CAccept) m_spec.m_accept_vector.elementAt(elem);
1108
1109             if (null != accept)
1110               {
1111                 is_start = (0 != (m_spec.m_anchor_array[elem] & CSpec.START));
1112                 is_end = (0 != (m_spec.m_anchor_array[elem] & CSpec.END));
1113
1114                 if (is_start && true == is_end)
1115                   {
1116                     m_outstream.writeBytes("\t\tYY_START | YY_END");
1117                   }
1118                 else if (is_start)
1119                   {
1120                     m_outstream.writeBytes("\t\tYY_START");
1121                   }
1122                 else if (is_end)
1123                   {
1124                     m_outstream.writeBytes("\t\tYY_END");
1125                   }
1126                 else
1127                   {
1128                     m_outstream.writeBytes("\t\tYY_NO_ANCHOR");
1129                   }
1130               }
1131             else
1132               {
1133                 m_outstream.writeBytes("\t\tYY_NOT_ACCEPT");
1134               }
1135
1136             if (elem < size - 1)
1137               {
1138                 m_outstream.writeBytes(",");
1139               }
1140
1141             m_outstream.writeBytes("\n");
1142           }
1143         m_outstream.writeBytes("\t};\n");
1144
1145         m_outstream.writeBytes("\tprivate int yy_cmap[] = {\n\t\t");
1146         for (i = 0; i < m_spec.m_col_map.length; ++i)
1147           {
1148             m_outstream.writeBytes((new Integer(m_spec.m_col_map[i])).toString());
1149
1150             if (i < m_spec.m_col_map.length - 1)
1151               {
1152                 m_outstream.writeBytes(",");
1153               }
1154
1155             if (0 == ((i + 1) % 8))
1156               {
1157                 m_outstream.writeBytes("\n\t\t");
1158               }
1159             else
1160               {
1161                 m_outstream.writeBytes(" ");
1162               }
1163           }
1164         m_outstream.writeBytes("\n\t};\n");
1165
1166         m_outstream.writeBytes("\tprivate int yy_rmap[] = {\n\t\t");
1167         for (i = 0; i < m_spec.m_row_map.length; ++i)
1168           {
1169             m_outstream.writeBytes((new Integer(m_spec.m_row_map[i])).toString());
1170
1171             if (i < m_spec.m_row_map.length - 1)
1172               {
1173                 m_outstream.writeBytes(",");
1174               }
1175
1176             if (0 == ((i + 1) % 8))
1177               {
1178                 m_outstream.writeBytes("\n\t\t");
1179               }
1180             else
1181               {
1182                 m_outstream.writeBytes(" ");
1183               }
1184           }
1185         m_outstream.writeBytes("\n\t};\n");
1186
1187         // 6/24/98 Raimondas Lencevicius
1188         // Code to the method end modified
1189         /* yy_nxt[][] is assigned result of
1190            "int [][] unpackFromString(int size1, int size2, String st)".
1191            yy_nxt[][] values are coded into a string
1192            by printing integers and representing
1193            integer sequences as "value:length" pairs. */
1194
1195         m_outstream.writeBytes
1196           ("\tprivate int yy_nxt[][] = unpackFromString(");
1197         size = m_spec.m_dtrans_vector.size();
1198         int sequenceLength = 0; // RL - length of the number sequence
1199         boolean sequenceStarted = false; // RL - has number sequence started?
1200         int previousInt = -20; // RL - Bogus -20 state.
1201
1202         // RL - Output matrix size
1203         m_outstream.writeBytes((new Integer(size)).toString());
1204         m_outstream.writeBytes(",");
1205         m_outstream.writeBytes((new Integer(m_spec.m_dtrans_ncols)).toString());
1206         m_outstream.writeBytes(",");
1207         m_outstream.writeBytes("\n\"");
1208
1209         //  RL - Output matrix
1210         for (elem = 0; elem < size; ++elem)
1211           {
1212
1213             dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(elem);
1214
1215             for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
1216               {
1217                 int writeInt = dtrans.m_dtrans[i];
1218                 if (writeInt == previousInt) // RL - sequence?
1219                   {
1220                     if (sequenceStarted)
1221                       {
1222                         sequenceLength++;
1223                       }
1224                     else
1225                       {
1226                         m_outstream.writeBytes((new Integer(writeInt)).toString());
1227                         m_outstream.writeBytes(":");
1228                         sequenceLength = 2;
1229                         sequenceStarted = true;
1230                       }
1231                   }
1232                 else // RL - no sequence or end sequence
1233                   {
1234                     if (sequenceStarted)
1235                       {
1236                         m_outstream.writeBytes((new Integer(sequenceLength)).toString());
1237                         m_outstream.writeBytes(",");
1238                         sequenceLength = 0;
1239                         sequenceStarted = false;
1240                       }
1241                     else
1242                       {
1243                         if (previousInt != -20)
1244                           {
1245                             m_outstream.writeBytes((new Integer(previousInt)).toString());
1246                             m_outstream.writeBytes(",");
1247                           }
1248                       }
1249                   }
1250                 previousInt = writeInt;
1251               }
1252           }
1253         if (sequenceStarted)
1254           {
1255             m_outstream.writeBytes((new Integer(sequenceLength)).toString());
1256           }
1257         else
1258           {
1259             m_outstream.writeBytes((new Integer(previousInt)).toString());
1260           }
1261         m_outstream.writeBytes("\");\n");
1262       }
1263
1264   /***************************************************************
1265     Function: emit_driver
1266     Description:
1267     **************************************************************/
1268   private void emit_driver
1269     (
1270      )
1271       throws java.io.IOException
1272         {
1273           if (CUtility.DEBUG)
1274             {
1275               CUtility.assert(null != m_spec);
1276               CUtility.assert(null != m_outstream);
1277             }
1278
1279           emit_table();
1280
1281           if (m_spec.m_integer_type)
1282             {
1283               m_outstream.writeBytes("\tpublic int ");
1284               m_outstream.writeBytes(new String(m_spec.m_function_name));
1285               m_outstream.writeBytes(" ()\n");
1286             }
1287           else if (m_spec.m_intwrap_type)
1288             {
1289               m_outstream.writeBytes("\tpublic java.lang.Integer ");
1290               m_outstream.writeBytes(new String(m_spec.m_function_name));
1291               m_outstream.writeBytes(" ()\n");
1292             }
1293           else
1294             {
1295               m_outstream.writeBytes("\tpublic ");
1296               m_outstream.writeBytes(new String(m_spec.m_type_name));
1297               m_outstream.writeBytes(" ");
1298               m_outstream.writeBytes(new String(m_spec.m_function_name));
1299               m_outstream.writeBytes(" ()\n");
1300             }
1301
1302           /*m_outstream.writeBytes("\t\tthrows java.io.IOException {\n");*/
1303           m_outstream.writeBytes("\t\tthrows java.io.IOException");
1304           if (null != m_spec.m_yylex_throw_code)
1305             {
1306               m_outstream.writeBytes(", ");
1307               m_outstream.writeBytes(new String(m_spec.m_yylex_throw_code,0,
1308                                                 m_spec.m_yylex_throw_read));
1309               m_outstream.writeBytes("\n\t\t{\n");
1310             }
1311           else
1312             {
1313               m_outstream.writeBytes(" {\n");
1314             }
1315
1316           m_outstream.writeBytes("\t\tchar yy_lookahead;\n");
1317           m_outstream.writeBytes("\t\tint yy_anchor = YY_NO_ANCHOR;\n");
1318           /*m_outstream.writeBytes("\t\tint yy_state "
1319             + "= yy_initial_dtrans(yy_lexical_state);\n");*/
1320           m_outstream.writeBytes("\t\tint yy_state "
1321                                  + "= yy_state_dtrans[yy_lexical_state];\n");
1322           m_outstream.writeBytes("\t\tint yy_next_state = YY_NO_STATE;\n");
1323           /*m_outstream.writeBytes("\t\tint yy_prev_stave = YY_NO_STATE;\n");*/
1324           m_outstream.writeBytes("\t\tint yy_last_accept_state = YY_NO_STATE;\n");
1325           m_outstream.writeBytes("\t\tboolean yy_initial = true;\n");
1326           m_outstream.writeBytes("\t\tint yy_this_accept;\n");
1327           m_outstream.writeBytes("\n");
1328
1329           m_outstream.writeBytes("\t\tyy_mark_start();\n");
1330           /*m_outstream.writeBytes("\t\tyy_this_accept = yy_accept(yy_state);\n");*/
1331           m_outstream.writeBytes("\t\tyy_this_accept = yy_acpt[yy_state];\n");
1332           m_outstream.writeBytes("\t\tif (YY_NOT_ACCEPT != yy_this_accept) {\n");
1333           m_outstream.writeBytes("\t\t\tyy_last_accept_state = yy_state;\n");
1334           m_outstream.writeBytes("\t\t\tyy_mark_end();\n");
1335           m_outstream.writeBytes("\t\t}\n");
1336
1337           if (NOT_EDBG)
1338             {
1339               m_outstream.writeBytes("\t\tjava.lang.System.out.println(\"Begin\");\n");
1340             }
1341
1342           m_outstream.writeBytes("\t\twhile (true) {\n");
1343
1344           m_outstream.writeBytes("\t\t\tyy_lookahead = yy_advance();\n");
1345           m_outstream.writeBytes("\t\t\tyy_next_state = YY_F;\n");
1346           m_outstream.writeBytes("\t\t\tif (YY_EOF != yy_lookahead) {\n");
1347           /*m_outstream.writeBytes("\t\t\t\tyy_next_state = "
1348                                  + "yy_next(yy_state,yy_lookahead);\n");*/
1349           m_outstream.writeBytes("\t\t\t\tyy_next_state = "
1350            + "yy_nxt[yy_rmap[yy_state]][yy_cmap[yy_lookahead]];\n");
1351
1352           m_outstream.writeBytes("\t\t\t}\n");
1353
1354           if (NOT_EDBG)
1355             {
1356               m_outstream.writeBytes("java.lang.System.out.println(\"Current state: \""
1357                                      + " + yy_state\n");
1358               m_outstream.writeBytes("+ \"\tCurrent input: \"\n");
1359               m_outstream.writeBytes(" + ((char) yy_lookahead));\n");
1360             }
1361           if (NOT_EDBG)
1362             {
1363               m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"State = \""
1364                                      + "+ yy_state);\n");
1365               m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"Accepting status = \""
1366                                      + "+ yy_this_accept);\n");
1367               m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"Last accepting state = \""
1368                                      + "+ yy_last_accept_state);\n");
1369               m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"Next state = \""
1370                                      + "+ yy_next_state);\n");
1371               m_outstream.writeBytes("\t\t\tjava.lang.System.out.println(\"Lookahead input = \""
1372                                      + "+ ((char) yy_lookahead));\n");
1373             }
1374
1375           m_outstream.writeBytes("\t\t\tif (YY_F != yy_next_state) {\n");
1376           m_outstream.writeBytes("\t\t\t\tyy_state = yy_next_state;\n");
1377           m_outstream.writeBytes("\t\t\t\tyy_initial = false;\n");
1378           /*m_outstream.writeBytes("\t\t\t\tyy_this_accept = yy_accept(yy_state);\n");*/
1379           m_outstream.writeBytes("\t\t\t\tyy_this_accept = yy_acpt[yy_state];\n");
1380           m_outstream.writeBytes("\t\t\t\tif (YY_NOT_ACCEPT != yy_this_accept) {\n");
1381           m_outstream.writeBytes("\t\t\t\t\tyy_last_accept_state = yy_state;\n");
1382           m_outstream.writeBytes("\t\t\t\t\tyy_mark_end();\n");
1383           m_outstream.writeBytes("\t\t\t\t}\n");
1384           /*m_outstream.writeBytes("\t\t\t\tyy_prev_state = yy_state;\n");*/
1385           /*m_outstream.writeBytes("\t\t\t\tyy_state = yy_next_state;\n");*/
1386           m_outstream.writeBytes("\t\t\t}\n");
1387
1388           m_outstream.writeBytes("\t\t\telse {\n");
1389           m_outstream.writeBytes("\t\t\t\tif (YY_EOF == yy_lookahead "
1390                                  + "&& true == yy_initial) {\n");
1391           if (null != m_spec.m_eof_code)
1392             {
1393               m_outstream.writeBytes("\t\t\t\t\tyy_do_eof();\n");
1394             }
1395
1396           if (m_spec.m_integer_type || true == m_spec.m_yyeof)
1397             {
1398               m_outstream.writeBytes("\t\t\t\t\treturn YYEOF;\n");
1399             }
1400           else if (null != m_spec.m_eof_value_code)
1401             {
1402               m_outstream.writeBytes(new String(m_spec.m_eof_value_code,0,
1403                                                 m_spec.m_eof_value_read));
1404             }
1405           else
1406             {
1407               m_outstream.writeBytes("\t\t\t\t\treturn null;\n");
1408             }
1409
1410           m_outstream.writeBytes("\t\t\t\t}\n");
1411
1412           m_outstream.writeBytes("\t\t\t\telse if (YY_NO_STATE == yy_last_accept_state) {\n");
1413
1414
1415           /*m_outstream.writeBytes("\t\t\t\t\tyy_error(YY_E_MATCH,false);\n");
1416           m_outstream.writeBytes("\t\t\t\t\tyy_initial = true;\n");
1417           m_outstream.writeBytes("\t\t\t\t\tyy_state "
1418                                  + "= yy_state_dtrans[yy_lexical_state];\n");
1419           m_outstream.writeBytes("\t\t\t\t\tyy_next_state = YY_NO_STATE;\n");*/
1420           /*m_outstream.writeBytes("\t\t\t\t\tyy_prev_state = YY_NO_STATE;\n");*/
1421           /*m_outstream.writeBytes("\t\t\t\t\tyy_last_accept_state = YY_NO_STATE;\n");
1422           m_outstream.writeBytes("\t\t\t\t\tyy_mark_start();\n");*/
1423           /*m_outstream.writeBytes("\t\t\t\t\tyy_this_accept = yy_accept(yy_state);\n");*/
1424           /*m_outstream.writeBytes("\t\t\t\t\tyy_this_accept = yy_acpt[yy_state];\n");
1425           m_outstream.writeBytes("\t\t\t\t\tif (YY_NOT_ACCEPT != yy_this_accept) {\n");
1426           m_outstream.writeBytes("\t\t\t\t\t\tyy_last_accept_state = yy_state;\n");
1427           m_outstream.writeBytes("\t\t\t\t\t}\n");*/
1428
1429           m_outstream.writeBytes("\t\t\t\t\tthrow (new Error(\"Lexical Error: Unmatched Input.\"));\n");
1430           m_outstream.writeBytes("\t\t\t\t}\n");
1431
1432           m_outstream.writeBytes("\t\t\t\telse {\n");
1433           m_outstream.writeBytes("\t\t\t\t\tyy_to_mark();\n");
1434
1435           m_outstream.writeBytes("\t\t\t\t\tyy_anchor = yy_acpt[yy_last_accept_state];\n");
1436           /*m_outstream.writeBytes("\t\t\t\t\tyy_anchor "
1437             + "= yy_accept(yy_last_accept_state);\n");*/
1438           m_outstream.writeBytes("\t\t\t\t\tif (0 != (YY_END & yy_anchor)) {\n");
1439           m_outstream.writeBytes("\t\t\t\t\t\tyy_pushback();\n");
1440           m_outstream.writeBytes("\t\t\t\t\t}\n");
1441           m_outstream.writeBytes("\t\t\t\t\tif (0 != (YY_START & yy_anchor)) {\n");
1442           m_outstream.writeBytes("\t\t\t\t\t\tyy_move_start();\n");
1443           m_outstream.writeBytes("\t\t\t\t\t}\n");
1444
1445
1446           m_outstream.writeBytes("\t\t\t\t\tswitch (yy_last_accept_state) {\n");
1447
1448           emit_actions("\t\t\t\t\t");
1449
1450           m_outstream.writeBytes("\t\t\t\t\tdefault:\n");
1451           m_outstream.writeBytes("\t\t\t\t\t\tyy_error(YY_E_INTERNAL,false);\n");
1452           /*m_outstream.writeBytes("\t\t\t\t\t\treturn null;\n");*/
1453           m_outstream.writeBytes("\t\t\t\t\tcase -1:\n");
1454           m_outstream.writeBytes("\t\t\t\t\t}\n");
1455
1456           m_outstream.writeBytes("\t\t\t\t\tyy_initial = true;\n");
1457           m_outstream.writeBytes("\t\t\t\t\tyy_state "
1458                                  + "= yy_state_dtrans[yy_lexical_state];\n");
1459           m_outstream.writeBytes("\t\t\t\t\tyy_next_state = YY_NO_STATE;\n");
1460           /*m_outstream.writeBytes("\t\t\t\t\tyy_prev_state = YY_NO_STATE;\n");*/
1461           m_outstream.writeBytes("\t\t\t\t\tyy_last_accept_state = YY_NO_STATE;\n");
1462
1463           m_outstream.writeBytes("\t\t\t\t\tyy_mark_start();\n");
1464
1465           /*m_outstream.writeBytes("\t\t\t\t\tyy_this_accept = yy_accept(yy_state);\n");*/
1466           m_outstream.writeBytes("\t\t\t\t\tyy_this_accept = yy_acpt[yy_state];\n");
1467           m_outstream.writeBytes("\t\t\t\t\tif (YY_NOT_ACCEPT != yy_this_accept) {\n");
1468           m_outstream.writeBytes("\t\t\t\t\t\tyy_last_accept_state = yy_state;\n");
1469           m_outstream.writeBytes("\t\t\t\t\t}\n");
1470
1471           m_outstream.writeBytes("\t\t\t\t}\n");
1472           m_outstream.writeBytes("\t\t\t}\n");
1473           m_outstream.writeBytes("\t\t}\n");
1474           m_outstream.writeBytes("\t}\n");
1475
1476           /*m_outstream.writeBytes("\t\t\t\t\n");
1477           m_outstream.writeBytes("\t\t\t\n");
1478           m_outstream.writeBytes("\t\t\t\n");
1479           m_outstream.writeBytes("\t\t\t\n");
1480           m_outstream.writeBytes("\t\t\t\n");
1481           m_outstream.writeBytes("\t\t}\n");*/
1482         }
1483
1484   /***************************************************************
1485     Function: emit_actions
1486     Description:
1487     **************************************************************/
1488   private void emit_actions
1489     (
1490      String tabs
1491      )
1492       throws java.io.IOException
1493         {
1494           int elem;
1495           int size;
1496           int bogus_index;
1497           CAccept accept;
1498
1499           if (CUtility.DEBUG)
1500             {
1501               CUtility.assert(m_spec.m_accept_vector.size()
1502                               == m_spec.m_anchor_array.length);
1503             }
1504
1505           bogus_index = -2;
1506           size = m_spec.m_accept_vector.size();
1507           for (elem = 0; elem < size; ++elem)
1508             {
1509               accept = (CAccept) m_spec.m_accept_vector.elementAt(elem);
1510               if (null != accept)
1511                 {
1512                   m_outstream.writeBytes(tabs + "case " + elem
1513                                          /*+ (new Integer(elem)).toString()*/
1514                                          + ":\n");
1515                   m_outstream.writeBytes(tabs + "\t");
1516                   m_outstream.writeBytes(new String(accept.m_action,0,
1517                                                     accept.m_action_read));
1518                   m_outstream.writeBytes("\n");
1519                   m_outstream.writeBytes(tabs + "case " + bogus_index + ":\n");
1520                   m_outstream.writeBytes(tabs + "\tbreak;\n");
1521                   --bogus_index;
1522                 }
1523             }
1524         }
1525
1526   /***************************************************************
1527     Function: emit_footer
1528     Description:
1529     **************************************************************/
1530   private void emit_footer
1531     (
1532      )
1533       throws java.io.IOException
1534       {
1535         if (CUtility.DEBUG)
1536           {
1537             CUtility.assert(null != m_spec);
1538             CUtility.assert(null != m_outstream);
1539           }
1540
1541         m_outstream.writeBytes("}\n");
1542       }
1543 }
1544
1545 /***************************************************************
1546   Class: CBunch
1547   **************************************************************/
1548 class CBunch
1549 {
1550   /***************************************************************
1551     Member Variables
1552     **************************************************************/
1553   Vector m_nfa_set; /* Vector of CNfa states in dfa state. */
1554   JavaLexBitSet m_nfa_bit; /* BitSet representation of CNfa labels. */
1555   CAccept m_accept; /* Accepting actions, or null if nonaccepting state. */
1556   int m_anchor; /* Anchors on regular expression. */
1557   int m_accept_index; /* CNfa index corresponding to accepting actions. */
1558
1559   /***************************************************************
1560     Function: CBunch
1561     Description: Constructor.
1562     **************************************************************/
1563   CBunch
1564     (
1565      )
1566       {
1567         m_nfa_set = null;
1568         m_nfa_bit = null;
1569         m_accept = null;
1570         m_anchor = CSpec.NONE;
1571         m_accept_index = -1;
1572       }
1573 }
1574
1575 /***************************************************************
1576   Class: CMakeNfa
1577   **************************************************************/
1578 class CMakeNfa
1579 {
1580   /***************************************************************
1581     Member Variables
1582     **************************************************************/
1583   private CSpec m_spec;
1584   private CLexGen m_lexGen;
1585   private CInput m_input;
1586
1587   /***************************************************************
1588     Function: CMakeNfa
1589     Description: Constructor.
1590     **************************************************************/
1591   CMakeNfa
1592     (
1593      )
1594       {
1595         reset();
1596       }
1597
1598   /***************************************************************
1599     Function: reset
1600     Description: Resets CMakeNfa member variables.
1601     **************************************************************/
1602   private void reset
1603     (
1604      )
1605       {
1606         m_input = null;
1607         m_lexGen = null;
1608         m_spec = null;
1609       }
1610
1611   /***************************************************************
1612     Function: set
1613     Description: Sets CMakeNfa member variables.
1614     **************************************************************/
1615   private void set
1616     (
1617      CLexGen lexGen,
1618      CSpec spec,
1619      CInput input
1620      )
1621       {
1622         if (CUtility.DEBUG)
1623           {
1624             CUtility.assert(null != input);
1625             CUtility.assert(null != lexGen);
1626             CUtility.assert(null != spec);
1627           }
1628
1629         m_input = input;
1630         m_lexGen = lexGen;
1631         m_spec = spec;
1632       }
1633
1634   /***************************************************************
1635     Function: thompson
1636     Description: High level access function to module.
1637     Deposits result in input CSpec.
1638     **************************************************************/
1639   void thompson
1640     (
1641      CLexGen lexGen,
1642      CSpec spec,
1643      CInput input
1644      )
1645       throws java.io.IOException
1646         {
1647           int i;
1648           CNfa elem;
1649           int size;
1650
1651           /* Set member variables. */
1652           reset();
1653           set(lexGen,spec,input);
1654
1655           size = m_spec.m_states.size();
1656           m_spec.m_state_rules = new Vector[size];
1657           for (i = 0; i < size; ++i)
1658             {
1659               m_spec.m_state_rules[i] = new Vector();
1660             }
1661
1662           /* Initialize current token variable
1663              and create nfa. */
1664           /*m_spec.m_current_token = m_lexGen.EOS;
1665           m_lexGen.advance();*/
1666
1667           m_spec.m_nfa_start = machine();
1668
1669           /* Set labels in created nfa machine. */
1670           size = m_spec.m_nfa_states.size();
1671           for (i = 0; i < size; ++i)
1672             {
1673               elem = (CNfa) m_spec.m_nfa_states.elementAt(i);
1674               elem.m_label = i;
1675             }
1676
1677           /* Debugging output. */
1678           if (CUtility.DO_DEBUG)
1679             {
1680               m_lexGen.print_nfa();
1681             }
1682
1683           if (m_spec.m_verbose)
1684             {
1685               System.out.println("NFA comprised of "
1686                                  + (m_spec.m_nfa_states.size() + 1)
1687                                  + " states.");
1688             }
1689
1690           reset();
1691         }
1692
1693   /***************************************************************
1694     Function: discardCNfa
1695     Description:
1696     **************************************************************/
1697   private void discardCNfa
1698     (
1699      CNfa nfa
1700      )
1701       {
1702         m_spec.m_nfa_states.removeElement(nfa);
1703       }
1704
1705   /***************************************************************
1706     Function: processStates
1707     Description:
1708     **************************************************************/
1709   private void processStates
1710     (
1711      JavaLexBitSet states,
1712      CNfa current
1713      )
1714       {
1715         int size;
1716         int i;
1717
1718         size = m_spec.m_states.size();
1719         for (i = 0; i <  size; ++i)
1720           {
1721             if (states.get(i))
1722               {
1723                 m_spec.m_state_rules[i].addElement(current);
1724               }
1725           }
1726       }
1727
1728   /***************************************************************
1729     Function: machine
1730     Description: Recursive descent regular expression parser.
1731     **************************************************************/
1732   private CNfa machine
1733     (
1734      )
1735       throws java.io.IOException
1736       {
1737         CNfa start;
1738         CNfa p;
1739         JavaLexBitSet states;
1740
1741         if (CUtility.DESCENT_DEBUG)
1742           {
1743             CUtility.enter("machine",m_spec.m_lexeme,m_spec.m_current_token);
1744           }
1745
1746         states = m_lexGen.getStates();
1747
1748         /* Begin: Added for states. */
1749         m_spec.m_current_token = m_lexGen.EOS;
1750         m_lexGen.advance();
1751         /* End: Added for states. */
1752
1753         start = CAlloc.newCNfa(m_spec);
1754         p = start;
1755         p.m_next = rule();
1756
1757         processStates(states,p.m_next);
1758
1759         while (m_lexGen.END_OF_INPUT != m_spec.m_current_token)
1760           {
1761             /* Make state changes HERE. */
1762             states = m_lexGen.getStates();
1763
1764             /* Begin: Added for states. */
1765             m_lexGen.advance();
1766             if (m_lexGen.END_OF_INPUT == m_spec.m_current_token)
1767               {
1768                 break;
1769               }
1770             /* End: Added for states. */
1771
1772             p.m_next2 = CAlloc.newCNfa(m_spec);
1773             p = p.m_next2;
1774             p.m_next = rule();
1775
1776             processStates(states,p.m_next);
1777           }
1778
1779         if (CUtility.DESCENT_DEBUG)
1780           {
1781             CUtility.leave("machine",m_spec.m_lexeme,m_spec.m_current_token);
1782           }
1783
1784         return start;
1785       }
1786
1787   /***************************************************************
1788     Function: rule
1789     Description: Recursive descent regular expression parser.
1790     **************************************************************/
1791   private CNfa rule
1792     (
1793      )
1794       throws java.io.IOException
1795       {
1796         CNfaPair pair;
1797         CNfa p;
1798         CNfa start = null;
1799         CNfa end = null;
1800         int anchor = CSpec.NONE;
1801
1802         if (CUtility.DESCENT_DEBUG)
1803           {
1804             CUtility.enter("rule",m_spec.m_lexeme,m_spec.m_current_token);
1805           }
1806
1807         pair = CAlloc.newCNfaPair();
1808
1809         if (m_lexGen.AT_BOL == m_spec.m_current_token)
1810           {
1811             start = CAlloc.newCNfa(m_spec);
1812             start.m_edge = '\n';
1813             anchor = anchor | CSpec.START;
1814             m_lexGen.advance();
1815
1816             expr(pair);
1817             start.m_next = pair.m_start;
1818             end = pair.m_end;
1819           }
1820         else
1821           {
1822             expr(pair);
1823             start = pair.m_start;
1824             end = pair.m_end;
1825           }
1826
1827         if (m_lexGen.AT_EOL == m_spec.m_current_token)
1828           {
1829             m_lexGen.advance();
1830             end.m_next = CAlloc.newCNfa(m_spec);
1831
1832             /* This is the way he does it. */
1833             end.m_edge = CNfa.CCL;
1834             end.m_set = new CSet();
1835
1836             end.m_set.add('\n');
1837
1838             if (false == m_spec.m_unix)
1839               {
1840                 end.m_set.add('\r');
1841               }
1842
1843             end = end.m_next;
1844             anchor = anchor | CSpec.END;
1845           }
1846
1847         /* Handle end of regular expression.  See page 103. */
1848         end.m_accept = m_lexGen.packAccept();
1849         end.m_anchor = anchor;
1850
1851         /* Begin: Removed for states. */
1852         /*m_lexGen.advance();*/
1853         /* End: Removed for states. */
1854
1855         if (CUtility.DESCENT_DEBUG)
1856           {
1857             CUtility.leave("rule",m_spec.m_lexeme,m_spec.m_current_token);
1858           }
1859
1860         return start;
1861       }
1862
1863   /***************************************************************
1864     Function: expr
1865     Description: Recursive descent regular expression parser.
1866     **************************************************************/
1867   private void expr
1868     (
1869      CNfaPair pair
1870      )
1871       throws java.io.IOException
1872       {
1873         CNfaPair e2_pair;
1874         CNfa p;
1875
1876         if (CUtility.DESCENT_DEBUG)
1877           {
1878             CUtility.enter("expr",m_spec.m_lexeme,m_spec.m_current_token);
1879           }
1880
1881         if (CUtility.DEBUG)
1882           {
1883             CUtility.assert(null != pair);
1884           }
1885
1886         e2_pair = CAlloc.newCNfaPair();
1887
1888         cat_expr(pair);
1889
1890         while (m_lexGen.OR == m_spec.m_current_token)
1891           {
1892             m_lexGen.advance();
1893             cat_expr(e2_pair);
1894
1895             p = CAlloc.newCNfa(m_spec);
1896             p.m_next2 = e2_pair.m_start;
1897             p.m_next = pair.m_start;
1898             pair.m_start = p;
1899
1900             p = CAlloc.newCNfa(m_spec);
1901             pair.m_end.m_next = p;
1902             e2_pair.m_end.m_next = p;
1903             pair.m_end = p;
1904           }
1905
1906         if (CUtility.DESCENT_DEBUG)
1907           {
1908             CUtility.leave("expr",m_spec.m_lexeme,m_spec.m_current_token);
1909           }
1910       }
1911
1912   /***************************************************************
1913     Function: cat_expr
1914     Description: Recursive descent regular expression parser.
1915     **************************************************************/
1916   private void cat_expr
1917     (
1918      CNfaPair pair
1919      )
1920       throws java.io.IOException
1921       {
1922         CNfaPair e2_pair;
1923
1924         if (CUtility.DESCENT_DEBUG)
1925           {
1926             CUtility.enter("cat_expr",m_spec.m_lexeme,m_spec.m_current_token);
1927           }
1928
1929         if (CUtility.DEBUG)
1930           {
1931             CUtility.assert(null != pair);
1932           }
1933
1934         e2_pair = CAlloc.newCNfaPair();
1935
1936         if (first_in_cat(m_spec.m_current_token))
1937           {
1938             factor(pair);
1939           }
1940
1941         while (first_in_cat(m_spec.m_current_token))
1942           {
1943             factor(e2_pair);
1944
1945             /* Destroy */
1946             pair.m_end.mimic(e2_pair.m_start);
1947             discardCNfa(e2_pair.m_start);
1948
1949             pair.m_end = e2_pair.m_end;
1950           }
1951
1952         if (CUtility.DESCENT_DEBUG)
1953           {
1954             CUtility.leave("cat_expr",m_spec.m_lexeme,m_spec.m_current_token);
1955           }
1956       }
1957
1958   /***************************************************************
1959     Function: first_in_cat
1960     Description: Recursive descent regular expression parser.
1961     **************************************************************/
1962   private boolean first_in_cat
1963     (
1964      int token
1965      )
1966       {
1967         switch (token)
1968           {
1969           case m_lexGen.CLOSE_PAREN:
1970           case m_lexGen.AT_EOL:
1971           case m_lexGen.OR:
1972           case m_lexGen.EOS:
1973             return false;
1974
1975           case m_lexGen.CLOSURE:
1976           case m_lexGen.PLUS_CLOSE:
1977           case m_lexGen.OPTIONAL:
1978             CError.parse_error(CError.E_CLOSE,m_input.m_line_number);
1979             return false;
1980
1981           case m_lexGen.CCL_END:
1982             CError.parse_error(CError.E_BRACKET,m_input.m_line_number);
1983             return false;
1984
1985           case m_lexGen.AT_BOL:
1986             CError.parse_error(CError.E_BOL,m_input.m_line_number);
1987             return false;
1988
1989           default:
1990             break;
1991           }
1992
1993         return true;
1994       }
1995
1996   /***************************************************************
1997     Function: factor
1998     Description: Recursive descent regular expression parser.
1999     **************************************************************/
2000   private void factor
2001     (
2002      CNfaPair pair
2003      )
2004       throws java.io.IOException
2005       {
2006         CNfa start = null;
2007         CNfa end = null;
2008
2009         if (CUtility.DESCENT_DEBUG)
2010           {
2011             CUtility.enter("factor",m_spec.m_lexeme,m_spec.m_current_token);
2012           }
2013
2014         term(pair);
2015
2016         if (m_lexGen.CLOSURE == m_spec.m_current_token
2017             || m_lexGen.PLUS_CLOSE == m_spec.m_current_token
2018             || m_lexGen.OPTIONAL == m_spec.m_current_token)
2019           {
2020             start = CAlloc.newCNfa(m_spec);
2021             end = CAlloc.newCNfa(m_spec);
2022
2023             start.m_next = pair.m_start;
2024             pair.m_end.m_next = end;
2025
2026             if (m_lexGen.CLOSURE == m_spec.m_current_token
2027                 || m_lexGen.OPTIONAL == m_spec.m_current_token)
2028               {
2029                 start.m_next2 = end;
2030               }
2031
2032             if (m_lexGen.CLOSURE == m_spec.m_current_token
2033                 || m_lexGen.PLUS_CLOSE == m_spec.m_current_token)
2034               {
2035                 pair.m_end.m_next2 = pair.m_start;
2036               }
2037
2038             pair.m_start = start;
2039             pair.m_end = end;
2040             m_lexGen.advance();
2041           }
2042
2043         if (CUtility.DESCENT_DEBUG)
2044           {
2045             CUtility.leave("factor",m_spec.m_lexeme,m_spec.m_current_token);
2046           }
2047       }
2048
2049   /***************************************************************
2050     Function: term
2051     Description: Recursive descent regular expression parser.
2052     **************************************************************/
2053   private void term
2054     (
2055      CNfaPair pair
2056      )
2057       throws java.io.IOException
2058       {
2059         CNfa start;
2060         boolean isAlphaL;
2061         int c;
2062
2063         if (CUtility.DESCENT_DEBUG)
2064           {
2065             CUtility.enter("term",m_spec.m_lexeme,m_spec.m_current_token);
2066           }
2067
2068         if (m_lexGen.OPEN_PAREN == m_spec.m_current_token)
2069           {
2070             m_lexGen.advance();
2071             expr(pair);
2072
2073             if (m_lexGen.CLOSE_PAREN == m_spec.m_current_token)
2074               {
2075                 m_lexGen.advance();
2076               }
2077             else
2078               {
2079                 CError.parse_error(CError.E_SYNTAX,m_input.m_line_number);
2080               }
2081           }
2082         else
2083           {
2084             start = CAlloc.newCNfa(m_spec);
2085             pair.m_start = start;
2086
2087             start.m_next = CAlloc.newCNfa(m_spec);
2088             pair.m_end = start.m_next;
2089
2090             if (m_lexGen.L == m_spec.m_current_token &&
2091                 Character.isLetter(m_spec.m_lexeme))
2092               {
2093                 isAlphaL = true;
2094               }
2095             else
2096               {
2097                 isAlphaL = false;
2098               }
2099             if (false == (m_lexGen.ANY == m_spec.m_current_token
2100                           || m_lexGen.CCL_START == m_spec.m_current_token
2101                           || (m_spec.m_ignorecase && isAlphaL)))
2102               {
2103                 start.m_edge = m_spec.m_lexeme;
2104                 m_lexGen.advance();
2105               }
2106             else
2107               {
2108                 start.m_edge = CNfa.CCL;
2109
2110                 start.m_set = new CSet();
2111
2112                 /* Match case-insensitive letters using character class. */
2113                 if (m_spec.m_ignorecase && isAlphaL)
2114                   {
2115                     start.m_set.addncase(m_spec.m_lexeme);
2116                   }
2117                 /* Match dot (.) using character class. */
2118                 else if (m_lexGen.ANY == m_spec.m_current_token)
2119                   {
2120                     start.m_set.add((byte) '\n');
2121                     if (false == m_spec.m_unix)
2122                       {
2123                         start.m_set.add((byte) '\r');
2124                       }
2125                     start.m_set.complement();
2126                   }
2127                 else
2128                   {
2129                     m_lexGen.advance();
2130                     if (m_lexGen.AT_BOL == m_spec.m_current_token)
2131                       {
2132                         m_lexGen.advance();
2133
2134                         /*start.m_set.add((byte) '\n');
2135                         if (false == m_spec.m_unix)
2136                           {
2137                             start.m_set.add((byte) '\r');
2138                           }*/
2139                         start.m_set.complement();
2140                       }
2141                     if (false == (m_lexGen.CCL_END == m_spec.m_current_token))
2142                       {
2143                         dodash(start.m_set);
2144                       }
2145                     /*else
2146                       {
2147                         for (c = 0; c <= ' '; ++c)
2148                           {
2149                             start.m_set.add((byte) c);
2150                           }
2151                       }*/
2152                   }
2153                 m_lexGen.advance();
2154               }
2155           }
2156
2157         if (CUtility.DESCENT_DEBUG)
2158           {
2159             CUtility.leave("term",m_spec.m_lexeme,m_spec.m_current_token);
2160           }
2161       }
2162
2163   /***************************************************************
2164     Function: dodash
2165     Description: Recursive descent regular expression parser.
2166     **************************************************************/
2167   private void dodash
2168     (
2169      CSet set
2170      )
2171       throws java.io.IOException
2172         {
2173           int first = -1;
2174
2175           if (CUtility.DESCENT_DEBUG)
2176             {
2177               CUtility.enter("dodash",m_spec.m_lexeme,m_spec.m_current_token);
2178             }
2179
2180           while (m_lexGen.EOS != m_spec.m_current_token
2181                  && m_lexGen.CCL_END != m_spec.m_current_token)
2182             {
2183               // DASH loses its special meaning if it is first in class.
2184               if (m_lexGen.DASH == m_spec.m_current_token && -1 != first)
2185                 {
2186                   m_lexGen.advance();
2187                   // DASH loses its special meaning if it is last in class.
2188                   if (m_spec.m_current_token == m_lexGen.CCL_END)
2189                     {
2190                       // 'first' already in set.
2191                       set.add('-');
2192                       break;
2193                     }
2194                   for ( ; first <= m_spec.m_lexeme; ++first)
2195                     {
2196                       if (m_spec.m_ignorecase)
2197                         set.addncase((char)first);
2198                       else
2199                         set.add(first);
2200                     }
2201                 }
2202               else
2203                 {
2204                   first = m_spec.m_lexeme;
2205                   if (m_spec.m_ignorecase)
2206                     set.addncase(m_spec.m_lexeme);
2207                   else
2208                     set.add(m_spec.m_lexeme);
2209                 }
2210
2211               m_lexGen.advance();
2212             }
2213
2214         if (CUtility.DESCENT_DEBUG)
2215           {
2216             CUtility.leave("dodash",m_spec.m_lexeme,m_spec.m_current_token);
2217           }
2218       }
2219 }
2220
2221 /***************************************************************
2222   Class: CMinimize
2223  **************************************************************/
2224 class CMinimize
2225 {
2226   /***************************************************************
2227     Member Variables
2228     **************************************************************/
2229   CSpec m_spec;
2230   Vector m_group;
2231   int m_ingroup[];
2232
2233   /***************************************************************
2234     Function: CMinimize
2235     Description: Constructor.
2236     **************************************************************/
2237   CMinimize
2238     (
2239      )
2240       {
2241         reset();
2242       }
2243
2244   /***************************************************************
2245     Function: reset
2246     Description: Resets member variables.
2247     **************************************************************/
2248   private void reset
2249     (
2250      )
2251       {
2252         m_spec = null;
2253         m_group = null;
2254         m_ingroup = null;
2255       }
2256
2257   /***************************************************************
2258     Function: set
2259     Description: Sets member variables.
2260     **************************************************************/
2261   private void set
2262     (
2263      CSpec spec
2264      )
2265       {
2266         if (CUtility.DEBUG)
2267           {
2268             CUtility.assert(null != spec);
2269           }
2270
2271         m_spec = spec;
2272         m_group = null;
2273         m_ingroup = null;
2274       }
2275
2276   /***************************************************************
2277     Function: min_dfa
2278     Description: High-level access function to module.
2279     **************************************************************/
2280   void min_dfa
2281     (
2282      CSpec spec
2283      )
2284       {
2285         set(spec);
2286
2287         /* Remove redundant states. */
2288         minimize();
2289
2290         /* Column and row compression.
2291            Save accept states in auxilary vector. */
2292         reduce();
2293
2294         reset();
2295       }
2296
2297   /***************************************************************
2298     Function: col_copy
2299     Description: Copies source column into destination column.
2300     **************************************************************/
2301   private void col_copy
2302     (
2303      int dest,
2304      int src
2305      )
2306       {
2307         int n;
2308         int i;
2309         CDTrans dtrans;
2310
2311         n = m_spec.m_dtrans_vector.size();
2312         for (i = 0; i < n; ++i)
2313           {
2314             dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
2315             dtrans.m_dtrans[dest] = dtrans.m_dtrans[src];
2316           }
2317       }
2318
2319   /***************************************************************
2320     Function: row_copy
2321     Description: Copies source row into destination row.
2322     **************************************************************/
2323   private void row_copy
2324     (
2325      int dest,
2326      int src
2327      )
2328       {
2329         CDTrans dtrans;
2330
2331         dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(src);
2332         m_spec.m_dtrans_vector.setElementAt(dtrans,dest);
2333       }
2334
2335   /***************************************************************
2336     Function: col_equiv
2337     Description:
2338     **************************************************************/
2339   private boolean col_equiv
2340     (
2341      int col1,
2342      int col2
2343      )
2344       {
2345         int n;
2346         int i;
2347         CDTrans dtrans;
2348
2349         n = m_spec.m_dtrans_vector.size();
2350         for (i = 0; i < n; ++i)
2351           {
2352             dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
2353             if (dtrans.m_dtrans[col1] != dtrans.m_dtrans[col2])
2354               {
2355                 return false;
2356               }
2357           }
2358
2359         return true;
2360       }
2361
2362   /***************************************************************
2363     Function: row_equiv
2364     Description:
2365     **************************************************************/
2366   private boolean row_equiv
2367     (
2368      int row1,
2369      int row2
2370      )
2371       {
2372         int i;
2373         CDTrans dtrans1;
2374         CDTrans dtrans2;
2375
2376         dtrans1 = (CDTrans) m_spec.m_dtrans_vector.elementAt(row1);
2377         dtrans2 = (CDTrans) m_spec.m_dtrans_vector.elementAt(row2);
2378
2379         for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
2380           {
2381             if (dtrans1.m_dtrans[i] != dtrans2.m_dtrans[i])
2382               {
2383                 return false;
2384               }
2385           }
2386
2387         return true;
2388       }
2389
2390   /***************************************************************
2391     Function: reduce
2392     Description:
2393     **************************************************************/
2394   private void reduce
2395     (
2396      )
2397       {
2398         int i;
2399         int j;
2400         int k;
2401         int nrows;
2402         int reduced_ncols;
2403         int reduced_nrows;
2404         JavaLexBitSet set;
2405         CDTrans dtrans;
2406         int size;
2407
2408         set = new JavaLexBitSet();
2409
2410         /* Save accept nodes and anchor entries. */
2411         size = m_spec.m_dtrans_vector.size();
2412         m_spec.m_anchor_array = new int[size];
2413         m_spec.m_accept_vector = new Vector();
2414         for (i = 0; i < size; ++i)
2415           {
2416             dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
2417             m_spec.m_accept_vector.addElement(dtrans.m_accept);
2418             m_spec.m_anchor_array[i] = dtrans.m_anchor;
2419             dtrans.m_accept = null;
2420           }
2421
2422         /* Allocate column map. */
2423         m_spec.m_col_map = new int[m_spec.m_dtrans_ncols];
2424         for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
2425           {
2426             m_spec.m_col_map[i] = -1;
2427           }
2428
2429         /* Process columns for reduction. */
2430         for (reduced_ncols = 0; ; ++reduced_ncols)
2431           {
2432             if (CUtility.DEBUG)
2433               {
2434                 for (i = 0; i < reduced_ncols; ++i)
2435                   {
2436                     CUtility.assert(-1 != m_spec.m_col_map[i]);
2437                   }
2438               }
2439
2440             for (i = reduced_ncols; i < m_spec.m_dtrans_ncols; ++i)
2441               {
2442                 if (-1 == m_spec.m_col_map[i])
2443                   {
2444                     break;
2445                   }
2446               }
2447
2448             if (i >= m_spec.m_dtrans_ncols)
2449               {
2450                 break;
2451               }
2452
2453             if (CUtility.DEBUG)
2454               {
2455                 CUtility.assert(false == set.get(i));
2456                 CUtility.assert(-1 == m_spec.m_col_map[i]);
2457               }
2458
2459             set.set(i);
2460
2461             m_spec.m_col_map[i] = reduced_ncols;
2462
2463             /* UNDONE: Optimize by doing all comparisons in one batch. */
2464             for (j = i + 1; j < m_spec.m_dtrans_ncols; ++j)
2465               {
2466                 if (-1 == m_spec.m_col_map[j] && true == col_equiv(i,j))
2467                   {
2468                     m_spec.m_col_map[j] = reduced_ncols;
2469                   }
2470               }
2471           }
2472
2473         /* Reduce columns. */
2474         k = 0;
2475         for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
2476           {
2477             if (set.get(i))
2478               {
2479                 ++k;
2480
2481                 set.clear(i);
2482
2483                 j = m_spec.m_col_map[i];
2484
2485                 if (CUtility.DEBUG)
2486                   {
2487                     CUtility.assert(j <= i);
2488                   }
2489
2490                 if (j == i)
2491                   {
2492                     continue;
2493                   }
2494
2495                 col_copy(j,i);
2496               }
2497           }
2498         m_spec.m_dtrans_ncols = reduced_ncols;
2499
2500         if (CUtility.DEBUG)
2501           {
2502             CUtility.assert(k == reduced_ncols);
2503           }
2504
2505         /* Allocate row map. */
2506         nrows = m_spec.m_dtrans_vector.size();
2507         m_spec.m_row_map = new int[nrows];
2508         for (i = 0; i < nrows; ++i)
2509           {
2510             m_spec.m_row_map[i] = -1;
2511           }
2512
2513         /* Process rows to reduce. */
2514         for (reduced_nrows = 0; ; ++reduced_nrows)
2515           {
2516             if (CUtility.DEBUG)
2517               {
2518                 for (i = 0; i < reduced_nrows; ++i)
2519                   {
2520                     CUtility.assert(-1 != m_spec.m_row_map[i]);
2521                   }
2522               }
2523
2524             for (i = reduced_nrows; i < nrows; ++i)
2525               {
2526                 if (-1 == m_spec.m_row_map[i])
2527                   {
2528                     break;
2529                   }
2530               }
2531
2532             if (i >= nrows)
2533               {
2534                 break;
2535               }
2536
2537             if (CUtility.DEBUG)
2538               {
2539                 CUtility.assert(false == set.get(i));
2540                 CUtility.assert(-1 == m_spec.m_row_map[i]);
2541               }
2542
2543             set.set(i);
2544
2545             m_spec.m_row_map[i] = reduced_nrows;
2546
2547             /* UNDONE: Optimize by doing all comparisons in one batch. */
2548             for (j = i + 1; j < nrows; ++j)
2549               {
2550                 if (-1 == m_spec.m_row_map[j] && true == row_equiv(i,j))
2551                   {
2552                     m_spec.m_row_map[j] = reduced_nrows;
2553                   }
2554               }
2555           }
2556
2557         /* Reduce rows. */
2558         k = 0;
2559         for (i = 0; i < nrows; ++i)
2560           {
2561             if (set.get(i))
2562               {
2563                 ++k;
2564
2565                 set.clear(i);
2566
2567                 j = m_spec.m_row_map[i];
2568
2569                 if (CUtility.DEBUG)
2570                   {
2571                     CUtility.assert(j <= i);
2572                   }
2573
2574                 if (j == i)
2575                   {
2576                     continue;
2577                   }
2578
2579                 row_copy(j,i);
2580               }
2581           }
2582         m_spec.m_dtrans_vector.setSize(reduced_nrows);
2583
2584         if (CUtility.DEBUG)
2585           {
2586             /*System.out.print("k = " + k + "\nreduced_nrows = " + reduced_nrows + "\n");*/
2587             CUtility.assert(k == reduced_nrows);
2588           }
2589       }
2590
2591   /***************************************************************
2592     Function: fix_dtrans
2593     Description: Updates CDTrans table after minimization
2594     using groups, removing redundant transition table states.
2595     **************************************************************/
2596   private void fix_dtrans
2597     (
2598      )
2599       {
2600         Vector new_vector;
2601         int i;
2602         int size;
2603         Vector dtrans_group;
2604         CDTrans first;
2605         int c;
2606
2607         new_vector = new Vector();
2608
2609         size = m_spec.m_state_dtrans.length;
2610         for (i = 0; i < size; ++i)
2611           {
2612             if (CDTrans.F != m_spec.m_state_dtrans[i])
2613               {
2614                 m_spec.m_state_dtrans[i] = m_ingroup[m_spec.m_state_dtrans[i]];
2615               }
2616           }
2617
2618         size = m_group.size();
2619         for (i = 0; i < size; ++i)
2620           {
2621             dtrans_group = (Vector) m_group.elementAt(i);
2622             first = (CDTrans) dtrans_group.elementAt(0);
2623             new_vector.addElement(first);
2624
2625             for (c = 0; c < m_spec.m_dtrans_ncols; ++c)
2626               {
2627                 if (CDTrans.F != first.m_dtrans[c])
2628                   {
2629                     first.m_dtrans[c] = m_ingroup[first.m_dtrans[c]];
2630                   }
2631               }
2632           }
2633
2634         m_group = null;
2635         m_spec.m_dtrans_vector = new_vector;
2636       }
2637
2638   /***************************************************************
2639     Function: minimize
2640     Description: Removes redundant transition table states.
2641     **************************************************************/
2642   private void minimize
2643     (
2644      )
2645       {
2646         Vector dtrans_group;
2647         Vector new_group;
2648         int i;
2649         int j;
2650         int old_group_count;
2651         int group_count;
2652         CDTrans next;
2653         CDTrans first;
2654         int goto_first;
2655         int goto_next;
2656         int c;
2657         int group_size;
2658         boolean added;
2659
2660         init_groups();
2661
2662         group_count = m_group.size();
2663         old_group_count = group_count - 1;
2664
2665         while (old_group_count != group_count)
2666           {
2667             old_group_count = group_count;
2668
2669             if (CUtility.DEBUG)
2670               {
2671                 CUtility.assert(m_group.size() == group_count);
2672               }
2673
2674             for (i = 0; i < group_count; ++i)
2675               {
2676                 dtrans_group = (Vector) m_group.elementAt(i);
2677
2678                 group_size = dtrans_group.size();
2679                 if (group_size <= 1)
2680                   {
2681                     continue;
2682                   }
2683
2684                 new_group = new Vector();
2685                 added = false;
2686
2687                 first = (CDTrans) dtrans_group.elementAt(0);
2688                 for (j = 1; j < group_size; ++j)
2689                   {
2690                     next = (CDTrans) dtrans_group.elementAt(j);
2691
2692                     for (c = 0; c < m_spec.m_dtrans_ncols; ++c)
2693                       {
2694                         goto_first = first.m_dtrans[c];
2695                         goto_next = next.m_dtrans[c];
2696
2697                         if (goto_first != goto_next
2698                             && (goto_first == CDTrans.F
2699                                 || goto_next == CDTrans.F
2700                                 || m_ingroup[goto_next] != m_ingroup[goto_first]))
2701                           {
2702                             if (CUtility.DEBUG)
2703                               {
2704                                 CUtility.assert(dtrans_group.elementAt(j) == next);
2705                               }
2706
2707                             dtrans_group.removeElementAt(j);
2708                             --j;
2709                             --group_size;
2710                             new_group.addElement(next);
2711                             if (false == added)
2712                               {
2713                                 added = true;
2714                                 ++group_count;
2715                                 m_group.addElement(new_group);
2716                               }
2717                             m_ingroup[next.m_label] = m_group.size() - 1;
2718
2719                             if (CUtility.DEBUG)
2720                               {
2721                                 CUtility.assert(m_group.contains(new_group)
2722                                                 == true);
2723                                 CUtility.assert(m_group.contains(dtrans_group)
2724                                                 == true);
2725                                 CUtility.assert(dtrans_group.contains(first)
2726                                                 == true);
2727                                 CUtility.assert(dtrans_group.contains(next)
2728                                                 == false);
2729                                 CUtility.assert(new_group.contains(first)
2730                                                 == false);
2731                                 CUtility.assert(new_group.contains(next)
2732                                                 == true);
2733                                 CUtility.assert(dtrans_group.size() == group_size);
2734                                 CUtility.assert(i == m_ingroup[first.m_label]);
2735                                 CUtility.assert((m_group.size() - 1)
2736                                                 == m_ingroup[next.m_label]);
2737                               }
2738
2739                             break;
2740                           }
2741                       }
2742                   }
2743               }
2744           }
2745
2746         System.out.println(m_group.size() + " states after removal of redundant states.");
2747
2748         if (m_spec.m_verbose
2749             && true == CUtility.OLD_DUMP_DEBUG)
2750           {
2751             System.out.println("\nStates grouped as follows after minimization");
2752             pgroups();
2753           }
2754
2755         fix_dtrans();
2756       }
2757
2758   /***************************************************************
2759     Function: init_groups
2760     Description:
2761     **************************************************************/
2762   private void init_groups
2763     (
2764      )
2765       {
2766         int i;
2767         int j;
2768         int group_count;
2769         int size;
2770         CAccept accept;
2771         CDTrans dtrans;
2772         Vector dtrans_group;
2773         CDTrans first;
2774         boolean group_found;
2775
2776         m_group = new Vector();
2777         group_count = 0;
2778
2779         size = m_spec.m_dtrans_vector.size();
2780         m_ingroup = new int[size];
2781
2782         for (i = 0; i < size; ++i)
2783           {
2784             group_found = false;
2785             dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
2786
2787             if (CUtility.DEBUG)
2788               {
2789                 CUtility.assert(i == dtrans.m_label);
2790                 CUtility.assert(false == group_found);
2791                 CUtility.assert(group_count == m_group.size());
2792               }
2793
2794             for (j = 0; j < group_count; ++j)
2795               {
2796                 dtrans_group = (Vector) m_group.elementAt(j);
2797
2798                 if (CUtility.DEBUG)
2799                   {
2800                     CUtility.assert(false == group_found);
2801                     CUtility.assert(0 < dtrans_group.size());
2802                   }
2803
2804                 first = (CDTrans) dtrans_group.elementAt(0);
2805
2806                 if (CUtility.SLOW_DEBUG)
2807                   {
2808                     CDTrans check;
2809                     int k;
2810                     int s;
2811
2812                     s = dtrans_group.size();
2813                     CUtility.assert(0 < s);
2814
2815                     for (k = 1; k < s; ++k)
2816                       {
2817                         check = (CDTrans) dtrans_group.elementAt(k);
2818                         CUtility.assert(check.m_accept == first.m_accept);
2819                       }
2820                   }
2821
2822                 if (first.m_accept == dtrans.m_accept)
2823                   {
2824                     dtrans_group.addElement(dtrans);
2825                     m_ingroup[i] = j;
2826                     group_found = true;
2827
2828                     if (CUtility.DEBUG)
2829                       {
2830                         CUtility.assert(j == m_ingroup[dtrans.m_label]);
2831                       }
2832
2833                     break;
2834                   }
2835               }
2836
2837             if (false == group_found)
2838               {
2839                 dtrans_group = new Vector();
2840                 dtrans_group.addElement(dtrans);
2841                 m_ingroup[i] = m_group.size();
2842                 m_group.addElement(dtrans_group);
2843                 ++group_count;
2844               }
2845           }
2846
2847         if (m_spec.m_verbose
2848             && true == CUtility.OLD_DUMP_DEBUG)
2849           {
2850             System.out.println("Initial grouping:");
2851             pgroups();
2852             System.out.println("");
2853           }
2854       }
2855
2856   /***************************************************************
2857     Function: pset
2858     **************************************************************/
2859   private void pset
2860     (
2861      Vector dtrans_group
2862      )
2863       {
2864         int i;
2865         int size;
2866         CDTrans dtrans;
2867
2868         size = dtrans_group.size();
2869         for (i = 0; i < size; ++i)
2870           {
2871             dtrans = (CDTrans) dtrans_group.elementAt(i);
2872             System.out.print(dtrans.m_label + " ");
2873           }
2874       }
2875
2876   /***************************************************************
2877     Function: pgroups
2878     **************************************************************/
2879   private void pgroups
2880     (
2881      )
2882       {
2883         int i;
2884         int dtrans_size;
2885         int group_size;
2886
2887         group_size = m_group.size();
2888         for (i = 0; i < group_size; ++i)
2889           {
2890             System.out.print("\tGroup " + i + " {");
2891             pset((Vector) m_group.elementAt(i));
2892             System.out.println("}\n");
2893           }
2894
2895         System.out.println("");
2896         dtrans_size = m_spec.m_dtrans_vector.size();
2897         for (i = 0; i < dtrans_size; ++i)
2898           {
2899             System.out.println("\tstate " + i
2900                                + " is in group "
2901                                + m_ingroup[i]);
2902           }
2903       }
2904 }
2905
2906 /***************************************************************
2907   Class: CNfa2Dfa
2908  **************************************************************/
2909 class CNfa2Dfa
2910 {
2911   /***************************************************************
2912     Member Variables
2913     **************************************************************/
2914   private CSpec m_spec;
2915   private int m_unmarked_dfa;
2916   private CLexGen m_lexGen;
2917
2918   /***************************************************************
2919     Constants
2920     **************************************************************/
2921   private static final int NOT_IN_DSTATES = -1;
2922
2923   /***************************************************************
2924     Function: CNfa2Dfa
2925     **************************************************************/
2926   CNfa2Dfa
2927     (
2928      )
2929       {
2930         reset();
2931       }
2932
2933   /***************************************************************
2934     Function: set
2935     Description:
2936     **************************************************************/
2937   private void set
2938     (
2939      CLexGen lexGen,
2940      CSpec spec
2941      )
2942       {
2943         m_lexGen = lexGen;
2944         m_spec = spec;
2945         m_unmarked_dfa = 0;
2946       }
2947
2948   /***************************************************************
2949     Function: reset
2950     Description:
2951     **************************************************************/
2952   private void reset
2953     (
2954      )
2955       {
2956         m_lexGen = null;
2957         m_spec = null;
2958         m_unmarked_dfa = 0;
2959       }
2960
2961   /***************************************************************
2962     Function: make_dfa
2963     Description: High-level access function to module.
2964     **************************************************************/
2965   void make_dfa
2966     (
2967      CLexGen lexGen,
2968      CSpec spec
2969      )
2970       {
2971         int i;
2972
2973         reset();
2974         set(lexGen,spec);
2975
2976         make_dtrans();
2977         free_nfa_states();
2978
2979         if (m_spec.m_verbose && true == CUtility.OLD_DUMP_DEBUG)
2980           {
2981             System.out.println(m_spec.m_dfa_states.size()
2982                                + " DFA states in original machine.");
2983           }
2984
2985         free_dfa_states();
2986       }
2987
2988    /***************************************************************
2989     Function: make_dtrans
2990     Description: Creates uncompressed CDTrans transition table.
2991     **************************************************************/
2992   private void make_dtrans
2993     (
2994      )
2995      /* throws java.lang.CloneNotSupportedException*/
2996       {
2997         CDfa next;
2998         CDfa dfa;
2999         CBunch bunch;
3000         int i;
3001         int nextstate;
3002         int size;
3003         CDTrans dtrans;
3004         CNfa nfa;
3005         int istate;
3006         int nstates;
3007
3008         System.out.print("Working on DFA states.");
3009
3010         /* Reference passing type and initializations. */
3011         bunch = new CBunch();
3012         m_unmarked_dfa = 0;
3013
3014         /* Allocate mapping array. */
3015         nstates = m_spec.m_state_rules.length;
3016         m_spec.m_state_dtrans = new int[nstates];
3017
3018         for (istate = 0; nstates > istate; ++istate)
3019           {
3020             if (0 == m_spec.m_state_rules[istate].size())
3021               {
3022                 m_spec.m_state_dtrans[istate] = CDTrans.F;
3023                 continue;
3024               }
3025
3026             /* Create start state and initialize fields. */
3027             bunch.m_nfa_set = (Vector) m_spec.m_state_rules[istate].clone();
3028             sortStates(bunch.m_nfa_set);
3029
3030             bunch.m_nfa_bit = new JavaLexBitSet();
3031
3032             /* Initialize bit set. */
3033             size = bunch.m_nfa_set.size();
3034             for (i = 0; size > i; ++i)
3035               {
3036                 nfa = (CNfa) bunch.m_nfa_set.elementAt(i);
3037                 bunch.m_nfa_bit.set(nfa.m_label);
3038               }
3039
3040             bunch.m_accept = null;
3041             bunch.m_anchor = CSpec.NONE;
3042             bunch.m_accept_index = CUtility.INT_MAX;
3043
3044             e_closure(bunch);
3045             add_to_dstates(bunch);
3046
3047             m_spec.m_state_dtrans[istate] = m_spec.m_dtrans_vector.size();
3048
3049             /* Main loop of CDTrans creation. */
3050             while (null != (dfa = get_unmarked()))
3051               {
3052                 System.out.print(".");
3053                 System.out.flush();
3054
3055                 if (CUtility.DEBUG)
3056                   {
3057                     CUtility.assert(false == dfa.m_mark);
3058                   }
3059
3060                 /* Get first unmarked node, then mark it. */
3061                 dfa.m_mark = true;
3062
3063                 /* Allocate new CDTrans, then initialize fields. */
3064                 dtrans = new CDTrans(m_spec.m_dtrans_vector.size(),m_spec);
3065                 dtrans.m_accept = dfa.m_accept;
3066                 dtrans.m_anchor = dfa.m_anchor;
3067
3068                 /* Set CDTrans array for each character transition. */
3069                 for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
3070                   {
3071                     if (CUtility.DEBUG)
3072                       {
3073                         CUtility.assert(0 <= i);
3074                         CUtility.assert(m_spec.m_dtrans_ncols > i);
3075                       }
3076
3077                     /* Create new dfa set by attempting character transition. */
3078                     move(dfa.m_nfa_set,dfa.m_nfa_bit,i,bunch);
3079                     if (null != bunch.m_nfa_set)
3080                       {
3081                         e_closure(bunch);
3082                       }
3083
3084                     if (CUtility.DEBUG)
3085                       {
3086                         CUtility.assert((null == bunch.m_nfa_set
3087                                          && null == bunch.m_nfa_bit)
3088                                         || (null != bunch.m_nfa_set
3089                                             && null != bunch.m_nfa_bit));
3090                       }
3091
3092                     /* Create new state or set state to empty. */
3093                     if (null == bunch.m_nfa_set)
3094                       {
3095                         nextstate = CDTrans.F;
3096                       }
3097                     else
3098                       {
3099                         nextstate = in_dstates(bunch);
3100
3101                         if (NOT_IN_DSTATES == nextstate)
3102                           {
3103                             nextstate = add_to_dstates(bunch);
3104                           }
3105                       }
3106
3107                     if (CUtility.DEBUG)
3108                       {
3109                         CUtility.assert(nextstate < m_spec.m_dfa_states.size());
3110                       }
3111
3112                     dtrans.m_dtrans[i] = nextstate;
3113                   }
3114
3115                 if (CUtility.DEBUG)
3116                   {
3117                     CUtility.assert(m_spec.m_dtrans_vector.size() == dfa.m_label);
3118                   }
3119
3120                 m_spec.m_dtrans_vector.addElement(dtrans);
3121               }
3122           }
3123
3124         System.out.println("");
3125       }
3126
3127   /***************************************************************
3128     Function: free_dfa_states
3129     **************************************************************/
3130   private void free_dfa_states
3131     (
3132      )
3133       {
3134         m_spec.m_dfa_states = null;
3135         m_spec.m_dfa_sets = null;
3136       }
3137
3138   /***************************************************************
3139     Function: free_nfa_states
3140     **************************************************************/
3141   private void free_nfa_states
3142     (
3143      )
3144       {
3145         /* UNDONE: Remove references to nfas from within dfas. */
3146         /* UNDONE: Don't free CAccepts. */
3147
3148         m_spec.m_nfa_states = null;
3149         m_spec.m_nfa_start = null;
3150         m_spec.m_state_rules = null;
3151       }
3152
3153   /***************************************************************
3154     Function: e_closure
3155     Description: Alters and returns input set.
3156     **************************************************************/
3157   private void e_closure
3158     (
3159      CBunch bunch
3160      )
3161       {
3162         Stack nfa_stack;
3163         int size;
3164         int i;
3165         CNfa state;
3166
3167         /* Debug checks. */
3168         if (CUtility.DEBUG)
3169           {
3170             CUtility.assert(null != bunch);
3171             CUtility.assert(null != bunch.m_nfa_set);
3172             CUtility.assert(null != bunch.m_nfa_bit);
3173           }
3174
3175         bunch.m_accept = null;
3176         bunch.m_anchor = CSpec.NONE;
3177         bunch.m_accept_index = CUtility.INT_MAX;
3178
3179         /* Create initial stack. */
3180         nfa_stack = new Stack();
3181         size = bunch.m_nfa_set.size();
3182         for (i = 0; i < size; ++i)
3183           {
3184             state = (CNfa) bunch.m_nfa_set.elementAt(i);
3185
3186             if (CUtility.DEBUG)
3187               {
3188                 CUtility.assert(bunch.m_nfa_bit.get(state.m_label));
3189               }
3190
3191             nfa_stack.push(state);
3192           }
3193
3194         /* Main loop. */
3195         while (false == nfa_stack.empty())
3196           {
3197             state = (CNfa) nfa_stack.pop();
3198
3199             if (CUtility.OLD_DUMP_DEBUG)
3200               {
3201                 if (null != state.m_accept)
3202                   {
3203                     System.out.println("Looking at accepting state " + state.m_label
3204                                        + " with <"
3205                                        + (new String(state.m_accept.m_action,0,
3206                                                      state.m_accept.m_action_read))
3207                                        + ">");
3208                   }
3209               }
3210
3211             if (null != state.m_accept
3212                 && state.m_label < bunch.m_accept_index)
3213               {
3214                 bunch.m_accept_index = state.m_label;
3215                 bunch.m_accept = state.m_accept;
3216                 bunch.m_anchor = state.m_anchor;
3217
3218                 if (CUtility.OLD_DUMP_DEBUG)
3219                   {
3220                     System.out.println("Found accepting state " + state.m_label
3221                                        + " with <"
3222                                        + (new String(state.m_accept.m_action,0,
3223                                                      state.m_accept.m_action_read))
3224                                        + ">");
3225                   }
3226
3227                 if (CUtility.DEBUG)
3228                   {
3229                     CUtility.assert(null != bunch.m_accept);
3230                     CUtility.assert(CSpec.NONE == bunch.m_anchor
3231                                     || 0 != (bunch.m_anchor & CSpec.END)
3232                                     || 0 != (bunch.m_anchor & CSpec.START));
3233                   }
3234               }
3235
3236             if (CNfa.EPSILON == state.m_edge)
3237               {
3238                 if (null != state.m_next)
3239                   {
3240                     if (false == bunch.m_nfa_set.contains(state.m_next))
3241                       {
3242                         if (CUtility.DEBUG)
3243                           {
3244                             CUtility.assert(false == bunch.m_nfa_bit.get(state.m_next.m_label));
3245                           }
3246
3247                         bunch.m_nfa_bit.set(state.m_next.m_label);
3248                         bunch.m_nfa_set.addElement(state.m_next);
3249                         nfa_stack.push(state.m_next);
3250                       }
3251                   }
3252
3253                 if (null != state.m_next2)
3254                   {
3255                     if (false == bunch.m_nfa_set.contains(state.m_next2))
3256                       {
3257                         if (CUtility.DEBUG)
3258                           {
3259                             CUtility.assert(false == bunch.m_nfa_bit.get(state.m_next2.m_label));
3260                           }
3261
3262                         bunch.m_nfa_bit.set(state.m_next2.m_label);
3263                         bunch.m_nfa_set.addElement(state.m_next2);
3264                         nfa_stack.push(state.m_next2);
3265                       }
3266                   }
3267               }
3268           }
3269
3270         if (null != bunch.m_nfa_set)
3271           {
3272             sortStates(bunch.m_nfa_set);
3273           }
3274
3275         return;
3276       }
3277
3278   /***************************************************************
3279     Function: move
3280     Description: Returns null if resulting NFA set is empty.
3281     **************************************************************/
3282   void move
3283     (
3284      Vector nfa_set,
3285      JavaLexBitSet nfa_bit,
3286      int b,
3287      CBunch bunch
3288      )
3289       {
3290         int size;
3291         int index;
3292         CNfa state;
3293
3294         bunch.m_nfa_set = null;
3295         bunch.m_nfa_bit = null;
3296
3297         size = nfa_set.size();
3298         for (index = 0; index < size; ++index)
3299           {
3300             state = (CNfa) nfa_set.elementAt(index);
3301
3302             if (b == state.m_edge
3303                 || (CNfa.CCL == state.m_edge
3304                     && true == state.m_set.contains(b)))
3305               {
3306                 if (null == bunch.m_nfa_set)
3307                   {
3308                     if (CUtility.DEBUG)
3309                       {
3310                         CUtility.assert(null == bunch.m_nfa_bit);
3311                       }
3312
3313                     bunch.m_nfa_set = new Vector();
3314                     /*bunch.m_nfa_bit
3315                         = new JavaLexBitSet(m_spec.m_nfa_states.size());*/
3316                     bunch.m_nfa_bit = new JavaLexBitSet();
3317                   }
3318
3319                 bunch.m_nfa_set.addElement(state.m_next);
3320                 /*System.out.println("Size of bitset: " + bunch.m_nfa_bit.size());
3321                 System.out.println("Reference index: " + state.m_next.m_label);
3322                 System.out.flush();*/
3323                 bunch.m_nfa_bit.set(state.m_next.m_label);
3324               }
3325           }
3326
3327         if (null != bunch.m_nfa_set)
3328           {
3329             if (CUtility.DEBUG)
3330               {
3331                 CUtility.assert(null != bunch.m_nfa_bit);
3332               }
3333
3334             sortStates(bunch.m_nfa_set);
3335           }
3336
3337         return;
3338       }
3339
3340   /***************************************************************
3341     Function: sortStates
3342     **************************************************************/
3343   private void sortStates
3344     (
3345      Vector nfa_set
3346      )
3347       {
3348         CNfa elem;
3349         int begin;
3350         int size;
3351         int index;
3352         int value;
3353         int smallest_index;
3354         int smallest_value;
3355         CNfa begin_elem;
3356
3357         size = nfa_set.size();
3358         for (begin = 0; begin < size; ++begin)
3359           {
3360             elem = (CNfa) nfa_set.elementAt(begin);
3361             smallest_value = elem.m_label;
3362             smallest_index = begin;
3363
3364             for (index = begin + 1; index < size; ++index)
3365               {
3366                 elem = (CNfa) nfa_set.elementAt(index);
3367                 value = elem.m_label;
3368
3369                 if (value < smallest_value)
3370                   {
3371                     smallest_index = index;
3372                     smallest_value = value;
3373                   }
3374               }
3375
3376             begin_elem = (CNfa) nfa_set.elementAt(begin);
3377             elem = (CNfa) nfa_set.elementAt(smallest_index);
3378             nfa_set.setElementAt(elem,begin);
3379             nfa_set.setElementAt(begin_elem,smallest_index);
3380           }
3381
3382         if (CUtility.OLD_DEBUG)
3383           {
3384             System.out.print("NFA vector indices: ");
3385
3386             for (index = 0; index < size; ++index)
3387               {
3388                 elem = (CNfa) nfa_set.elementAt(index);
3389                 System.out.print(elem.m_label + " ");
3390               }
3391             System.out.print("\n");
3392           }
3393
3394         return;
3395       }
3396
3397   /***************************************************************
3398     Function: get_unmarked
3399     Description: Returns next unmarked DFA state.
3400     **************************************************************/
3401   private CDfa get_unmarked
3402     (
3403      )
3404       {
3405         int size;
3406         CDfa dfa;
3407
3408         size = m_spec.m_dfa_states.size();
3409         while (m_unmarked_dfa < size)
3410           {
3411             dfa = (CDfa) m_spec.m_dfa_states.elementAt(m_unmarked_dfa);
3412
3413             if (false == dfa.m_mark)
3414               {
3415                 if (CUtility.OLD_DUMP_DEBUG)
3416                   {
3417                     System.out.print("*");
3418                     System.out.flush();
3419                   }
3420
3421                 if (m_spec.m_verbose && true == CUtility.OLD_DUMP_DEBUG)
3422                   {
3423                     System.out.println("---------------");
3424                     System.out.print("working on DFA state "
3425                                      + m_unmarked_dfa
3426                                      + " = NFA states: ");
3427                     m_lexGen.print_set(dfa.m_nfa_set);
3428                     System.out.print("\n");
3429                   }
3430
3431                 return dfa;
3432               }
3433
3434             ++m_unmarked_dfa;
3435           }
3436
3437         return null;
3438       }
3439
3440   /***************************************************************
3441     function: add_to_dstates
3442     Description: Takes as input a CBunch with details of
3443     a dfa state that needs to be created.
3444     1) Allocates a new dfa state and saves it in
3445     the appropriate CSpec vector.
3446     2) Initializes the fields of the dfa state
3447     with the information in the CBunch.
3448     3) Returns index of new dfa.
3449     **************************************************************/
3450   private int add_to_dstates
3451     (
3452      CBunch bunch
3453      )
3454       {
3455         CDfa dfa;
3456
3457         if (CUtility.DEBUG)
3458           {
3459             CUtility.assert(null != bunch.m_nfa_set);
3460             CUtility.assert(null != bunch.m_nfa_bit);
3461             CUtility.assert(null != bunch.m_accept
3462                             || CSpec.NONE == bunch.m_anchor);
3463           }
3464
3465         /* Allocate, passing CSpec so dfa label can be set. */
3466         dfa = CAlloc.newCDfa(m_spec);
3467
3468         /* Initialize fields, including the mark field. */
3469         dfa.m_nfa_set = (Vector) bunch.m_nfa_set.clone();
3470         dfa.m_nfa_bit = (JavaLexBitSet) bunch.m_nfa_bit.clone();
3471         dfa.m_accept = bunch.m_accept;
3472         dfa.m_anchor = bunch.m_anchor;
3473         dfa.m_mark = false;
3474
3475         /* Register dfa state using BitSet in CSpec Hashtable. */
3476         m_spec.m_dfa_sets.put(dfa.m_nfa_bit,dfa);
3477         /*registerCDfa(dfa);*/
3478
3479         if (CUtility.OLD_DUMP_DEBUG)
3480           {
3481             System.out.print("Registering set : ");
3482             m_lexGen.print_set(dfa.m_nfa_set);
3483             System.out.println("");
3484           }
3485
3486         return dfa.m_label;
3487       }
3488
3489   /***************************************************************
3490     Function: in_dstates
3491     **************************************************************/
3492   private int in_dstates
3493     (
3494      CBunch bunch
3495      )
3496       {
3497         CDfa dfa;
3498
3499         if (CUtility.OLD_DEBUG)
3500           {
3501             System.out.print("Looking for set : ");
3502             m_lexGen.print_set(bunch.m_nfa_set);
3503           }
3504
3505         dfa = (CDfa) m_spec.m_dfa_sets.get(bunch.m_nfa_bit);
3506
3507         if (null != dfa)
3508           {
3509             if (CUtility.OLD_DUMP_DEBUG)
3510               {
3511                 System.out.println(" FOUND!");
3512               }
3513
3514             return dfa.m_label;
3515           }
3516
3517         if (CUtility.OLD_DUMP_DEBUG)
3518           {
3519             System.out.println(" NOT FOUND!");
3520           }
3521         return NOT_IN_DSTATES;
3522       }
3523
3524 }
3525
3526 /***************************************************************
3527   Class: CAlloc
3528   **************************************************************/
3529 class CAlloc
3530 {
3531   /***************************************************************
3532     Function: newCDfa
3533     **************************************************************/
3534   static CDfa newCDfa
3535     (
3536      CSpec spec
3537      )
3538       {
3539         CDfa dfa;
3540
3541         dfa = new CDfa(spec.m_dfa_states.size());
3542         spec.m_dfa_states.addElement(dfa);
3543
3544         return dfa;
3545       }
3546
3547   /***************************************************************
3548     Function: newCNfaPair
3549     Description:
3550     **************************************************************/
3551   static CNfaPair newCNfaPair
3552     (
3553      )
3554       {
3555         CNfaPair pair = new CNfaPair();
3556
3557         return pair;
3558       }
3559
3560   /***************************************************************
3561     Function: newCNfa
3562     Description:
3563     **************************************************************/
3564   static CNfa newCNfa
3565     (
3566      CSpec spec
3567      )
3568       {
3569         CNfa p;
3570
3571         /* UNDONE: Buffer this? */
3572
3573         p = new CNfa();
3574
3575         /*p.m_label = spec.m_nfa_states.size();*/
3576         spec.m_nfa_states.addElement(p);
3577         p.m_edge = CNfa.EPSILON;
3578
3579         return p;
3580       }
3581 }
3582
3583 /***************************************************************
3584   Class: Main
3585   Description: Top-level lexical analyzer generator function.
3586  **************************************************************/
3587 public class Main
3588 {
3589   /***************************************************************
3590     Function: main
3591     **************************************************************/
3592   public static void main
3593     (
3594      String arg[]
3595      )
3596     throws java.io.IOException
3597       {
3598         CLexGen lg;
3599
3600         if (arg.length < 1)
3601           {
3602             System.out.println("Usage: JLex.Main <filename>");
3603             return;
3604           }
3605
3606         /* Note: For debuging, it may be helpful to remove the try/catch
3607            block and permit the Exception to propagate to the top level.
3608            This gives more information. */
3609         try
3610           {
3611             lg = new CLexGen(arg[0]);
3612             lg.generate();
3613           }
3614         catch (Error e)
3615           {
3616             System.out.println(e.getMessage());
3617           }
3618       }
3619 }
3620
3621 /***************************************************************
3622   Class: CDTrans
3623   **************************************************************/
3624 class CDTrans
3625 {
3626   /*************************************************************
3627     Member Variables
3628     ***********************************************************/
3629   int m_dtrans[];
3630   CAccept m_accept;
3631   int m_anchor;
3632   int m_label;
3633
3634   /*************************************************************
3635     Constants
3636     ***********************************************************/
3637   static final int F = -1;
3638
3639   /*************************************************************
3640     Function: CTrans
3641     ***********************************************************/
3642   CDTrans
3643     (
3644      int label,
3645      CSpec spec
3646      )
3647       {
3648         m_dtrans = new int[spec.m_dtrans_ncols];
3649         m_accept = null;
3650         m_anchor = CSpec.NONE;
3651         m_label = label;
3652       }
3653 }
3654
3655 /***************************************************************
3656   Class: CDfa
3657   **************************************************************/
3658 class CDfa
3659 {
3660   /***************************************************************
3661     Member Variables
3662     ***********************************************************/
3663   int m_group;
3664   boolean m_mark;
3665   CAccept m_accept;
3666   int m_anchor;
3667   Vector m_nfa_set;
3668   JavaLexBitSet m_nfa_bit;
3669   int m_label;
3670
3671   /***************************************************************
3672     Function: CDfa
3673     **************************************************************/
3674   CDfa
3675     (
3676      int label
3677      )
3678       {
3679         m_group = 0;
3680         m_mark = false;
3681
3682         m_accept = null;
3683         m_anchor = CSpec.NONE;
3684
3685         m_nfa_set = null;
3686         m_nfa_bit = null;
3687
3688         m_label = label;
3689       }
3690 }
3691
3692 /***************************************************************
3693   Class: CAccept
3694  **************************************************************/
3695 class CAccept
3696 {
3697   /***************************************************************
3698     Member Variables
3699     **************************************************************/
3700   char m_action[];
3701   int m_action_read;
3702   int m_line_number;
3703
3704   /***************************************************************
3705     Function: CAccept
3706     **************************************************************/
3707   CAccept
3708     (
3709      char action[],
3710      int action_read,
3711      int line_number
3712      )
3713       {
3714         int elem;
3715
3716         m_action_read = action_read;
3717
3718         m_action = new char[m_action_read];
3719         for (elem = 0; elem < m_action_read; ++elem)
3720           {
3721             m_action[elem] = action[elem];
3722           }
3723
3724         m_line_number = line_number;
3725       }
3726
3727   /***************************************************************
3728     Function: CAccept
3729     **************************************************************/
3730   CAccept
3731     (
3732      CAccept accept
3733      )
3734       {
3735         int elem;
3736
3737         m_action_read = accept.m_action_read;
3738
3739         m_action = new char[m_action_read];
3740         for (elem = 0; elem < m_action_read; ++elem)
3741           {
3742             m_action[elem] = accept.m_action[elem];
3743           }
3744
3745         m_line_number = accept.m_line_number;
3746       }
3747
3748   /***************************************************************
3749     Function: mimic
3750     **************************************************************/
3751   void mimic
3752     (
3753      CAccept accept
3754      )
3755       {
3756         int elem;
3757
3758         m_action_read = accept.m_action_read;
3759
3760         m_action = new char[m_action_read];
3761         for (elem = 0; elem < m_action_read; ++elem)
3762           {
3763             m_action[elem] = accept.m_action[elem];
3764           }
3765       }
3766 }
3767
3768 /***************************************************************
3769   Class: CAcceptAnchor
3770   **************************************************************/
3771 class CAcceptAnchor
3772 {
3773   /***************************************************************
3774     Member Variables
3775     **************************************************************/
3776   CAccept m_accept;
3777   int m_anchor;
3778
3779   /***************************************************************
3780     Function: CAcceptAnchor
3781     **************************************************************/
3782   CAcceptAnchor
3783     (
3784      )
3785       {
3786         m_accept = null;
3787         m_anchor = CSpec.NONE;
3788       }
3789 }
3790
3791 /***************************************************************
3792   Class: CNfaPair
3793   **************************************************************/
3794 class CNfaPair
3795 {
3796   /***************************************************************
3797     Member Variables
3798     **************************************************************/
3799   CNfa m_start;
3800   CNfa m_end;
3801
3802   /***************************************************************
3803     Function: CNfaPair
3804     **************************************************************/
3805   CNfaPair
3806     (
3807      )
3808       {
3809         m_start = null;
3810         m_end = null;
3811       }
3812 }
3813
3814 /***************************************************************
3815   Class: CInput
3816   Description:
3817  **************************************************************/
3818 class CInput
3819 {
3820   /***************************************************************
3821     Member Variables
3822     **************************************************************/
3823   private InputStream m_input; /* JLex specification file. */
3824
3825   private byte m_buffer[]; /* Input buffer. */
3826   private int m_buffer_read; /* Number of bytes read into input buffer. */
3827   private int m_buffer_index; /* Current index into input buffer. */
3828
3829   boolean m_eof_reached; /* Whether EOF has been encountered. */
3830   boolean m_pushback_line;
3831
3832   char m_line[]; /* Line buffer. */
3833   int m_line_read; /* Number of bytes read into line buffer. */
3834   int m_line_index; /* Current index into line buffer. */
3835
3836   int m_line_number; /* Current line number. */
3837
3838   /***************************************************************
3839     Constants
3840     **************************************************************/
3841   private static final int BUFFER_SIZE = 1024;
3842   static final boolean EOF = true;
3843   static final boolean NOT_EOF = false;
3844
3845   /***************************************************************
3846     Function: CInput
3847     Description:
3848     **************************************************************/
3849   CInput
3850     (
3851      InputStream input
3852      )
3853       {
3854         if (CUtility.DEBUG)
3855           {
3856             CUtility.assert(null != input);
3857           }
3858
3859         /* Initialize input stream. */
3860         m_input = input;
3861
3862         /* Initialize buffers and index counters. */
3863         m_buffer = new byte[BUFFER_SIZE];
3864         m_buffer_read = 0;
3865         m_buffer_index = 0;
3866
3867         m_line = new char[BUFFER_SIZE];
3868         m_line_read = 0;
3869         m_line_index = 0;
3870
3871         /* Initialize state variables. */
3872         m_eof_reached = false;
3873         m_line_number = 0;
3874         m_pushback_line = false;
3875       }
3876
3877   /***************************************************************
3878     Function: getLine
3879     Description: Returns true on EOF, false otherwise.
3880     Guarantees not to return a blank line, or a line
3881     of zero length.
3882     **************************************************************/
3883   boolean getLine
3884     (
3885      )
3886       throws java.io.IOException
3887       {
3888         int elem;
3889
3890         /* Has EOF already been reached? */
3891         if (m_eof_reached)
3892           {
3893             if (CUtility.OLD_DEBUG)
3894               {
3895                 System.out.print("Line 1: ");
3896                 System.out.print(new String(m_line,0,
3897                                             m_line_read));
3898               }
3899
3900             return true;
3901           }
3902
3903         /* Pushback current line? */
3904         if (m_pushback_line)
3905           {
3906             m_pushback_line = false;
3907
3908             /* Check for empty line. */
3909             for (elem = 0; elem < m_line_read; ++elem)
3910               {
3911                 if (false == CUtility.isspace(m_line[elem]))
3912                   {
3913                     break;
3914                   }
3915               }
3916
3917             /* Nonempty? */
3918             if (elem < m_line_read)
3919               {
3920                 m_line_index = 0;
3921                 return false;
3922               }
3923
3924             if (CUtility.OLD_DEBUG)
3925               {
3926                 System.out.print("Line 2: ");
3927                 System.out.print(new String(m_line,0,
3928                                             m_line_read));
3929               }
3930           }
3931
3932         while (true)
3933           {
3934             /* Refill buffer? */
3935             if (m_buffer_index >= m_buffer_read)
3936               {
3937                 m_buffer_read = m_input.read(m_buffer);
3938                 if (-1 == m_buffer_read)
3939                   {
3940                     m_eof_reached = true;
3941
3942                     if (CUtility.OLD_DEBUG)
3943                       {
3944                         System.out.print("Line 3: ");
3945                         System.out.print(new String(m_line,0,
3946                                                     m_line_read));
3947                       }
3948
3949                     m_line_index = 0;
3950                     return true;
3951                   }
3952                 m_buffer_index = 0;
3953               }
3954
3955             m_line_read = 0;
3956             while ((byte) '\n' != m_buffer[m_buffer_index]
3957                    && (byte) '\r' != m_buffer[m_buffer_index])
3958               {
3959                 m_line[m_line_read] = (char) m_buffer[m_buffer_index];
3960
3961                 ++m_buffer_index;
3962
3963                 /* Refill buffer? */
3964                 if (m_buffer_index >= m_buffer_read)
3965                   {
3966                     m_buffer_read = m_input.read(m_buffer);
3967                     if (-1 == m_buffer_read)
3968                       {
3969                         m_eof_reached = true;
3970
3971                         /* Check for empty lines and discard them. */
3972                         elem = 0;
3973                         while (CUtility.isspace(m_line[elem]))
3974                           {
3975                             ++elem;
3976                             if (elem == m_line_read)
3977                               {
3978                                 break;
3979                               }
3980                           }
3981
3982                         if (elem < m_line_read)
3983                           {
3984                             if (CUtility.OLD_DEBUG)
3985                               {
3986                                 System.out.print("Line 4:");
3987                                 System.out.print(new String(m_line,
3988                                                             0,
3989                                                             m_line_read));
3990                               }
3991
3992
3993                             m_line_index = 0;
3994                             return false;
3995                           }
3996
3997                         if (CUtility.OLD_DEBUG)
3998                           {
3999                             System.out.print("Line <e>:");
4000                             System.out.print(new String(m_line,0,
4001                                                     m_line_read));
4002                           }
4003
4004                         m_line_index = 0;
4005                         return true;
4006                       }
4007
4008                     m_buffer_index = 0;
4009                   }
4010
4011                 ++m_line_read;
4012
4013                 /* Resize line buffer? */
4014                 if (m_line_read >= m_line.length)
4015                   {
4016                     m_line = CUtility.doubleSize(m_line);
4017                   }
4018               }
4019             /* Save newline in buffer. */
4020             /* Old Source:
4021                m_line[m_line_read] = (char) m_buffer[m_buffer_index];
4022                ++m_line_read;
4023                ++m_buffer_index;
4024                New Source: */
4025             if ((byte) '\r' == m_buffer[m_buffer_index])
4026               {
4027                 /* This is a hack and may propagate
4028                    some (pre-existing) line counting bugs on non-UNIX,
4029                    but otherwise it will work fine.
4030                    The only effect will be some bad error messages.
4031                    The other option is to do some massive
4032                    hacking on the input routines
4033                    that may destabilize the code. */
4034                 m_line[m_line_read] = '\n';
4035                 ++m_line_read;
4036                 ++m_buffer_index;
4037               }
4038             else
4039               {
4040                 m_line[m_line_read] = (char) m_buffer[m_buffer_index];
4041                 ++m_line_read;
4042                 ++m_buffer_index;
4043               }
4044             /* End of New Source. */
4045             ++m_line_number;
4046
4047             /* Check for empty lines and discard them. */
4048             elem = 0;
4049             while (CUtility.isspace(m_line[elem]))
4050               {
4051                 ++elem;
4052                 if (elem == m_line_read)
4053                   {
4054                     break;
4055                   }
4056               }
4057
4058             if (elem < m_line_read)
4059               {
4060                 break;
4061               }
4062           }
4063
4064         if (CUtility.OLD_DEBUG)
4065           {
4066             System.out.print("Line <f>:");
4067             System.out.print(new String(m_line,0,m_line_read));
4068           }
4069
4070         m_line_index = 0;
4071         return false;
4072       }
4073 }
4074
4075 /********************************************************
4076   Class: Utility
4077   *******************************************************/
4078 class CUtility
4079 {
4080   /********************************************************
4081     Constants
4082     *******************************************************/
4083   static final boolean DEBUG = true;
4084   static final boolean SLOW_DEBUG = true;
4085   static final boolean DUMP_DEBUG = true;
4086   /*static final boolean DEBUG = false;
4087   static final boolean SLOW_DEBUG = false;
4088   static final boolean DUMP_DEBUG = false;*/
4089   static final boolean DESCENT_DEBUG = false;
4090   static final boolean OLD_DEBUG = false;
4091   static final boolean OLD_DUMP_DEBUG = false;
4092   static final boolean FOODEBUG = false;
4093   static final boolean DO_DEBUG = false;
4094
4095   /********************************************************
4096     Constants: Integer Bounds
4097     *******************************************************/
4098   static final int INT_MAX = 2147483647;
4099
4100   /* UNDONE: What about other character values??? */
4101   static final int MAX_SEVEN_BIT = 127;
4102   static final int MAX_EIGHT_BIT = 256;
4103
4104   /********************************************************
4105     Function: enter
4106     Description: Debugging routine.
4107     *******************************************************/
4108   static void enter
4109     (
4110      String descent,
4111      char lexeme,
4112      int token
4113      )
4114       {
4115         System.out.println("Entering " + descent
4116                            + " [lexeme: " + lexeme
4117                            + "] [token: " + token + "]");
4118       }
4119
4120   /********************************************************
4121     Function: leave
4122     Description: Debugging routine.
4123     *******************************************************/
4124   static void leave
4125     (
4126      String descent,
4127      char lexeme,
4128      int token
4129      )
4130       {
4131         System.out.println("Leaving " + descent
4132                            + " [lexeme:" + lexeme
4133                            + "] [token:" + token + "]");
4134       }
4135
4136   /********************************************************
4137     Function: assert
4138     Description: Debugging routine.
4139     *******************************************************/
4140   static void assert
4141     (
4142      boolean expr
4143      )
4144       {
4145         if (DEBUG && false == expr)
4146           {
4147             System.out.println("Assertion Failed");
4148             throw new Error("Assertion Failed.");
4149           }
4150       }
4151
4152   /***************************************************************
4153     Function: doubleSize
4154     **************************************************************/
4155   static char[] doubleSize
4156     (
4157      char oldBuffer[]
4158      )
4159       {
4160         char newBuffer[] = new char[2 * oldBuffer.length];
4161         int elem;
4162
4163         for (elem = 0; elem < oldBuffer.length; ++elem)
4164           {
4165             newBuffer[elem] = oldBuffer[elem];
4166           }
4167
4168         return newBuffer;
4169       }
4170
4171   /***************************************************************
4172     Function: doubleSize
4173     **************************************************************/
4174   static byte[] doubleSize
4175     (
4176      byte oldBuffer[]
4177      )
4178       {
4179         byte newBuffer[] = new byte[2 * oldBuffer.length];
4180         int elem;
4181
4182         for (elem = 0; elem < oldBuffer.length; ++elem)
4183           {
4184             newBuffer[elem] = oldBuffer[elem];
4185           }
4186
4187         return newBuffer;
4188       }
4189
4190   /********************************************************
4191     Function: hex2bin
4192     *******************************************************/
4193   static char hex2bin
4194     (
4195      char c
4196      )
4197       {
4198         if ('0' <= c && '9' >= c)
4199           {
4200             return (char) (c - '0');
4201           }
4202         else if ('a' <= c && 'f' >= c)
4203           {
4204             return (char) (c - 'a' + 10);
4205           }
4206         else if ('A' <= c && 'F' >= c)
4207           {
4208             return (char) (c - 'A' + 10);
4209           }
4210
4211         CError.impos("Bad hexidecimal digit" + c);
4212         return 0;
4213       }
4214
4215   /********************************************************
4216     Function: ishexdigit
4217     *******************************************************/
4218   static boolean ishexdigit
4219     (
4220      char c
4221      )
4222       {
4223         if (('0' <= c && '9' >= c)
4224             || ('a' <= c && 'f' >= c)
4225             || ('A' <= c && 'F' >= c))
4226           {
4227             return true;
4228           }
4229
4230         return false;
4231       }
4232
4233   /********************************************************
4234     Function: oct2bin
4235     *******************************************************/
4236   static char oct2bin
4237     (
4238      char c
4239      )
4240       {
4241         if ('0' <= c && '7' >= c)
4242           {
4243             return (char) (c - '0');
4244           }
4245
4246         CError.impos("Bad octal digit " + c);
4247         return 0;
4248       }
4249
4250   /********************************************************
4251     Function: isoctdigit
4252     *******************************************************/
4253   static boolean isoctdigit
4254     (
4255      char c
4256      )
4257       {
4258         if ('0' <= c && '7' >= c)
4259           {
4260             return true;
4261           }
4262
4263         return false;
4264       }
4265
4266   /********************************************************
4267     Function: isspace
4268     *******************************************************/
4269   static boolean isspace
4270     (
4271      char c
4272      )
4273       {
4274         if ('\b' == c
4275             || '\t' == c
4276             || '\n' == c
4277             || '\f' == c
4278             || '\r' == c
4279             || ' ' == c)
4280           {
4281             return true;
4282           }
4283
4284         return false;
4285       }
4286
4287   /********************************************************
4288     Function: isnewline
4289     *******************************************************/
4290   static boolean isnewline
4291     (
4292      char c
4293      )
4294       {
4295         if ('\n' == c
4296             || '\r' == c)
4297             {
4298             return true;
4299           }
4300
4301         return false;
4302       }
4303
4304   /********************************************************
4305     Function: isalpha
4306     *******************************************************/
4307   static boolean isalpha
4308     (
4309      char c
4310      )
4311       {
4312         if (('a' <= c && 'z' >= c)
4313             || ('A' <= c && 'Z' >= c))
4314           {
4315             return true;
4316           }
4317
4318         return false;
4319       }
4320
4321   /********************************************************
4322     Function: toupper
4323     *******************************************************/
4324   static char toupper
4325     (
4326      char c
4327      )
4328       {
4329         if (('a' <= c && 'z' >= c))
4330           {
4331             return (char) (c - 'a' + 'A');
4332           }
4333
4334         return c;
4335       }
4336
4337   /********************************************************
4338     Function: bytencmp
4339     Description: Compares up to n elements of
4340     byte array a[] against byte array b[].
4341     The first byte comparison is made between
4342     a[a_first] and b[b_first].  Comparisons continue
4343     until the null terminating byte '\0' is reached
4344     or until n bytes are compared.
4345     Return Value: Returns 0 if arrays are the
4346     same up to and including the null terminating byte
4347     or up to and including the first n bytes,
4348     whichever comes first.
4349     *******************************************************/
4350   static int bytencmp
4351     (
4352      byte a[],
4353      int a_first,
4354      byte b[],
4355      int b_first,
4356      int n
4357      )
4358       {
4359         int elem;
4360
4361         for (elem = 0; elem < n; ++elem)
4362           {
4363             /*System.out.print((char) a[a_first + elem]);
4364             System.out.print((char) b[b_first + elem]);*/
4365
4366             if ('\0' == a[a_first + elem] && '\0' == b[b_first + elem])
4367               {
4368                 /*System.out.println("return 0");*/
4369                 return 0;
4370               }
4371             if (a[a_first + elem] < b[b_first + elem])
4372               {
4373                 /*System.out.println("return 1");*/
4374                 return 1;
4375               }
4376             else if (a[a_first + elem] > b[b_first + elem])
4377               {
4378                 /*System.out.println("return -1");*/
4379                 return -1;
4380               }
4381           }
4382
4383         /*System.out.println("return 0");*/
4384         return 0;
4385       }
4386
4387   /********************************************************
4388     Function: charncmp
4389     *******************************************************/
4390   static int charncmp
4391     (
4392      char a[],
4393      int a_first,
4394      char b[],
4395      int b_first,
4396      int n
4397      )
4398       {
4399         int elem;
4400
4401         for (elem = 0; elem < n; ++elem)
4402           {
4403             if ('\0' == a[a_first + elem] && '\0' == b[b_first + elem])
4404               {
4405                 return 0;
4406               }
4407             if (a[a_first + elem] < b[b_first + elem])
4408               {
4409                 return 1;
4410               }
4411             else if (a[a_first + elem] > b[b_first + elem])
4412               {
4413                 return -1;
4414               }
4415           }
4416
4417         return 0;
4418       }
4419 }
4420
4421 /********************************************************
4422   Class: CError
4423   *******************************************************/
4424 class CError
4425 {
4426   /********************************************************
4427     Function: impos
4428     Description:
4429     *******************************************************/
4430   static void impos
4431     (
4432      String message
4433      )
4434       {
4435         System.out.println("JLex Error: " + message);
4436       }
4437
4438   /********************************************************
4439     Constants
4440     Description: Error codes for parse_error().
4441     *******************************************************/
4442   static final int E_BADEXPR = 0;
4443   static final int E_PAREN = 1;
4444   static final int E_LENGTH = 2;
4445   static final int E_BRACKET = 3;
4446   static final int E_BOL = 4;
4447   static final int E_CLOSE = 5;
4448   static final int E_NEWLINE = 6;
4449   static final int E_BADMAC = 7;
4450   static final int E_NOMAC = 8;
4451   static final int E_MACDEPTH = 9;
4452   static final int E_INIT = 10;
4453   static final int E_EOF = 11;
4454   static final int E_DIRECT = 12;
4455   static final int E_INTERNAL = 13;
4456   static final int E_STATE = 14;
4457   static final int E_MACDEF = 15;
4458   static final int E_SYNTAX = 16;
4459   static final int E_BRACE = 17;
4460   static final int E_DASH = 18;
4461
4462   /********************************************************
4463     Constants
4464     Description: String messages for parse_error();
4465     *******************************************************/
4466   static final String errmsg[] =
4467     {
4468       "Malformed regular expression.",
4469       "Missing close parenthesis.",
4470       "Too many regular expressions or expression too long.",
4471       "Missing [ in character class.",
4472       "^ must be at start of expression or after [.",
4473       "+ ? or * must follow an expression or subexpression.",
4474       "Newline in quoted string.",
4475       "Missing } in macro expansion.",
4476       "Macro does not exist.",
4477       "Macro expansions nested too deeply.",
4478       "JLex has not been successfully initialized.",
4479       "Unexpected end-of-file found.",
4480       "Undefined or badly-formed JLex directive.",
4481       "Internal JLex error.",
4482       "Unitialized state name.",
4483       "Badly formed macro definition.",
4484       "Syntax error.",
4485       "Missing brace at start of lexical action.",
4486       "Special character dash - in character class [...] must\n"
4487         + "\tbe preceded by start-of-range character."
4488     };
4489
4490   /********************************************************
4491     Function: parse_error
4492     Description:
4493     *******************************************************/
4494   static void parse_error
4495     (
4496      int error_code,
4497      int line_number
4498      )
4499       {
4500         System.out.println("Error: Parse error at line "
4501                            + line_number + ".");
4502         System.out.println("Description: " + errmsg[error_code]);
4503         throw new Error("Parse error.");
4504       }
4505 }
4506
4507 /********************************************************
4508   Class: CSet
4509   *******************************************************/
4510 class CSet
4511 {
4512   /********************************************************
4513     Member Variables
4514     *******************************************************/
4515   private JavaLexBitSet m_set;
4516   private boolean m_complement;
4517
4518   /********************************************************
4519     Function: CSet
4520     *******************************************************/
4521   CSet
4522     (
4523      )
4524     {
4525       m_set = new JavaLexBitSet();
4526       m_complement = false;
4527     }
4528
4529   /********************************************************
4530     Function: complement
4531     *******************************************************/
4532   void complement
4533     (
4534      )
4535       {
4536         m_complement = true;
4537       }
4538
4539   /********************************************************
4540     Function: add
4541     *******************************************************/
4542   void add
4543     (
4544      int i
4545      )
4546       {
4547         m_set.set(i);
4548       }
4549
4550   /********************************************************
4551     Function: addncase
4552     *******************************************************/
4553   void addncase // add, ignoring case.
4554     (
4555      char c
4556      )
4557       {
4558         /* Do this in a Unicode-friendly way. */
4559         /* (note that duplicate adds have no effect) */
4560         add(c);
4561         add(Character.toLowerCase(c));
4562         add(Character.toTitleCase(c));
4563         add(Character.toUpperCase(c));
4564       }
4565
4566   /********************************************************
4567     Function: contains
4568     *******************************************************/
4569   boolean contains
4570     (
4571      int i
4572      )
4573       {
4574         boolean result;
4575
4576         result = m_set.get(i);
4577
4578         if (m_complement)
4579           {
4580             return (false == result);
4581           }
4582
4583         return result;
4584       }
4585
4586   /********************************************************
4587     Function: mimic
4588     *******************************************************/
4589   void mimic
4590     (
4591      CSet set
4592      )
4593       {
4594         m_complement = set.m_complement;
4595         m_set = (JavaLexBitSet) set.m_set.clone();
4596       }
4597 }
4598
4599 /********************************************************
4600   Class: CNfa
4601   *******************************************************/
4602 class CNfa
4603 {
4604   /********************************************************
4605     Member Variables
4606     *******************************************************/
4607   int m_edge;  /* Label for edge type:
4608                          character code,
4609                          CCL (character class),
4610                          [STATE,
4611                          SCL (state class),]
4612                          EMPTY,
4613                          EPSILON. */
4614
4615   CSet m_set;  /* Set to store character classes. */
4616   CNfa m_next;  /* Next state (or null if none). */
4617
4618   CNfa m_next2;  /* Another state with type == EPSILON
4619                            and null if not used.
4620                            The NFA construction should result in two
4621                            outgoing edges only if both are EPSILON edges. */
4622
4623   CAccept m_accept;  /* Set to null if nonaccepting state. */
4624   int m_anchor;  /* Says if and where pattern is anchored. */
4625
4626   int m_label;
4627
4628   JavaLexBitSet m_states;
4629
4630   /********************************************************
4631     Constants
4632     *******************************************************/
4633   static final int NO_LABEL = -1;
4634
4635   /********************************************************
4636     Constants: Edge Types
4637     Note: Edge transitions on one specific character
4638     are labelled with the character Ascii (Unicode)
4639     codes.  So none of the constants below should
4640     overlap with the natural character codes.
4641     *******************************************************/
4642   static final int CCL = -1;
4643   static final int EMPTY = -2;
4644   static final int EPSILON = -3;
4645
4646   /********************************************************
4647     Function: CNfa
4648     *******************************************************/
4649  CNfa
4650     (
4651      )
4652     {
4653       m_edge = EMPTY;
4654       m_set = null;
4655       m_next = null;
4656       m_next2 = null;
4657       m_accept = null;
4658       m_anchor = CSpec.NONE;
4659       m_label = NO_LABEL;
4660       m_states = null;
4661     }
4662
4663   /********************************************************
4664     Function: mimic
4665     Description: Converts this NFA state into a copy of
4666     the input one.
4667     *******************************************************/
4668   void mimic
4669     (
4670      CNfa nfa
4671      )
4672       {
4673         m_edge = nfa.m_edge;
4674
4675         if (null != nfa.m_set)
4676           {
4677             if (null == m_set)
4678               {
4679                 m_set = new CSet();
4680               }
4681             m_set.mimic(nfa.m_set);
4682           }
4683         else
4684           {
4685             m_set = null;
4686           }
4687
4688         m_next = nfa.m_next;
4689         m_next2 = nfa.m_next2;
4690         m_accept = nfa.m_accept;
4691         m_anchor = nfa.m_anchor;
4692
4693         if (null != nfa.m_states)
4694           {
4695             m_states = (JavaLexBitSet) nfa.m_states.clone();
4696           }
4697         else
4698           {
4699             m_states = null;
4700           }
4701       }
4702 }
4703
4704 /***************************************************************
4705   Class: CLexGen
4706   **************************************************************/
4707 class CLexGen
4708 {
4709   /***************************************************************
4710     Member Variables
4711     **************************************************************/
4712   private InputStream m_instream; /* JLex specification file. */
4713   private DataOutputStream m_outstream; /* Lexical analyzer source file. */
4714
4715   private CInput m_input; /* Input buffer class. */
4716
4717   private Hashtable m_tokens; /* Hashtable that maps characters to their
4718                                  corresponding lexical code for
4719                                  the internal lexical analyzer. */
4720   private CSpec m_spec; /* Spec class holds information
4721                            about the generated lexer. */
4722   private boolean m_init_flag; /* Flag set to true only upon
4723                                   successful initialization. */
4724
4725   private CMakeNfa m_makeNfa; /* NFA machine generator module. */
4726   private CNfa2Dfa m_nfa2dfa; /* NFA to DFA machine (transition table)
4727                                  conversion module. */
4728   private CMinimize m_minimize; /* Transition table compressor. */
4729   private CEmit m_emit; /* Output module that emits source code
4730                            into the generated lexer file. */
4731
4732
4733   /********************************************************
4734     Constants
4735     *******************************************************/
4736   private static final boolean ERROR = false;
4737   private static final boolean NOT_ERROR = true;
4738   private static final int BUFFER_SIZE = 1024;
4739
4740   /********************************************************
4741     Constants: Token Types
4742     *******************************************************/
4743   static final int EOS = 1;
4744   static final int ANY = 2;
4745   static final int AT_BOL = 3;
4746   static final int AT_EOL = 4;
4747   static final int CCL_END = 5;
4748   static final int CCL_START = 6;
4749   static final int CLOSE_CURLY = 7;
4750   static final int CLOSE_PAREN = 8;
4751   static final int CLOSURE = 9;
4752   static final int DASH = 10;
4753   static final int END_OF_INPUT = 11;
4754   static final int L = 12;
4755   static final int OPEN_CURLY = 13;
4756   static final int OPEN_PAREN = 14;
4757   static final int OPTIONAL = 15;
4758   static final int OR = 16;
4759   static final int PLUS_CLOSE = 17;
4760
4761   /***************************************************************
4762     Function: CLexGen
4763     **************************************************************/
4764   CLexGen
4765     (
4766      String filename
4767      )
4768       throws java.io.FileNotFoundException, java.io.IOException
4769       {
4770         /* Successful initialization flag. */
4771         m_init_flag = false;
4772
4773         /* Open input stream. */
4774         m_instream = new FileInputStream(filename);
4775         if (null == m_instream)
4776           {
4777             System.out.println("Error: Unable to open input file "
4778                                + filename + ".");
4779             return;
4780           }
4781
4782         /* Open output stream. */
4783         m_outstream
4784           = new DataOutputStream(new BufferedOutputStream(
4785                 new FileOutputStream(filename + ".java")));
4786         if (null == m_outstream)
4787           {
4788             System.out.println("Error: Unable to open output file "
4789                                + filename + ".java.");
4790             return;
4791           }
4792
4793         /* Create input buffer class. */
4794         m_input = new CInput(m_instream);
4795
4796         /* Initialize character hash table. */
4797         m_tokens = new Hashtable();
4798         m_tokens.put(new Character('$'),new Integer(AT_EOL));
4799         m_tokens.put(new Character('('),new Integer(OPEN_PAREN));
4800         m_tokens.put(new Character(')'),new Integer(CLOSE_PAREN));
4801         m_tokens.put(new Character('*'),new Integer(CLOSURE));
4802         m_tokens.put(new Character('+'),new Integer(PLUS_CLOSE));
4803         m_tokens.put(new Character('-'),new Integer(DASH));
4804         m_tokens.put(new Character('.'),new Integer(ANY));
4805         m_tokens.put(new Character('?'),new Integer(OPTIONAL));
4806         m_tokens.put(new Character('['),new Integer(CCL_START));
4807         m_tokens.put(new Character(']'),new Integer(CCL_END));
4808         m_tokens.put(new Character('^'),new Integer(AT_BOL));
4809         m_tokens.put(new Character('{'),new Integer(OPEN_CURLY));
4810         m_tokens.put(new Character('|'),new Integer(OR));
4811         m_tokens.put(new Character('}'),new Integer(CLOSE_CURLY));
4812
4813         /* Initialize spec structure. */
4814         m_spec = new CSpec(this);
4815
4816         /* Nfa to dfa converter. */
4817         m_nfa2dfa = new CNfa2Dfa();
4818         m_minimize = new CMinimize();
4819         m_makeNfa = new CMakeNfa();
4820
4821         m_emit = new CEmit();
4822
4823         /* Successful initialization flag. */
4824         m_init_flag = true;
4825       }
4826
4827   /***************************************************************
4828     Function: generate
4829     Description:
4830     **************************************************************/
4831   void generate
4832     (
4833      )
4834       throws java.io.IOException, java.io.FileNotFoundException
4835       {
4836         if (false == m_init_flag)
4837           {
4838             CError.parse_error(CError.E_INIT,0);
4839           }
4840
4841         if (CUtility.DEBUG)
4842           {
4843             CUtility.assert(null != this);
4844             CUtility.assert(null != m_outstream);
4845             CUtility.assert(null != m_input);
4846             CUtility.assert(null != m_tokens);
4847             CUtility.assert(null != m_spec);
4848             CUtility.assert(m_init_flag);
4849           }
4850
4851         /*m_emit.emit_imports(m_spec,m_outstream);*/
4852
4853         if (m_spec.m_verbose)
4854           {
4855             System.out.println("Processing first section -- user code.");
4856           }
4857         userCode();
4858         if (m_input.m_eof_reached)
4859           {
4860             CError.parse_error(CError.E_EOF,m_input.m_line_number);
4861           }
4862
4863         if (m_spec.m_verbose)
4864           {
4865             System.out.println("Processing second section -- "
4866                                + "JLex declarations.");
4867           }
4868         userDeclare();
4869         if (m_input.m_eof_reached)
4870           {
4871             CError.parse_error(CError.E_EOF,m_input.m_line_number);
4872           }
4873
4874         if (m_spec.m_verbose)
4875           {
4876             System.out.println("Processing third section -- lexical rules.");
4877           }
4878         userRules();
4879         if (CUtility.DO_DEBUG)
4880           {
4881             print_header();
4882           }
4883
4884         if (m_spec.m_verbose)
4885           {
4886             System.out.println("Outputting lexical analyzer code.");
4887           }
4888         m_emit.emit(m_spec,m_outstream);
4889
4890         if (m_spec.m_verbose && true == CUtility.OLD_DUMP_DEBUG)
4891           {
4892             details();
4893           }
4894
4895         m_outstream.close();
4896       }
4897
4898   /***************************************************************
4899     Function: userCode
4900     Description: Process first section of specification,
4901     echoing it into output file.
4902     **************************************************************/
4903   private void userCode
4904     (
4905      )
4906       throws java.io.IOException
4907       {
4908         int count = 0;
4909
4910         if (false == m_init_flag)
4911           {
4912             CError.parse_error(CError.E_INIT,0);
4913           }
4914
4915         if (CUtility.DEBUG)
4916           {
4917             CUtility.assert(null != this);
4918             CUtility.assert(null != m_outstream);
4919             CUtility.assert(null != m_input);
4920             CUtility.assert(null != m_tokens);
4921             CUtility.assert(null != m_spec);
4922           }
4923
4924         if (m_input.m_eof_reached)
4925           {
4926             CError.parse_error(CError.E_EOF,0);
4927           }
4928
4929         while (true)
4930           {
4931             if (m_input.getLine())
4932               {
4933                 /* Eof reached. */
4934                 CError.parse_error(CError.E_EOF,0);
4935               }
4936
4937             if (2 <= m_input.m_line_read
4938                 && '%' == m_input.m_line[0]
4939                 && '%' == m_input.m_line[1])
4940               {
4941                 /* Discard remainder of line. */
4942                 m_input.m_line_index = m_input.m_line_read;
4943                 return;
4944               }
4945
4946             m_outstream.writeBytes(new String(m_input.m_line,0,
4947                                               m_input.m_line_read));
4948           }
4949       }
4950
4951   /***************************************************************
4952     Function: getName
4953     **************************************************************/
4954   private char[] getName
4955     (
4956      )
4957       {
4958         char buffer[];
4959         int elem;
4960
4961         /* Skip white space. */
4962         while (m_input.m_line_index < m_input.m_line_read
4963                && true == CUtility.isspace(m_input.m_line[m_input.m_line_index]))
4964           {
4965             ++m_input.m_line_index;
4966           }
4967
4968         /* No name? */
4969         if (m_input.m_line_index >= m_input.m_line_read)
4970           {
4971             CError.parse_error(CError.E_DIRECT,0);
4972           }
4973
4974         /* Determine length. */
4975         elem = m_input.m_line_index;
4976         while (elem < m_input.m_line_read
4977                && false == CUtility.isnewline(m_input.m_line[elem]))
4978           {
4979             ++elem;
4980           }
4981
4982         /* Allocate non-terminated buffer of exact length. */
4983         buffer = new char[elem - m_input.m_line_index];
4984
4985         /* Copy. */
4986         elem = 0;
4987         while (m_input.m_line_index < m_input.m_line_read
4988                && false == CUtility.isnewline(m_input.m_line[m_input.m_line_index]))
4989           {
4990             buffer[elem] = m_input.m_line[m_input.m_line_index];
4991             ++elem;
4992             ++m_input.m_line_index;
4993           }
4994
4995         return buffer;
4996       }
4997
4998   private final int CLASS_CODE = 0;
4999   private final int INIT_CODE = 1;
5000   private final int EOF_CODE = 2;
5001   private final int INIT_THROW_CODE = 3;
5002   private final int YYLEX_THROW_CODE = 4;
5003   private final int EOF_THROW_CODE = 5;
5004   private final int EOF_VALUE_CODE = 6;
5005
5006   /***************************************************************
5007     Function: packCode
5008     Description:
5009     **************************************************************/
5010   private char[] packCode
5011     (
5012      char start_dir[],
5013      char end_dir[],
5014      char prev_code[],
5015      int prev_read,
5016      int specified
5017      )
5018       throws java.io.IOException
5019       {
5020         if (CUtility.DEBUG)
5021           {
5022             CUtility.assert(INIT_CODE == specified
5023                             || CLASS_CODE == specified
5024                             || EOF_CODE == specified
5025                             || EOF_VALUE_CODE == specified
5026                             || INIT_THROW_CODE == specified
5027                             || YYLEX_THROW_CODE == specified
5028                             || EOF_THROW_CODE == specified);
5029           }
5030
5031         if (0 != CUtility.charncmp(m_input.m_line,
5032                                    0,
5033                                    start_dir,
5034                                    0,
5035                                    start_dir.length - 1))
5036           {
5037             CError.parse_error(CError.E_INTERNAL,0);
5038           }
5039
5040         if (null == prev_code)
5041           {
5042             prev_code = new char[BUFFER_SIZE];
5043             prev_read = 0;
5044           }
5045
5046         if (prev_read >= prev_code.length)
5047           {
5048             prev_code = CUtility.doubleSize(prev_code);
5049           }
5050
5051         m_input.m_line_index = start_dir.length - 1;
5052         while (true)
5053           {
5054             while (m_input.m_line_index >= m_input.m_line_read)
5055               {
5056                 if (m_input.getLine())
5057                   {
5058                     CError.parse_error(CError.E_EOF,m_input.m_line_number);
5059                   }
5060
5061                 if (0 == CUtility.charncmp(m_input.m_line,
5062                                            0,
5063                                            end_dir,
5064                                            0,
5065                                            end_dir.length - 1))
5066                   {
5067                     m_input.m_line_index = end_dir.length - 1;
5068
5069                     switch (specified)
5070                       {
5071                       case CLASS_CODE:
5072                         m_spec.m_class_read = prev_read;
5073                         break;
5074
5075                       case INIT_CODE:
5076                         m_spec.m_init_read = prev_read;
5077                         break;
5078
5079                       case EOF_CODE:
5080                         m_spec.m_eof_read = prev_read;
5081                         break;
5082
5083                       case EOF_VALUE_CODE:
5084                         m_spec.m_eof_value_read = prev_read;
5085                         break;
5086
5087                       case INIT_THROW_CODE:
5088                         m_spec.m_init_throw_read = prev_read;
5089                         break;
5090
5091                       case YYLEX_THROW_CODE:
5092                         m_spec.m_yylex_throw_read = prev_read;
5093                         break;
5094
5095                       case EOF_THROW_CODE:
5096                         m_spec.m_eof_throw_read = prev_read;
5097                         break;
5098
5099                       default:
5100                         CError.parse_error(CError.E_INTERNAL,m_input.m_line_number);
5101                         break;
5102                       }
5103
5104                     return prev_code;
5105                   }
5106               }
5107
5108             while (m_input.m_line_index < m_input.m_line_read)
5109               {
5110                 prev_code[prev_read] = m_input.m_line[m_input.m_line_index];
5111                 ++prev_read;
5112                 ++m_input.m_line_index;
5113
5114                 if (prev_read >= prev_code.length)
5115                   {
5116                     prev_code = CUtility.doubleSize(prev_code);
5117                   }
5118               }
5119           }
5120       }
5121
5122   /***************************************************************
5123     Member Variables: JLex directives.
5124     **************************************************************/
5125   private char m_state_dir[] = {
5126     '%', 's', 't',
5127     'a', 't', 'e',
5128     '\0'
5129     };
5130
5131   private char m_char_dir[] = {
5132     '%', 'c', 'h',
5133     'a', 'r',
5134     '\0'
5135     };
5136
5137   private char m_line_dir[] = {
5138     '%', 'l', 'i',
5139     'n', 'e',
5140     '\0'
5141     };
5142
5143   private char m_cup_dir[] = {
5144     '%', 'c', 'u',
5145     'p',
5146     '\0'
5147     };
5148
5149   private char m_class_dir[] = {
5150     '%', 'c', 'l',
5151     'a', 's', 's',
5152     '\0'
5153     };
5154
5155   private char m_implements_dir[] = {
5156     '%', 'i', 'm', 'p', 'l', 'e', 'm', 'e', 'n', 't', 's',
5157     '\0'
5158     };
5159
5160   private char m_function_dir[] = {
5161     '%', 'f', 'u',
5162     'n', 'c', 't',
5163     'i', 'o', 'n',
5164     '\0'
5165     };
5166
5167   private char m_type_dir[] = {
5168     '%', 't', 'y',
5169     'p', 'e',
5170     '\0'
5171     };
5172
5173   private char m_integer_dir[] = {
5174     '%', 'i', 'n',
5175     't', 'e', 'g',
5176     'e', 'r',
5177     '\0'
5178     };
5179
5180   private char m_intwrap_dir[] = {
5181     '%', 'i', 'n',
5182     't', 'w', 'r',
5183     'a', 'p',
5184     '\0'
5185     };
5186
5187   private char m_full_dir[] = {
5188     '%', 'f', 'u',
5189     'l', 'l',
5190     '\0'
5191     };
5192
5193   private char m_unicode_dir[] = {
5194     '%', 'u', 'n',
5195     'i', 'c', 'o',
5196     'd', 'e',
5197     '\0'
5198     };
5199
5200   private char m_ignorecase_dir[] = {
5201     '%', 'i', 'g',
5202     'n', 'o', 'r',
5203     'e', 'c', 'a',
5204     's', 'e',
5205     '\0'
5206     };
5207
5208   private char m_notunix_dir[] = {
5209     '%', 'n', 'o',
5210     't', 'u', 'n',
5211     'i', 'x',
5212     '\0'
5213     };
5214
5215   private char m_init_code_dir[] = {
5216     '%', 'i', 'n',
5217     'i', 't', '{',
5218     '\0'
5219     };
5220
5221   private char m_init_code_end_dir[] = {
5222     '%', 'i', 'n',
5223     'i', 't', '}',
5224     '\0'
5225     };
5226
5227   private char m_init_throw_code_dir[] = {
5228     '%', 'i', 'n',
5229     'i', 't', 't',
5230     'h', 'r', 'o',
5231     'w', '{',
5232     '\0'
5233     };
5234
5235   private char m_init_throw_code_end_dir[] = {
5236     '%', 'i', 'n',
5237     'i', 't', 't',
5238     'h', 'r', 'o',
5239     'w', '}',
5240     '\0'
5241     };
5242
5243   private char m_yylex_throw_code_dir[] = {
5244     '%', 'y', 'y', 'l',
5245     'e', 'x', 't',
5246     'h', 'r', 'o',
5247     'w', '{',
5248     '\0'
5249     };
5250
5251   private char m_yylex_throw_code_end_dir[] = {
5252     '%', 'y', 'y', 'l',
5253     'e', 'x', 't',
5254     'h', 'r', 'o',
5255     'w', '}',
5256     '\0'
5257     };
5258
5259   private char m_eof_code_dir[] = {
5260     '%', 'e', 'o',
5261     'f', '{',
5262     '\0'
5263     };
5264
5265   private char m_eof_code_end_dir[] = {
5266     '%', 'e', 'o',
5267     'f', '}',
5268     '\0'
5269     };
5270
5271   private char m_eof_value_code_dir[] = {
5272     '%', 'e', 'o',
5273     'f', 'v', 'a',
5274     'l', '{',
5275     '\0'
5276     };
5277
5278   private char m_eof_value_code_end_dir[] = {
5279     '%', 'e', 'o',
5280     'f', 'v', 'a',
5281     'l', '}',
5282     '\0'
5283     };
5284
5285   private char m_eof_throw_code_dir[] = {
5286     '%', 'e', 'o',
5287     'f', 't', 'h',
5288     'r', 'o', 'w',
5289     '{',
5290     '\0'
5291     };
5292
5293   private char m_eof_throw_code_end_dir[] = {
5294     '%', 'e', 'o',
5295     'f', 't', 'h',
5296     'r', 'o', 'w',
5297     '}',
5298     '\0'
5299     };
5300
5301   private char m_class_code_dir[] = {
5302     '%', '{',
5303     '\0'
5304     };
5305
5306   private char m_class_code_end_dir[] = {
5307     '%', '}',
5308     '\0'
5309     };
5310
5311   private char m_yyeof_dir[] = {
5312     '%', 'y', 'y',
5313     'e', 'o', 'f',
5314     '\0'
5315     };
5316
5317   private char m_public_dir[] = {
5318     '%', 'p', 'u',
5319     'b', 'l', 'i',
5320     'c', '\0'
5321     };
5322
5323   /***************************************************************
5324     Function: userDeclare
5325     Description:
5326     **************************************************************/
5327   private void userDeclare
5328     (
5329      )
5330       throws java.io.IOException
5331         {
5332           int elem;
5333
5334           if (CUtility.DEBUG)
5335             {
5336               CUtility.assert(null != this);
5337               CUtility.assert(null != m_outstream);
5338               CUtility.assert(null != m_input);
5339               CUtility.assert(null != m_tokens);
5340               CUtility.assert(null != m_spec);
5341             }
5342
5343           if (m_input.m_eof_reached)
5344             {
5345               /* End-of-file. */
5346               CError.parse_error(CError.E_EOF,
5347                                  m_input.m_line_number);
5348             }
5349
5350           while (false == m_input.getLine())
5351             {
5352               /* Look for double percent. */
5353               if (2 <= m_input.m_line_read
5354                   && '%' == m_input.m_line[0]
5355                   && '%' == m_input.m_line[1])
5356                 {
5357                   /* Mess around with line. */
5358                   for (elem = 0; elem < m_input.m_line.length - 2; ++elem)
5359                     {
5360                       m_input.m_line[elem] = m_input.m_line[elem + 2];
5361                     }
5362                   m_input.m_line_read = m_input.m_line_read - 2;
5363
5364                   m_input.m_pushback_line = true;
5365                   /* Check for and discard empty line. */
5366                   if (0 == m_input.m_line_read
5367                       || '\r' == m_input.m_line[0]
5368                       || '\n' == m_input.m_line[0])
5369                     {
5370                       m_input.m_pushback_line = false;
5371                     }
5372
5373                   return;
5374                 }
5375
5376               if (0 == m_input.m_line_read)
5377                 {
5378                   continue;
5379                 }
5380
5381               if ('%' == m_input.m_line[0])
5382                 {
5383                   /* Special lex declarations. */
5384                   if (1 >= m_input.m_line_read)
5385                     {
5386                       CError.parse_error(CError.E_DIRECT,
5387                                          m_input.m_line_number);
5388                       continue;
5389                     }
5390
5391                   switch (m_input.m_line[1])
5392                     {
5393                     case '{':
5394                       if (0 == CUtility.charncmp(m_input.m_line,
5395                                                  0,
5396                                                  m_class_code_dir,
5397                                                  0,
5398                                                  m_class_code_dir.length - 1))
5399                         {
5400                           m_spec.m_class_code = packCode(m_class_code_dir,
5401                                                          m_class_code_end_dir,
5402                                                          m_spec.m_class_code,
5403                                                          m_spec.m_class_read,
5404                                                          CLASS_CODE);
5405                           break;
5406                         }
5407
5408                       /* Bad directive. */
5409                       CError.parse_error(CError.E_DIRECT,
5410                                          m_input.m_line_number);
5411                       break;
5412
5413                     case 'c':
5414                       if (0 == CUtility.charncmp(m_input.m_line,
5415                                                  0,
5416                                                  m_char_dir,
5417                                                  0,
5418                                                  m_char_dir.length - 1))
5419                         {
5420                           /* Set line counting to ON. */
5421                           m_input.m_line_index = m_char_dir.length;
5422                           m_spec.m_count_chars = true;
5423                           break;
5424                         }
5425                       else if (0 == CUtility.charncmp(m_input.m_line,
5426                                                       0,
5427                                                       m_class_dir,
5428                                                       0,
5429                                                       m_class_dir.length - 1))
5430                         {
5431                           m_input.m_line_index = m_class_dir.length;
5432                           m_spec.m_class_name = getName();
5433                           break;
5434                         }
5435                       else if (0 == CUtility.charncmp(m_input.m_line,
5436                                                       0,
5437                                                       m_cup_dir,
5438                                                       0,
5439                                                       m_cup_dir.length - 1))
5440                         {
5441                           /* Set Java CUP compatibility to ON. */
5442                           m_input.m_line_index = m_cup_dir.length;
5443                           m_spec.m_cup_compatible = true;
5444                           break;
5445                         }
5446
5447                       /* Bad directive. */
5448                       CError.parse_error(CError.E_DIRECT,
5449                                          m_input.m_line_number);
5450                       break;
5451
5452                     case 'e':
5453                       if (0 == CUtility.charncmp(m_input.m_line,
5454                                                  0,
5455                                                  m_eof_code_dir,
5456                                                  0,
5457                                                  m_eof_code_dir.length - 1))
5458                         {
5459                           m_spec.m_eof_code = packCode(m_eof_code_dir,
5460                                                        m_eof_code_end_dir,
5461                                                        m_spec.m_eof_code,
5462                                                        m_spec.m_eof_read,
5463                                                        EOF_CODE);
5464                           break;
5465                         }
5466                       else if (0 == CUtility.charncmp(m_input.m_line,
5467                                                       0,
5468                                                       m_eof_value_code_dir,
5469                                                       0,
5470                                                       m_eof_value_code_dir.length - 1))
5471                         {
5472                           m_spec.m_eof_value_code = packCode(m_eof_value_code_dir,
5473                                                              m_eof_value_code_end_dir,
5474                                                              m_spec.m_eof_value_code,
5475                                                              m_spec.m_eof_value_read,
5476                                                              EOF_VALUE_CODE);
5477                           break;
5478                         }
5479                       else if (0 == CUtility.charncmp(m_input.m_line,
5480                                                       0,
5481                                                       m_eof_throw_code_dir,
5482                                                       0,
5483                                                       m_eof_throw_code_dir.length - 1))
5484                         {
5485                           m_spec.m_eof_throw_code = packCode(m_eof_throw_code_dir,
5486                                                        m_eof_throw_code_end_dir,
5487                                                        m_spec.m_eof_throw_code,
5488                                                        m_spec.m_eof_throw_read,
5489                                                        EOF_THROW_CODE);
5490                           break;
5491                         }
5492
5493                       /* Bad directive. */
5494                       CError.parse_error(CError.E_DIRECT,
5495                                          m_input.m_line_number);
5496                       break;
5497
5498                     case 'f':
5499                       if (0 == CUtility.charncmp(m_input.m_line,
5500                                                  0,
5501                                                  m_function_dir,
5502                                                  0,
5503                                                  m_function_dir.length - 1))
5504                         {
5505                           /* Set line counting to ON. */
5506                           m_input.m_line_index = m_function_dir.length;
5507                           m_spec.m_function_name = getName();
5508                           break;
5509                         }
5510                       else if (0 == CUtility.charncmp(m_input.m_line,
5511                                                       0,
5512                                                       m_full_dir,
5513                                                       0,
5514                                                       m_full_dir.length - 1))
5515                         {
5516                           m_input.m_line_index = m_full_dir.length;
5517                           m_spec.m_dtrans_ncols = CUtility.MAX_EIGHT_BIT + 1;
5518                           break;
5519                         }
5520
5521                       /* Bad directive. */
5522                       CError.parse_error(CError.E_DIRECT,
5523                                          m_input.m_line_number);
5524                       break;
5525
5526                     case 'i':
5527                       if (0 == CUtility.charncmp(m_input.m_line,
5528                                                  0,
5529                                                  m_integer_dir,
5530                                                  0,
5531                                                  m_integer_dir.length - 1))
5532                         {
5533                           /* Set line counting to ON. */
5534                           m_input.m_line_index = m_integer_dir.length;
5535                           m_spec.m_integer_type = true;
5536                           break;
5537                         }
5538                       else if (0 == CUtility.charncmp(m_input.m_line,
5539                                                       0,
5540                                                       m_intwrap_dir,
5541                                                       0,
5542                                                       m_intwrap_dir.length - 1))
5543                         {
5544                           /* Set line counting to ON. */
5545                           m_input.m_line_index = m_integer_dir.length;
5546                           m_spec.m_intwrap_type = true;
5547                           break;
5548                         }
5549                       else if (0 == CUtility.charncmp(m_input.m_line,
5550                                                       0,
5551                                                       m_init_code_dir,
5552                                                       0,
5553                                                       m_init_code_dir.length - 1))
5554                         {
5555                           m_spec.m_init_code = packCode(m_init_code_dir,
5556                                                         m_init_code_end_dir,
5557                                                         m_spec.m_init_code,
5558                                                         m_spec.m_init_read,
5559                                                         INIT_CODE);
5560                           break;
5561                         }
5562                       else if (0 == CUtility.charncmp(m_input.m_line,
5563                                                       0,
5564                                                       m_init_throw_code_dir,
5565                                                       0,
5566                                                       m_init_throw_code_dir.length - 1))
5567                         {
5568                           m_spec.m_init_throw_code = packCode(m_init_throw_code_dir,
5569                                                        m_init_throw_code_end_dir,
5570                                                        m_spec.m_init_throw_code,
5571                                                        m_spec.m_init_throw_read,
5572                                                        INIT_THROW_CODE);
5573                           break;
5574                         }
5575                       else if (0 == CUtility.charncmp(m_input.m_line,
5576                                                       0,
5577                                                       m_implements_dir,
5578                                                       0,
5579                                                       m_implements_dir.length - 1))
5580                         {
5581                           m_input.m_line_index = m_implements_dir.length;
5582                           m_spec.m_implements_name = getName();
5583                           break;
5584                         }
5585                       else if (0 == CUtility.charncmp(m_input.m_line,
5586                                                       0,
5587                                                       m_ignorecase_dir,
5588                                                       0,
5589                                                       m_ignorecase_dir.length-1))
5590                         {
5591                           /* Set m_ignorecase to ON. */
5592                           m_input.m_line_index = m_ignorecase_dir.length;
5593                           m_spec.m_ignorecase = true;
5594                           break;
5595                         }
5596
5597                       /* Bad directive. */
5598                       CError.parse_error(CError.E_DIRECT,
5599                                          m_input.m_line_number);
5600                       break;
5601
5602                     case 'l':
5603                       if (0 == CUtility.charncmp(m_input.m_line,
5604                                                  0,
5605                                                  m_line_dir,
5606                                                  0,
5607                                                  m_line_dir.length - 1))
5608                         {
5609                           /* Set line counting to ON. */
5610                           m_input.m_line_index = m_line_dir.length;
5611                           m_spec.m_count_lines = true;
5612                           break;
5613                         }
5614
5615                       /* Bad directive. */
5616                       CError.parse_error(CError.E_DIRECT,
5617                                          m_input.m_line_number);
5618                       break;
5619
5620                     case 'n':
5621                       if (0 == CUtility.charncmp(m_input.m_line,
5622                                                  0,
5623                                                  m_notunix_dir,
5624                                                  0,
5625                                                  m_notunix_dir.length - 1))
5626                         {
5627                           /* Set line counting to ON. */
5628                           m_input.m_line_index = m_notunix_dir.length;
5629                           m_spec.m_unix = false;
5630                           break;
5631                         }
5632
5633                       /* Bad directive. */
5634                       CError.parse_error(CError.E_DIRECT,
5635                                          m_input.m_line_number);
5636                       break;
5637
5638                     case 'p':
5639                       if (0 == CUtility.charncmp(m_input.m_line,
5640                                                  0,
5641                                                  m_public_dir,
5642                                                  0,
5643                                                  m_public_dir.length - 1))
5644                         {
5645                           /* Set public flag. */
5646                           m_input.m_line_index = m_public_dir.length;
5647                           m_spec.m_public = true;
5648                           break;
5649                         }
5650
5651                       /* Bad directive. */
5652                       CError.parse_error(CError.E_DIRECT,
5653                                          m_input.m_line_number);
5654                       break;
5655
5656                     case 's':
5657                       if (0 == CUtility.charncmp(m_input.m_line,
5658                                                  0,
5659                                                  m_state_dir,
5660                                                  0,
5661                                                  m_state_dir.length - 1))
5662                         {
5663                           /* Recognize state list. */
5664                           m_input.m_line_index = m_state_dir.length;
5665                           saveStates();
5666                           break;
5667                         }
5668
5669                       /* Undefined directive. */
5670                       CError.parse_error(CError.E_DIRECT,
5671                                          m_input.m_line_number);
5672                       break;
5673
5674                     case 't':
5675                       if (0 == CUtility.charncmp(m_input.m_line,
5676                                                  0,
5677                                                  m_type_dir,
5678                                                  0,
5679                                                  m_type_dir.length - 1))
5680                         {
5681                           /* Set Java CUP compatibility to ON. */
5682                           m_input.m_line_index = m_type_dir.length;
5683                           m_spec.m_type_name = getName();
5684                           break;
5685                         }
5686
5687                       /* Undefined directive. */
5688                       CError.parse_error(CError.E_DIRECT,
5689                                          m_input.m_line_number);
5690                       break;
5691
5692                     case 'u':
5693                       if (0 == CUtility.charncmp(m_input.m_line,
5694                                                  0,
5695                                                  m_unicode_dir,
5696                                                  0,
5697                                                  m_unicode_dir.length - 1))
5698                         {
5699                           m_input.m_line_index = m_unicode_dir.length;
5700                           /* UNDONE: What to do here? */
5701                           break;
5702                         }
5703
5704                       /* Bad directive. */
5705                       CError.parse_error(CError.E_DIRECT,
5706                                          m_input.m_line_number);
5707                       break;
5708
5709                     case 'y':
5710                       if (0 == CUtility.charncmp(m_input.m_line,
5711                                                  0,
5712                                                  m_yyeof_dir,
5713                                                  0,
5714                                                  m_yyeof_dir.length - 1))
5715                         {
5716                           m_input.m_line_index = m_yyeof_dir.length;
5717                           m_spec.m_yyeof = true;
5718                           break;
5719                         } else if (0 == CUtility.charncmp(m_input.m_line,
5720                                                           0,
5721                                                           m_yylex_throw_code_dir,
5722                                                           0,
5723                                                           m_yylex_throw_code_dir.length - 1))
5724                         {
5725                           m_spec.m_yylex_throw_code = packCode(m_yylex_throw_code_dir,
5726                                                                m_yylex_throw_code_end_dir,
5727                                                        m_spec.m_yylex_throw_code,
5728                                                        m_spec.m_yylex_throw_read,
5729                                                        YYLEX_THROW_CODE);
5730                           break;
5731                         }
5732
5733
5734                       /* Bad directive. */
5735                       CError.parse_error(CError.E_DIRECT,
5736                                          m_input.m_line_number);
5737                       break;
5738
5739                     default:
5740                       /* Undefined directive. */
5741                       CError.parse_error(CError.E_DIRECT,
5742                                          m_input.m_line_number);
5743                       break;
5744                     }
5745                 }
5746               else
5747                 {
5748                   /* Regular expression macro. */
5749                   m_input.m_line_index = 0;
5750                   saveMacro();
5751                 }
5752
5753               if (CUtility.OLD_DEBUG)
5754                 {
5755                   System.out.println("Line number "
5756                                      + m_input.m_line_number + ":");
5757                   System.out.print(new String(m_input.m_line,
5758                                               0,m_input.m_line_read));
5759                 }
5760             }
5761         }
5762
5763   /***************************************************************
5764     Function: userRules
5765     Description: Processes third section of JLex
5766     specification and creates minimized transition table.
5767     **************************************************************/
5768   private void userRules
5769     (
5770      )
5771       throws java.io.IOException
5772       {
5773         int code;
5774
5775         if (false == m_init_flag)
5776           {
5777             CError.parse_error(CError.E_INIT,0);
5778           }
5779
5780         if (CUtility.DEBUG)
5781           {
5782             CUtility.assert(null != this);
5783             CUtility.assert(null != m_outstream);
5784             CUtility.assert(null != m_input);
5785             CUtility.assert(null != m_tokens);
5786             CUtility.assert(null != m_spec);
5787           }
5788
5789         /* UNDONE: Need to handle states preceding rules. */
5790
5791         if (m_spec.m_verbose)
5792           {
5793             System.out.println("Creating NFA machine representation.");
5794           }
5795         m_makeNfa.thompson(this,m_spec,m_input);
5796
5797         /*print_nfa();*/
5798
5799         if (CUtility.DEBUG)
5800           {
5801             CUtility.assert(END_OF_INPUT == m_spec.m_current_token);
5802           }
5803
5804         if (m_spec.m_verbose)
5805           {
5806             System.out.println("Creating DFA transition table.");
5807           }
5808         m_nfa2dfa.make_dfa(this,m_spec);
5809
5810         if (CUtility.FOODEBUG) {
5811           print_header();
5812         }
5813
5814         if (m_spec.m_verbose)
5815           {
5816             System.out.println("Minimizing DFA transition table.");
5817           }
5818         m_minimize.min_dfa(m_spec);
5819       }
5820
5821   /***************************************************************
5822     Function: printccl
5823     Description: Debuggng routine that outputs readable form
5824     of character class.
5825     **************************************************************/
5826   private void printccl
5827     (
5828      CSet set
5829      )
5830       {
5831         int i;
5832
5833         System.out.print(" [");
5834         for (i = 0; i < m_spec.m_dtrans_ncols; ++i)
5835           {
5836             if (set.contains(i))
5837               {
5838                 System.out.print(interp_int(i));
5839               }
5840           }
5841         System.out.print(']');
5842       }
5843
5844   /***************************************************************
5845     Function: plab
5846     Description:
5847     **************************************************************/
5848   private String plab
5849     (
5850      CNfa state
5851      )
5852       {
5853         int index;
5854
5855         if (null == state)
5856           {
5857             return (new String("--"));
5858           }
5859
5860         index = m_spec.m_nfa_states.indexOf(state);
5861
5862         return ((new Integer(index)).toString());
5863       }
5864
5865   /***************************************************************
5866     Function: interp_int
5867     Description:
5868     **************************************************************/
5869   private String interp_int
5870     (
5871      int i
5872      )
5873       {
5874         switch (i)
5875           {
5876           case (int) '\b':
5877             return (new String("\\b"));
5878
5879           case (int) '\t':
5880             return (new String("\\t"));
5881
5882           case (int) '\n':
5883             return (new String("\\n"));
5884
5885           case (int) '\f':
5886             return (new String("\\f"));
5887
5888           case (int) '\r':
5889             return (new String("\\r"));
5890
5891           case (int) ' ':
5892             return (new String("\\ "));
5893
5894           default:
5895             return ((new Character((char) i)).toString());
5896           }
5897       }
5898
5899   /***************************************************************
5900     Function: print_nfa
5901     Description:
5902     **************************************************************/
5903   void print_nfa
5904     (
5905      )
5906       {
5907         int elem;
5908         CNfa nfa;
5909         int size;
5910         Enumeration states;
5911         Integer index;
5912         int i;
5913         int j;
5914         int vsize;
5915         String state;
5916
5917         System.out.println("--------------------- NFA -----------------------");
5918
5919         size = m_spec.m_nfa_states.size();
5920         for (elem = 0; elem < size; ++elem)
5921           {
5922             nfa = (CNfa) m_spec.m_nfa_states.elementAt(elem);
5923
5924             System.out.print("Nfa state " + plab(nfa) + ": ");
5925
5926             if (null == nfa.m_next)
5927               {
5928                 System.out.print("(TERMINAL)");
5929               }
5930             else
5931               {
5932                 System.out.print("--> " + plab(nfa.m_next));
5933                 System.out.print("--> " + plab(nfa.m_next2));
5934
5935                 switch (nfa.m_edge)
5936                   {
5937                   case CNfa.CCL:
5938                     printccl(nfa.m_set);
5939                     break;
5940
5941                   case CNfa.EPSILON:
5942                     System.out.print(" EPSILON ");
5943                     break;
5944
5945                   default:
5946                     System.out.print(" " + interp_int(nfa.m_edge));
5947                     break;
5948                   }
5949               }
5950
5951             if (0 == elem)
5952               {
5953                 System.out.print(" (START STATE)");
5954               }
5955
5956             if (null != nfa.m_accept)
5957               {
5958                 System.out.print(" accepting "
5959                                  + ((0 != (nfa.m_anchor & CSpec.START)) ? "^" : "")
5960                                  + "<"
5961                                  + (new String(nfa.m_accept.m_action,0,
5962                                                nfa.m_accept.m_action_read))
5963                                  + ">"
5964                                  + ((0 != (nfa.m_anchor & CSpec.END)) ? "$" : ""));
5965               }
5966
5967             System.out.println("");
5968           }
5969
5970         states = m_spec.m_states.keys();
5971         while (states.hasMoreElements())
5972           {
5973             state = (String) states.nextElement();
5974             index = (Integer) m_spec.m_states.get(state);
5975
5976             if (CUtility.DEBUG)
5977               {
5978                 CUtility.assert(null != state);
5979                 CUtility.assert(null != index);
5980               }
5981
5982             System.out.println("State \"" + state
5983                                + "\" has identifying index "
5984                                + index.toString() + ".");
5985             System.out.print("\tStart states of matching rules: ");
5986
5987             i = index.intValue();
5988             vsize = m_spec.m_state_rules[i].size();
5989
5990             for (j = 0; j < vsize; ++j)
5991               {
5992                 nfa = (CNfa) m_spec.m_state_rules[i].elementAt(j);
5993
5994                 System.out.print(m_spec.m_nfa_states.indexOf(nfa) + " ");
5995               }
5996
5997             System.out.println("");
5998           }
5999
6000         System.out.println("-------------------- NFA ----------------------");
6001       }
6002
6003   /***************************************************************
6004     Function: getStates
6005     Description: Parses the state area of a rule,
6006     from the beginning of a line.
6007     < state1, state2 ... > regular_expression { action }
6008     Returns null on only EOF.  Returns all_states,
6009     initialied properly to correspond to all states,
6010     if no states are found.
6011     Special Notes: This function treats commas as optional
6012     and permits states to be spread over multiple lines.
6013     **************************************************************/
6014   private JavaLexBitSet all_states = null;
6015   JavaLexBitSet getStates
6016     (
6017      )
6018       throws java.io.IOException
6019       {
6020         int start_state;
6021         int count_state;
6022         JavaLexBitSet states;
6023         String name;
6024         Integer index;
6025         int i;
6026         int size;
6027
6028         if (CUtility.DEBUG)
6029           {
6030             CUtility.assert(null != this);
6031             CUtility.assert(null != m_outstream);
6032             CUtility.assert(null != m_input);
6033             CUtility.assert(null != m_tokens);
6034             CUtility.assert(null != m_spec);
6035           }
6036
6037         states = null;
6038
6039         /* Skip white space. */
6040         while (CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6041           {
6042             ++m_input.m_line_index;
6043
6044             while (m_input.m_line_index >= m_input.m_line_read)
6045               {
6046                 /* Must just be an empty line. */
6047                 if (m_input.getLine())
6048                   {
6049                     /* EOF found. */
6050                     return null;
6051                   }
6052               }
6053           }
6054
6055         /* Look for states. */
6056         if ('<' == m_input.m_line[m_input.m_line_index])
6057           {
6058             ++m_input.m_line_index;
6059
6060             states = new JavaLexBitSet();
6061
6062             /* Parse states. */
6063             while (true)
6064               {
6065                 /* We may have reached the end of the line. */
6066                 while (m_input.m_line_index >= m_input.m_line_read)
6067                   {
6068                     if (m_input.getLine())
6069                       {
6070                         /* EOF found. */
6071                         CError.parse_error(CError.E_EOF,m_input.m_line_number);
6072                         return states;
6073                       }
6074                   }
6075
6076                 while (true)
6077                   {
6078                     /* Skip white space. */
6079                     while (CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6080                       {
6081                         ++m_input.m_line_index;
6082
6083                         while (m_input.m_line_index >= m_input.m_line_read)
6084                           {
6085                             if (m_input.getLine())
6086                               {
6087                                 /* EOF found. */
6088                                 CError.parse_error(CError.E_EOF,m_input.m_line_number);
6089                                 return states;
6090                               }
6091                           }
6092                       }
6093
6094                     if (',' != m_input.m_line[m_input.m_line_index])
6095                       {
6096                         break;
6097                       }
6098
6099                     ++m_input.m_line_index;
6100                   }
6101
6102                 if ('>' == m_input.m_line[m_input.m_line_index])
6103                   {
6104                     ++m_input.m_line_index;
6105                     if (m_input.m_line_index < m_input.m_line_read)
6106                       {
6107                         m_advance_stop = true;
6108                       }
6109                     return states;
6110                   }
6111
6112                 /* Read in state name. */
6113                 start_state = m_input.m_line_index;
6114                 while (false == CUtility.isspace(m_input.m_line[m_input.m_line_index])
6115                        && ',' != m_input.m_line[m_input.m_line_index]
6116                        && '>' != m_input.m_line[m_input.m_line_index])
6117                   {
6118                     ++m_input.m_line_index;
6119
6120                     if (m_input.m_line_index >= m_input.m_line_read)
6121                       {
6122                         /* End of line means end of state name. */
6123                         break;
6124                       }
6125                   }
6126                 count_state = m_input.m_line_index - start_state;
6127
6128                 /* Save name after checking definition. */
6129                 name = new String(m_input.m_line,
6130                                   start_state,
6131                                   count_state);
6132                 index = (Integer) m_spec.m_states.get(name);
6133                 if (null == index)
6134                   {
6135                     /* Uninitialized state. */
6136                     System.out.println("Uninitialized State Name: " + name);
6137                     CError.parse_error(CError.E_STATE,m_input.m_line_number);
6138                   }
6139                 states.set(index.intValue());
6140               }
6141           }
6142
6143         if (null == all_states)
6144           {
6145             all_states = new JavaLexBitSet();
6146
6147             size = m_spec.m_states.size();
6148             for (i = 0; i < size; ++i)
6149               {
6150                 all_states.set(i);
6151               }
6152           }
6153
6154         if (m_input.m_line_index < m_input.m_line_read)
6155           {
6156             m_advance_stop = true;
6157           }
6158         return all_states;
6159       }
6160
6161   /********************************************************
6162     Function: expandMacro
6163     Description: Returns false on error, true otherwise.
6164     *******************************************************/
6165   private boolean expandMacro
6166     (
6167      )
6168       {
6169         int elem;
6170         int start_macro;
6171         int end_macro;
6172         int start_name;
6173         int count_name;
6174         String def;
6175         int def_elem;
6176         String name;
6177         char replace[];
6178         int rep_elem;
6179
6180         if (CUtility.DEBUG)
6181           {
6182             CUtility.assert(null != this);
6183             CUtility.assert(null != m_outstream);
6184             CUtility.assert(null != m_input);
6185             CUtility.assert(null != m_tokens);
6186             CUtility.assert(null != m_spec);
6187           }
6188
6189         /* Check for macro. */
6190         if ('{' != m_input.m_line[m_input.m_line_index])
6191           {
6192             CError.parse_error(CError.E_INTERNAL,m_input.m_line_number);
6193             return ERROR;
6194           }
6195
6196         start_macro = m_input.m_line_index;
6197         elem = m_input.m_line_index + 1;
6198         if (elem >= m_input.m_line_read)
6199           {
6200             CError.impos("Unfinished macro name");
6201             return ERROR;
6202           }
6203
6204         /* Get macro name. */
6205         start_name = elem;
6206         while ('}' != m_input.m_line[elem])
6207           {
6208             ++elem;
6209             if (elem >= m_input.m_line_read)
6210               {
6211                 CError.impos("Unfinished macro name at line "
6212                              + m_input.m_line_number);
6213                 return ERROR;
6214               }
6215           }
6216         count_name = elem - start_name;
6217         end_macro = elem;
6218
6219         /* Check macro name. */
6220         if (0 == count_name)
6221           {
6222             CError.impos("Nonexistent macro name");
6223             return ERROR;
6224           }
6225
6226         /* Debug checks. */
6227         if (CUtility.DEBUG)
6228           {
6229             CUtility.assert(0 < count_name);
6230           }
6231
6232         /* Retrieve macro definition. */
6233         name = new String(m_input.m_line,start_name,count_name);
6234         def = (String) m_spec.m_macros.get(name);
6235         if (null == def)
6236           {
6237             /*CError.impos("Undefined macro \"" + name + "\".");*/
6238             System.out.println("Error: Undefined macro \"" + name + "\".");
6239             CError.parse_error(CError.E_NOMAC, m_input.m_line_number);
6240             return ERROR;
6241           }
6242         if (CUtility.OLD_DUMP_DEBUG)
6243           {
6244             System.out.println("expanded escape: " + def);
6245           }
6246
6247         /* Replace macro in new buffer,
6248            beginning by copying first part of line buffer. */
6249         replace = new char[m_input.m_line.length];
6250         for (rep_elem = 0; rep_elem < start_macro; ++rep_elem)
6251           {
6252             replace[rep_elem] = m_input.m_line[rep_elem];
6253
6254             if (CUtility.DEBUG)
6255               {
6256                 CUtility.assert(rep_elem < replace.length);
6257               }
6258           }
6259
6260         /* Copy macro definition. */
6261         if (rep_elem >= replace.length)
6262           {
6263             replace = CUtility.doubleSize(replace);
6264           }
6265         for (def_elem = 0; def_elem < def.length(); ++def_elem)
6266           {
6267             replace[rep_elem] = def.charAt(def_elem);
6268
6269             ++rep_elem;
6270             if (rep_elem >= replace.length)
6271               {
6272                 replace = CUtility.doubleSize(replace);
6273               }
6274           }
6275
6276         /* Copy last part of line. */
6277         if (rep_elem >= replace.length)
6278           {
6279             replace = CUtility.doubleSize(replace);
6280           }
6281         for (elem = end_macro + 1; elem < m_input.m_line_read; ++elem)
6282           {
6283             replace[rep_elem] = m_input.m_line[elem];
6284
6285             ++rep_elem;
6286             if (rep_elem >= replace.length)
6287               {
6288                 replace = CUtility.doubleSize(replace);
6289               }
6290           }
6291
6292         /* Replace buffer. */
6293         m_input.m_line = replace;
6294         m_input.m_line_read = rep_elem;
6295
6296         if (CUtility.OLD_DEBUG)
6297           {
6298             System.out.println(new String(m_input.m_line,0,m_input.m_line_read));
6299           }
6300         return NOT_ERROR;
6301       }
6302
6303   /***************************************************************
6304     Function: saveMacro
6305     Description: Saves macro definition of form:
6306     macro_name = macro_definition
6307     **************************************************************/
6308   private void saveMacro
6309     (
6310      )
6311       {
6312         int elem;
6313         int start_name;
6314         int count_name;
6315         int start_def;
6316         int count_def;
6317         boolean saw_escape;
6318         boolean in_quote;
6319         boolean in_ccl;
6320
6321         if (CUtility.DEBUG)
6322           {
6323             CUtility.assert(null != this);
6324             CUtility.assert(null != m_outstream);
6325             CUtility.assert(null != m_input);
6326             CUtility.assert(null != m_tokens);
6327             CUtility.assert(null != m_spec);
6328           }
6329
6330         /* Macro declarations are of the following form:
6331            macro_name macro_definition */
6332
6333         elem = 0;
6334
6335         /* Skip white space preceding macro name. */
6336         while (CUtility.isspace(m_input.m_line[elem]))
6337           {
6338             ++elem;
6339             if (elem >= m_input.m_line_read)
6340               {
6341                 /* End of line has been reached,
6342                    and line was found to be empty. */
6343                 return;
6344               }
6345           }
6346
6347         /* Read macro name. */
6348         start_name = elem;
6349         while (false == CUtility.isspace(m_input.m_line[elem])
6350                && '=' != m_input.m_line[elem])
6351           {
6352             ++elem;
6353             if (elem >= m_input.m_line_read)
6354               {
6355                 /* Macro name but no associated definition. */
6356                 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6357               }
6358           }
6359         count_name = elem - start_name;
6360
6361         /* Check macro name. */
6362         if (0 == count_name)
6363           {
6364             /* Nonexistent macro name. */
6365             CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6366           }
6367
6368         /* Skip white space between name and definition. */
6369         while (CUtility.isspace(m_input.m_line[elem]))
6370           {
6371             ++elem;
6372             if (elem >= m_input.m_line_read)
6373               {
6374                 /* Macro name but no associated definition. */
6375                 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6376               }
6377           }
6378
6379         if ('=' == m_input.m_line[elem])
6380           {
6381             ++elem;
6382             if (elem >= m_input.m_line_read)
6383               {
6384                 /* Macro name but no associated definition. */
6385                 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6386               }
6387           }
6388
6389         /* Skip white space between name and definition. */
6390         while (CUtility.isspace(m_input.m_line[elem]))
6391           {
6392             ++elem;
6393             if (elem >= m_input.m_line_read)
6394               {
6395                 /* Macro name but no associated definition. */
6396                 CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6397               }
6398           }
6399
6400         /* Read macro definition. */
6401         start_def = elem;
6402         in_quote = false;
6403         in_ccl = false;
6404         saw_escape = false;
6405         while (false == CUtility.isspace(m_input.m_line[elem])
6406                || true == in_quote
6407                || true == in_ccl
6408                || true == saw_escape)
6409           {
6410             if ('\"' == m_input.m_line[elem] && false == saw_escape)
6411               {
6412                 in_quote = !in_quote;
6413               }
6414
6415             if ('\\' == m_input.m_line[elem] && false == saw_escape)
6416               {
6417                 saw_escape = true;
6418               }
6419             else
6420               {
6421                 saw_escape = false;
6422               }
6423             if (false == saw_escape && false == in_quote) { // CSA, 24-jul-99
6424               if ('[' == m_input.m_line[elem] && false == in_ccl)
6425                 in_ccl = true;
6426               if (']' == m_input.m_line[elem] && true == in_ccl)
6427                 in_ccl = false;
6428             }
6429
6430             ++elem;
6431             if (elem >= m_input.m_line_read)
6432               {
6433                 /* End of line. */
6434                 break;
6435               }
6436           }
6437         count_def = elem - start_def;
6438
6439         /* Check macro definition. */
6440         if (0 == count_def)
6441           {
6442             /* Nonexistent macro name. */
6443             CError.parse_error(CError.E_MACDEF,m_input.m_line_number);
6444           }
6445
6446         /* Debug checks. */
6447         if (CUtility.DEBUG)
6448           {
6449             CUtility.assert(0 < count_def);
6450             CUtility.assert(0 < count_name);
6451             CUtility.assert(null != m_spec.m_macros);
6452           }
6453
6454         if (CUtility.OLD_DEBUG)
6455           {
6456             System.out.println("macro name \""
6457                                + new String(m_input.m_line,start_name,count_name)
6458                                + "\".");
6459             System.out.println("macro definition \""
6460                                + new String(m_input.m_line,start_def,count_def)
6461                                + "\".");
6462           }
6463
6464         /* Add macro name and definition to table. */
6465         m_spec.m_macros.put(new String(m_input.m_line,start_name,count_name),
6466                             new String(m_input.m_line,start_def,count_def));
6467       }
6468
6469   /***************************************************************
6470     Function: saveStates
6471     Description: Takes state declaration and makes entries
6472     for them in state hashtable in CSpec structure.
6473     State declaration should be of the form:
6474     %state name0[, name1, name2 ...]
6475     (But commas are actually optional as long as there is
6476     white space in between them.)
6477     **************************************************************/
6478   private void saveStates
6479     (
6480      )
6481       {
6482         int start_state;
6483         int count_state;
6484
6485         if (CUtility.DEBUG)
6486           {
6487             CUtility.assert(null != this);
6488             CUtility.assert(null != m_outstream);
6489             CUtility.assert(null != m_input);
6490             CUtility.assert(null != m_tokens);
6491             CUtility.assert(null != m_spec);
6492           }
6493
6494         /* EOF found? */
6495         if (m_input.m_eof_reached)
6496           {
6497             return;
6498           }
6499
6500         /* Debug checks. */
6501         if (CUtility.DEBUG)
6502           {
6503             CUtility.assert('%' == m_input.m_line[0]);
6504             CUtility.assert('s' == m_input.m_line[1]);
6505             CUtility.assert(m_input.m_line_index <= m_input.m_line_read);
6506             CUtility.assert(0 <= m_input.m_line_index);
6507             CUtility.assert(0 <= m_input.m_line_read);
6508           }
6509
6510         /* Blank line?  No states? */
6511         if (m_input.m_line_index >= m_input.m_line_read)
6512           {
6513             return;
6514           }
6515
6516         while (m_input.m_line_index < m_input.m_line_read)
6517           {
6518             if (CUtility.OLD_DEBUG)
6519               {
6520                 System.out.println("line read " + m_input.m_line_read
6521                                    + "\tline index = " + m_input.m_line_index);
6522               }
6523
6524             /* Skip white space. */
6525             while (CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6526               {
6527                 ++m_input.m_line_index;
6528                 if (m_input.m_line_index >= m_input.m_line_read)
6529                   {
6530                     /* No more states to be found. */
6531                     return;
6532                   }
6533               }
6534
6535             /* Look for state name. */
6536             start_state = m_input.m_line_index;
6537             while (false == CUtility.isspace(m_input.m_line[m_input.m_line_index])
6538                    && ',' != m_input.m_line[m_input.m_line_index])
6539               {
6540                 ++m_input.m_line_index;
6541                 if (m_input.m_line_index >= m_input.m_line_read)
6542                   {
6543                     /* End of line and end of state name. */
6544                     break;
6545                   }
6546               }
6547             count_state = m_input.m_line_index - start_state;
6548
6549             if (CUtility.OLD_DEBUG)
6550               {
6551                 System.out.println("State name \""
6552                                    + new String(m_input.m_line,start_state,count_state)
6553                                    + "\".");
6554                 System.out.println("Integer index \""
6555                                    + m_spec.m_states.size()
6556                                    + "\".");
6557               }
6558
6559             /* Enter new state name, along with unique index. */
6560             m_spec.m_states.put(new String(m_input.m_line,start_state,count_state),
6561                                 new Integer(m_spec.m_states.size()));
6562
6563             /* Skip comma. */
6564             if (',' == m_input.m_line[m_input.m_line_index])
6565               {
6566                 ++m_input.m_line_index;
6567                 if (m_input.m_line_index >= m_input.m_line_read)
6568                   {
6569                     /* End of line. */
6570                     return;
6571                   }
6572               }
6573           }
6574       }
6575
6576   /********************************************************
6577     Function: expandEscape
6578     Description: Takes escape sequence and returns
6579     corresponding character code.
6580     *******************************************************/
6581   private char expandEscape
6582     (
6583      )
6584       {
6585         char r;
6586
6587         /* Debug checks. */
6588         if (CUtility.DEBUG)
6589           {
6590             CUtility.assert(m_input.m_line_index < m_input.m_line_read);
6591             CUtility.assert(0 < m_input.m_line_read);
6592             CUtility.assert(0 <= m_input.m_line_index);
6593           }
6594
6595         if ('\\' != m_input.m_line[m_input.m_line_index])
6596           {
6597             ++m_input.m_line_index;
6598             return m_input.m_line[m_input.m_line_index - 1];
6599           }
6600         else
6601           {
6602             ++m_input.m_line_index;
6603             switch (CUtility.toupper(m_input.m_line[m_input.m_line_index]))
6604               {
6605               case 'B':
6606                 ++m_input.m_line_index;
6607                 return '\b';
6608
6609               case 'T':
6610                 ++m_input.m_line_index;
6611                 return '\t';
6612
6613               case 'N':
6614                 ++m_input.m_line_index;
6615                 return '\n';
6616
6617               case 'F':
6618                 ++m_input.m_line_index;
6619                 return '\f';
6620
6621               case 'R':
6622                 ++m_input.m_line_index;
6623                 return '\r';
6624
6625               case '^':
6626                 ++m_input.m_line_index;
6627                 r = (char) (CUtility.toupper(m_input.m_line[m_input.m_line_index])
6628                      - '@');
6629                 ++m_input.m_line_index;
6630                 return r;
6631
6632               case 'X':
6633                 ++m_input.m_line_index;
6634                 r = 0;
6635                 if (CUtility.ishexdigit(m_input.m_line[m_input.m_line_index]))
6636                   {
6637                     r = CUtility.hex2bin(m_input.m_line[m_input.m_line_index]);
6638                     ++m_input.m_line_index;
6639                   }
6640                 if (CUtility.ishexdigit(m_input.m_line[m_input.m_line_index]))
6641                   {
6642                     r = (char) (r << 4);
6643                     r = (char) (r | CUtility.hex2bin(m_input.m_line[m_input.m_line_index]));
6644                     ++m_input.m_line_index;
6645                   }
6646                 if (CUtility.ishexdigit(m_input.m_line[m_input.m_line_index]))
6647                   {
6648                     r = (char) (r << 4);
6649                     r = (char) (r | CUtility.hex2bin(m_input.m_line[m_input.m_line_index]));
6650                     ++m_input.m_line_index;
6651                   }
6652                 return r;
6653
6654               default:
6655                 if (false == CUtility.isoctdigit(m_input.m_line[m_input.m_line_index]))
6656                   {
6657                     r = m_input.m_line[m_input.m_line_index];
6658                     ++m_input.m_line_index;
6659                   }
6660                 else
6661                   {
6662                     r = CUtility.oct2bin(m_input.m_line[m_input.m_line_index]);
6663                     ++m_input.m_line_index;
6664
6665                     if (CUtility.isoctdigit(m_input.m_line[m_input.m_line_index]))
6666                       {
6667                         r = (char) (r << 3);
6668                         r = (char) (r | CUtility.oct2bin(m_input.m_line[m_input.m_line_index]));
6669                         ++m_input.m_line_index;
6670                       }
6671
6672                     if (CUtility.isoctdigit(m_input.m_line[m_input.m_line_index]))
6673                       {
6674                         r = (char) (r << 3);
6675                         r = (char) (r | CUtility.oct2bin(m_input.m_line[m_input.m_line_index]));
6676                         ++m_input.m_line_index;
6677                       }
6678                   }
6679                 return r;
6680               }
6681           }
6682       }
6683
6684   /********************************************************
6685     Function: packAccept
6686     Description: Packages and returns CAccept
6687     for action next in input stream.
6688     *******************************************************/
6689   CAccept packAccept
6690     (
6691      )
6692       throws java.io.IOException
6693       {
6694         CAccept accept;
6695         char action[];
6696         int action_index;
6697         int brackets;
6698         boolean inquotes;
6699         boolean instarcomment;
6700         boolean inslashcomment;
6701         boolean escaped;
6702         boolean slashed;
6703
6704         action = new char[BUFFER_SIZE];
6705         action_index = 0;
6706
6707         if (CUtility.DEBUG)
6708           {
6709             CUtility.assert(null != this);
6710             CUtility.assert(null != m_outstream);
6711             CUtility.assert(null != m_input);
6712             CUtility.assert(null != m_tokens);
6713             CUtility.assert(null != m_spec);
6714           }
6715
6716         /* Get a new line, if needed. */
6717         while (m_input.m_line_index >= m_input.m_line_read)
6718           {
6719             if (m_input.getLine())
6720               {
6721                 CError.parse_error(CError.E_EOF,m_input.m_line_number);
6722                 return null;
6723               }
6724           }
6725
6726         /* Look for beginning of action. */
6727         while (CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6728           {
6729             ++m_input.m_line_index;
6730
6731             /* Get a new line, if needed. */
6732             while (m_input.m_line_index >= m_input.m_line_read)
6733               {
6734                 if (m_input.getLine())
6735                   {
6736                     CError.parse_error(CError.E_EOF,m_input.m_line_number);
6737                     return null;
6738                   }
6739               }
6740           }
6741
6742         /* Look for brackets. */
6743         if ('{' != m_input.m_line[m_input.m_line_index])
6744           {
6745             CError.parse_error(CError.E_BRACE,m_input.m_line_number);
6746           }
6747
6748         /* Copy new line into action buffer. */
6749         brackets = 0;
6750         inquotes = inslashcomment = instarcomment = false;
6751         escaped  = slashed = false;
6752         while (true)
6753           {
6754             action[action_index] = m_input.m_line[m_input.m_line_index];
6755
6756             /* Look for quotes. */
6757             if (inquotes && escaped)
6758                 escaped=false; // only protects one char, but this is enough.
6759             else if (inquotes && '\\' == m_input.m_line[m_input.m_line_index])
6760                 escaped=true;
6761             else if ('\"' == m_input.m_line[m_input.m_line_index])
6762                 inquotes=!inquotes; // unescaped quote.
6763             /* Look for comments. */
6764             if (instarcomment) { // inside "/*" comment; look for "*/"
6765                 if (slashed && '/' == m_input.m_line[m_input.m_line_index])
6766                     instarcomment = slashed = false;
6767                 else // note that inside a star comment, slashed means starred
6768                     slashed = ('*' == m_input.m_line[m_input.m_line_index]);
6769             } else if (!inslashcomment) { // not in comment, look for /* or //
6770                 inslashcomment =
6771                     (slashed && '/' == m_input.m_line[m_input.m_line_index]);
6772                 instarcomment =
6773                     (slashed && '*' == m_input.m_line[m_input.m_line_index]);
6774                 slashed = ('/' == m_input.m_line[m_input.m_line_index]);
6775             }
6776
6777             /* Look for brackets. */
6778             if (!inquotes && !instarcomment && !inslashcomment) {
6779               if ('{' == m_input.m_line[m_input.m_line_index])
6780                 {
6781                   ++brackets;
6782                 }
6783               else if ('}' == m_input.m_line[m_input.m_line_index])
6784                 {
6785                   --brackets;
6786
6787                   if (0 == brackets)
6788                     {
6789                       ++action_index;
6790                       ++m_input.m_line_index;
6791
6792                       break;
6793                     }
6794                 }
6795             }
6796
6797             ++action_index;
6798             /* Double the buffer size, if needed. */
6799             if (action_index >= action.length)
6800               {
6801                 action = CUtility.doubleSize(action);
6802               }
6803
6804             ++m_input.m_line_index;
6805             /* Get a new line, if needed. */
6806             while (m_input.m_line_index >= m_input.m_line_read)
6807               {
6808                 inslashcomment = slashed = false;
6809                 if (inquotes) { // non-fatal
6810                     CError.parse_error(CError.E_NEWLINE,m_input.m_line_number);
6811                     inquotes=false;
6812                 }
6813                 if (m_input.getLine())
6814                   {
6815                     CError.parse_error(CError.E_SYNTAX,m_input.m_line_number);
6816                     return null;
6817                   }
6818               }
6819           }
6820
6821         accept = new CAccept(action,action_index,m_input.m_line_number);
6822
6823         if (CUtility.DEBUG)
6824           {
6825             CUtility.assert(null != accept);
6826           }
6827
6828         if (CUtility.DESCENT_DEBUG)
6829           {
6830             System.out.print("Accepting action:");
6831             System.out.println(new String(accept.m_action,0,accept.m_action_read));
6832           }
6833
6834         return accept;
6835       }
6836
6837   /********************************************************
6838     Function: advance
6839     Description: Returns code for next token.
6840     *******************************************************/
6841   private boolean m_advance_stop = false;
6842   int advance
6843     (
6844      )
6845       throws java.io.IOException
6846       {
6847         boolean saw_escape = false;
6848         Integer code;
6849
6850         /*if (m_input.m_line_index > m_input.m_line_read) {
6851           System.out.println("m_input.m_line_index = " + m_input.m_line_index);
6852           System.out.println("m_input.m_line_read = " + m_input.m_line_read);
6853           CUtility.assert(m_input.m_line_index <= m_input.m_line_read);
6854         }*/
6855
6856         if (m_input.m_eof_reached)
6857           {
6858             /* EOF has already been reached,
6859                so return appropriate code. */
6860
6861             m_spec.m_current_token = END_OF_INPUT;
6862             m_spec.m_lexeme = '\0';
6863             return m_spec.m_current_token;
6864           }
6865
6866         /* End of previous regular expression?
6867            Refill line buffer? */
6868         if (EOS == m_spec.m_current_token
6869             /* ADDED */
6870             || m_input.m_line_index >= m_input.m_line_read)
6871             /* ADDED */
6872           {
6873             if (m_spec.m_in_quote)
6874               {
6875                 CError.parse_error(CError.E_SYNTAX,m_input.m_line_number);
6876               }
6877
6878             while (true)
6879               {
6880                 if (false == m_advance_stop
6881                     || m_input.m_line_index >= m_input.m_line_read)
6882                   {
6883                     if (m_input.getLine())
6884                       {
6885                         /* EOF has already been reached,
6886                            so return appropriate code. */
6887
6888                         m_spec.m_current_token = END_OF_INPUT;
6889                         m_spec.m_lexeme = '\0';
6890                         return m_spec.m_current_token;
6891                       }
6892                     m_input.m_line_index = 0;
6893                   }
6894                 else
6895                   {
6896                     m_advance_stop = false;
6897                   }
6898
6899                 while (m_input.m_line_index < m_input.m_line_read
6900                        && true == CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6901                   {
6902                     ++m_input.m_line_index;
6903                   }
6904
6905                 if (m_input.m_line_index < m_input.m_line_read)
6906                   {
6907                     break;
6908                   }
6909               }
6910           }
6911
6912         if (CUtility.DEBUG) {
6913           CUtility.assert(m_input.m_line_index <= m_input.m_line_read);
6914         }
6915
6916         while (true)
6917           {
6918             if (false == m_spec.m_in_quote
6919                 && '{' == m_input.m_line[m_input.m_line_index])
6920               {
6921                 if (false == expandMacro())
6922                   {
6923                     break;
6924                   }
6925
6926                 if (m_input.m_line_index >= m_input.m_line_read)
6927                   {
6928                     m_spec.m_current_token = EOS;
6929                     m_spec.m_lexeme = '\0';
6930                     return m_spec.m_current_token;
6931                   }
6932               }
6933             else if ('\"' == m_input.m_line[m_input.m_line_index])
6934               {
6935                 m_spec.m_in_quote = !m_spec.m_in_quote;
6936                 ++m_input.m_line_index;
6937
6938                 if (m_input.m_line_index >= m_input.m_line_read)
6939                   {
6940                     m_spec.m_current_token = EOS;
6941                     m_spec.m_lexeme = '\0';
6942                     return m_spec.m_current_token;
6943                   }
6944               }
6945             else
6946               {
6947                 break;
6948               }
6949           }
6950
6951         if (m_input.m_line_index > m_input.m_line_read) {
6952           System.out.println("m_input.m_line_index = " + m_input.m_line_index);
6953           System.out.println("m_input.m_line_read = " + m_input.m_line_read);
6954           CUtility.assert(m_input.m_line_index <= m_input.m_line_read);
6955         }
6956
6957         /* Look for backslash, and corresponding
6958            escape sequence. */
6959         if ('\\' == m_input.m_line[m_input.m_line_index])
6960           {
6961             saw_escape = true;
6962           }
6963         else
6964           {
6965             saw_escape = false;
6966           }
6967
6968         if (false == m_spec.m_in_quote)
6969           {
6970             if (false == m_spec.m_in_ccl &&
6971                 CUtility.isspace(m_input.m_line[m_input.m_line_index]))
6972               {
6973                 /* White space means the end of
6974                    the current regular expression. */
6975
6976                 m_spec.m_current_token = EOS;
6977                 m_spec.m_lexeme = '\0';
6978                 return m_spec.m_current_token;
6979               }
6980
6981             /* Process escape sequence, if needed. */
6982             if (saw_escape)
6983               {
6984                 m_spec.m_lexeme = expandEscape();
6985               }
6986             else
6987               {
6988                 m_spec.m_lexeme = m_input.m_line[m_input.m_line_index];
6989                 ++m_input.m_line_index;
6990               }
6991           }
6992         else
6993           {
6994             if (saw_escape
6995                 && (m_input.m_line_index + 1) < m_input.m_line_read
6996                 && '\"' == m_input.m_line[m_input.m_line_index + 1])
6997               {
6998                 m_spec.m_lexeme = '\"';
6999                 m_input.m_line_index = m_input.m_line_index + 2;
7000               }
7001             else
7002               {
7003                 m_spec.m_lexeme = m_input.m_line[m_input.m_line_index];
7004                 ++m_input.m_line_index;
7005               }
7006           }
7007
7008         code = (Integer) m_tokens.get(new Character(m_spec.m_lexeme));
7009         if (m_spec.m_in_quote || true == saw_escape)
7010           {
7011             m_spec.m_current_token = L;
7012           }
7013         else
7014           {
7015             if (null == code)
7016               {
7017                 m_spec.m_current_token = L;
7018               }
7019             else
7020               {
7021                 m_spec.m_current_token = code.intValue();
7022               }
7023           }
7024
7025         if (CCL_START == m_spec.m_current_token) m_spec.m_in_ccl = true;
7026         if (CCL_END   == m_spec.m_current_token) m_spec.m_in_ccl = false;
7027
7028         if (CUtility.FOODEBUG)
7029           {
7030             System.out.println("Lexeme: " + m_spec.m_lexeme
7031                                + "\tToken: " + m_spec.m_current_token
7032                                + "\tIndex: " + m_input.m_line_index);
7033           }
7034
7035         return m_spec.m_current_token;
7036       }
7037
7038   /***************************************************************
7039     Function: details
7040     Description: High level debugging routine.
7041     **************************************************************/
7042   private void details
7043     (
7044      )
7045       {
7046         Enumeration names;
7047         String name;
7048         String def;
7049         Enumeration states;
7050         String state;
7051         Integer index;
7052         int elem;
7053         int size;
7054
7055         System.out.println("\n\t** Macros **");
7056         names = m_spec.m_macros.keys();
7057         while (names.hasMoreElements())
7058           {
7059             name = (String) names.nextElement();
7060             def = (String) m_spec.m_macros.get(name);
7061
7062             if (CUtility.DEBUG)
7063               {
7064                 CUtility.assert(null != name);
7065                 CUtility.assert(null != def);
7066               }
7067
7068             System.out.println("Macro name \"" + name
7069                                + "\" has definition \""
7070                                + def + "\".");
7071           }
7072
7073         System.out.println("\n\t** States **");
7074         states = m_spec.m_states.keys();
7075         while (states.hasMoreElements())
7076           {
7077             state = (String) states.nextElement();
7078             index = (Integer) m_spec.m_states.get(state);
7079
7080             if (CUtility.DEBUG)
7081               {
7082                 CUtility.assert(null != state);
7083                 CUtility.assert(null != index);
7084               }
7085
7086             System.out.println("State \"" + state
7087                                + "\" has identifying index "
7088                                + index.toString() + ".");
7089           }
7090
7091         System.out.println("\n\t** Character Counting **");
7092         if (false == m_spec.m_count_chars)
7093           {
7094             System.out.println("Character counting is off.");
7095           }
7096         else
7097           {
7098             if (CUtility.DEBUG)
7099               {
7100                 CUtility.assert(m_spec.m_count_lines);
7101               }
7102
7103             System.out.println("Character counting is on.");
7104           }
7105
7106         System.out.println("\n\t** Line Counting **");
7107         if (false == m_spec.m_count_lines)
7108           {
7109             System.out.println("Line counting is off.");
7110           }
7111         else
7112           {
7113             if (CUtility.DEBUG)
7114               {
7115                 CUtility.assert(m_spec.m_count_lines);
7116               }
7117
7118             System.out.println("Line counting is on.");
7119           }
7120
7121         System.out.println("\n\t** Operating System Specificity **");
7122         if (false == m_spec.m_unix)
7123           {
7124             System.out.println("Not generating UNIX-specific code.");
7125             System.out.println("(This means that \"\\r\\n\" is a "
7126                                + "newline, rather than \"\\n\".)");
7127           }
7128         else
7129           {
7130             System.out.println("Generating UNIX-specific code.");
7131             System.out.println("(This means that \"\\n\" is a "
7132                                + "newline, rather than \"\\r\\n\".)");
7133           }
7134
7135         System.out.println("\n\t** Java CUP Compatibility **");
7136         if (false == m_spec.m_cup_compatible)
7137           {
7138             System.out.println("Generating CUP compatible code.");
7139             System.out.println("(No current results.)");
7140           }
7141         else
7142           {
7143             System.out.println("Not generating CUP compatible code.");
7144             System.out.println("(No current results.)");
7145           }
7146
7147         if (CUtility.FOODEBUG) {
7148           if (null != m_spec.m_nfa_states && null != m_spec.m_nfa_start)
7149             {
7150               System.out.println("\n\t** NFA machine **");
7151               print_nfa();
7152           }
7153         }
7154
7155         if (null != m_spec.m_dtrans_vector)
7156           {
7157             System.out.println("\n\t** DFA transition table **");
7158             /*print_header();*/
7159           }
7160
7161         /*if (null != m_spec.m_accept_vector && null != m_spec.m_anchor_array)
7162           {
7163             System.out.println("\n\t** Accept States and Anchor Vector **");
7164             print_accept();
7165           }*/
7166       }
7167
7168   /***************************************************************
7169     function: print_set
7170     **************************************************************/
7171   void print_set
7172     (
7173      Vector nfa_set
7174      )
7175       {
7176         int size;
7177         int elem;
7178         CNfa nfa;
7179
7180         size = nfa_set.size();
7181
7182         if (0 == size)
7183           {
7184             System.out.print("empty ");
7185           }
7186
7187         for (elem = 0; elem < size; ++elem)
7188           {
7189             nfa = (CNfa) nfa_set.elementAt(elem);
7190             /*System.out.print(m_spec.m_nfa_states.indexOf(nfa) + " ");*/
7191             System.out.print(nfa.m_label + " ");
7192           }
7193       }
7194
7195    /***************************************************************
7196      Function: print_header
7197      **************************************************************/
7198   private void print_header
7199     (
7200      )
7201       {
7202         Enumeration states;
7203         int i;
7204         int j;
7205         int chars_printed=0;
7206         CDTrans dtrans;
7207         int last_transition;
7208         String str;
7209         CAccept accept;
7210         String state;
7211         Integer index;
7212
7213         System.out.println("/*---------------------- DFA -----------------------");
7214
7215         states = m_spec.m_states.keys();
7216         while (states.hasMoreElements())
7217           {
7218             state = (String) states.nextElement();
7219             index = (Integer) m_spec.m_states.get(state);
7220
7221             if (CUtility.DEBUG)
7222               {
7223                 CUtility.assert(null != state);
7224                 CUtility.assert(null != index);
7225               }
7226
7227             System.out.println("State \"" + state
7228                                + "\" has identifying index "
7229                                + index.toString() + ".");
7230
7231             i = index.intValue();
7232             if (CDTrans.F != m_spec.m_state_dtrans[i])
7233               {
7234                 System.out.println("\tStart index in transition table: "
7235                                    + m_spec.m_state_dtrans[i]);
7236               }
7237             else
7238               {
7239                 System.out.println("\tNo associated transition states.");
7240               }
7241           }
7242
7243         for (i = 0; i < m_spec.m_dtrans_vector.size(); ++i)
7244           {
7245             dtrans = (CDTrans) m_spec.m_dtrans_vector.elementAt(i);
7246
7247             if (null == m_spec.m_accept_vector && null == m_spec.m_anchor_array)
7248               {
7249                 if (null == dtrans.m_accept)
7250                   {
7251                     System.out.print(" * State " + i + " [nonaccepting]");
7252                   }
7253                 else
7254                   {
7255                     System.out.print(" * State " + i
7256                                      + " [accepting, line "
7257                                      + dtrans.m_accept.m_line_number
7258                                      + " <"
7259                                      + (new String(dtrans.m_accept.m_action,0,
7260                                                    dtrans.m_accept.m_action_read))
7261                                      + ">]");
7262                     if (CSpec.NONE != dtrans.m_anchor)
7263                       {
7264                         System.out.print(" Anchor: "
7265                                          + ((0 != (dtrans.m_anchor & CSpec.START))
7266                                             ? "start " : "")
7267                                          + ((0 != (dtrans.m_anchor & CSpec.END))
7268                                             ? "end " : ""));
7269                       }
7270                   }
7271               }
7272             else
7273               {
7274                 accept = (CAccept) m_spec.m_accept_vector.elementAt(i);
7275
7276                 if (null == accept)
7277                   {
7278                     System.out.print(" * State " + i + " [nonaccepting]");
7279                   }
7280                 else
7281                   {
7282                     System.out.print(" * State " + i
7283                                      + " [accepting, line "
7284                                      + accept.m_line_number
7285                                      + " <"
7286                                      + (new String(accept.m_action,0,
7287                                                    accept.m_action_read))
7288                                      + ">]");
7289                     if (CSpec.NONE != m_spec.m_anchor_array[i])
7290                       {
7291                         System.out.print(" Anchor: "
7292                                          + ((0 != (m_spec.m_anchor_array[i] & CSpec.START))
7293                                             ? "start " : "")
7294                                          + ((0 != (m_spec.m_anchor_array[i] & CSpec.END))
7295                                             ? "end " : ""));
7296                       }
7297                   }
7298               }
7299
7300             last_transition = -1;
7301             for (j = 0; j < m_spec.m_dtrans_ncols; ++j)
7302               {
7303                 if (CDTrans.F != dtrans.m_dtrans[j])
7304                   {
7305                     if (last_transition != dtrans.m_dtrans[j])
7306                       {
7307                         System.out.print("\n *    goto " + dtrans.m_dtrans[j]
7308                                          + " on ");
7309                         chars_printed = 0;
7310                       }
7311
7312                     str = interp_int((int) j);
7313                     System.out.print(str);
7314
7315                     chars_printed = chars_printed + str.length();
7316                     if (56 < chars_printed)
7317                       {
7318                         System.out.print("\n *             ");
7319                         chars_printed = 0;
7320                       }
7321
7322                     last_transition = dtrans.m_dtrans[j];
7323                   }
7324               }
7325             System.out.println("");
7326           }
7327         System.out.println(" */\n");
7328       }
7329 }
7330
7331 /*
7332  * @(#)BitSet.java      1.12 95/12/01
7333  *
7334  * Revision: 10/21/96
7335  * Modified by Elliot Joel Berk (ejberk@princeton.edu), for the purposes
7336  * of fixing a bug that was causing unjustified exceptions and crashes.
7337  * This bug was the result of resizing the internal buffer not large enough
7338  * in some cases.  The name was then changed to JavaLexBitSet.
7339  *
7340  * Copyright (c) 1995 Sun Microsystems, Inc. All Rights Reserved.
7341  *
7342  * Permission to use, copy, modify, and distribute this software
7343  * and its documentation for NON-COMMERCIAL purposes and without
7344  * fee is hereby granted provided that this copyright notice
7345  * appears in all copies. Please refer to the file "copyright.html"
7346  * for further important copyright and licensing information.
7347  *
7348  * SUN MAKES NO REPRESENTATIONS OR WARRANTIES ABOUT THE SUITABILITY OF
7349  * THE SOFTWARE, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
7350  * TO THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
7351  * PARTICULAR PURPOSE, OR NON-INFRINGEMENT. SUN SHALL NOT BE LIABLE FOR
7352  * ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF USING, MODIFYING OR
7353  * DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES.
7354  */
7355
7356 /**
7357  * A set of bits. The set automatically grows as more bits are
7358  * needed.
7359  *
7360  * @version     1.12, 01 Dec 1995
7361  * @author Arthur van Hoff, revised by Elliot Joel Berk
7362  */
7363 final class JavaLexBitSet implements Cloneable {
7364     final static int BITS = 6;
7365     final static int MASK = (1<<BITS)-1;
7366     long bits[];
7367
7368     /**
7369      * Creates an empty set.
7370      */
7371     public JavaLexBitSet() {
7372         this(1<<BITS);
7373     }
7374
7375     /**
7376      * Creates an empty set with the specified size.
7377      * @param nbits the size of the set
7378      */
7379     public JavaLexBitSet(int nbits) {
7380         bits = new long[nbits2size(nbits)];
7381     }
7382
7383     private int nbits2size (int nbits) {
7384         return ((nbits >> BITS) + 1);
7385     }
7386
7387     private void resize(int nbits) {
7388         int newsize = Math.max(bits.length, nbits2size(nbits));
7389         long newbits[] = new long[newsize];
7390         System.arraycopy(bits, 0, newbits, 0, bits.length);
7391         bits = newbits;
7392     }
7393
7394     /**
7395      * Sets a bit.
7396      * @param bit the bit to be set
7397      */
7398     public void set(int bit) {
7399         int n = bit>>BITS;
7400         if (n >= bits.length) {
7401             resize(bit);
7402         }
7403         bits[n] |= (1L << (bit & MASK));
7404     }
7405
7406     /**
7407      * Clears a bit.
7408      * @param bit the bit to be cleared
7409      */
7410     public void clear(int bit) {
7411         int n = bit>>BITS;
7412         if (n >= bits.length) {
7413             resize(bit);
7414         }
7415         bits[n] &= ~(1L << (bit & MASK));
7416     }
7417
7418     /**
7419      * Gets a bit.
7420      * @param bit the bit to be gotten
7421      */
7422     public boolean get(int bit) {
7423         int n = bit>>BITS;
7424         return (n < bits.length) ?
7425                 ((bits[n] & (1L << (bit & MASK))) != 0) :
7426                 false;
7427     }
7428
7429     /**
7430      * Logically ANDs this bit set with the specified set of bits.
7431      * @param set the bit set to be ANDed with
7432      */
7433     public void and(JavaLexBitSet set) {
7434         int n = Math.min(bits.length, set.bits.length);
7435         for (int i = n ; i-- > 0 ; ) {
7436             bits[i] &= set.bits[i];
7437         }
7438         for (; n < bits.length ; n++) {
7439             bits[n] = 0;
7440         }
7441     }
7442
7443     /**
7444      * Logically ORs this bit set with the specified set of bits.
7445      * @param set the bit set to be ORed with
7446      */
7447     public void or(JavaLexBitSet set) {
7448         for (int i = Math.min(bits.length, set.bits.length) ; i-- > 0 ;) {
7449             bits[i] |= set.bits[i];
7450         }
7451     }
7452
7453     /**
7454      * Logically XORs this bit set with the specified set of bits.
7455      * @param set the bit set to be XORed with
7456      */
7457     public void xor(JavaLexBitSet set) {
7458         for (int i = Math.min(bits.length, set.bits.length) ; i-- > 0 ;) {
7459             bits[i] ^= set.bits[i];
7460         }
7461     }
7462
7463     /**
7464      * Gets the hashcode.
7465      */
7466     public int hashCode() {
7467         long h = 1234;
7468         for (int i = bits.length; --i >= 0; ) {
7469             h ^= bits[i] * i;
7470         }
7471         return (int)((h >> 32) ^ h);
7472     }
7473
7474     /**
7475      * Calculates and returns the set's size
7476      */
7477     public int size() {
7478         return bits.length << BITS;
7479     }
7480
7481     /**
7482      * Compares this object against the specified object.
7483      * @param obj the object to commpare with
7484      * @return true if the objects are the same; false otherwise.
7485      */
7486     public boolean equals(Object obj) {
7487         if ((obj != null) && (obj instanceof JavaLexBitSet)) {
7488             JavaLexBitSet set = (JavaLexBitSet)obj;
7489
7490             int n = Math.min(bits.length, set.bits.length);
7491             for (int i = n ; i-- > 0 ;) {
7492                 if (bits[i] != set.bits[i]) {
7493                     return false;
7494                 }
7495             }
7496             if (bits.length > n) {
7497                 for (int i = bits.length ; i-- > n ;) {
7498                     if (bits[i] != 0) {
7499                         return false;
7500                     }
7501                 }
7502             } else if (set.bits.length > n) {
7503                 for (int i = set.bits.length ; i-- > n ;) {
7504                     if (set.bits[i] != 0) {
7505                         return false;
7506                     }
7507                 }
7508             }
7509             return true;
7510         }
7511         return false;
7512     }
7513
7514     /**
7515      * Clones the JavaLexBitSet.
7516      */
7517     public Object clone() {
7518         try {
7519             JavaLexBitSet set = (JavaLexBitSet)super.clone();
7520             set.bits = new long[bits.length];
7521             System.arraycopy(bits, 0, set.bits, 0, bits.length);
7522             return set;
7523         } catch (CloneNotSupportedException e) {
7524             // this shouldn't happen, since we are Cloneable
7525             throw new InternalError();
7526         }
7527     }
7528
7529     /**
7530      * Converts the JavaLexBitSet to a String.
7531      */
7532     public String toString() {
7533         String str = "";
7534         for (int i = 0 ; i < (bits.length << BITS) ; i++) {
7535             if (get(i)) {
7536                 if (str.length() > 0) {
7537                     str += ", ";
7538                 }
7539                 str = str + i;
7540             }
7541         }
7542         return "{" + str + "}";
7543     }
7544 }
7545
7546 /************************************************************************
7547   JLEX COPYRIGHT NOTICE, LICENSE AND DISCLAIMER.
7548
7549   Copyright 1996 by Elliot Joel Berk
7550
7551   Permission to use, copy, modify, and distribute this software and its
7552   documentation for any purpose and without fee is hereby granted,
7553   provided that the above copyright notice appear in all copies and that
7554   both the copyright notice and this permission notice and warranty
7555   disclaimer appear in supporting documentation, and that the name of
7556   Elliot Joel Berk not be used in advertising or publicity pertaining
7557   to distribution of the software without specific, written prior permission.
7558
7559   Elliot Joel Berk disclaims all warranties with regard to this software,
7560   including all implied warranties of merchantability and fitness.  In no event
7561   shall Elliot Joel Berk be liable for any special, indirect or consequential
7562   damages or any damages whatsoever resulting from loss of use, data or
7563   profits, whether in an action of contract, negligence or other
7564   tortious action, arising out of or in connection with the use or
7565   performance of this software.
7566   ***********************************************************************/