編譯原理課程設(shè)計(jì)-C-詞法掃描器及語(yǔ)法分析器實(shí)現(xiàn)

1、編譯原理課程設(shè)計(jì)報(bào)告 xx xxxxxxxxx 編譯原理課程設(shè)計(jì)報(bào)告課題名稱(chēng)： C-詞法掃描器及語(yǔ)法分析器實(shí)現(xiàn) 提交文檔學(xué)生姓名： 提交文檔學(xué)生學(xué)號(hào)： 同組 成 員 名 單： 無(wú) 指導(dǎo) 教 師 姓 名： 金軍 指導(dǎo)教師評(píng)閱成績(jī)： 指導(dǎo)教師評(píng)閱意見(jiàn)： . . 提交報(bào)告時(shí)間：2014年 6月 xx日目錄目錄21 課程設(shè)計(jì)目標(biāo)22 分析與設(shè)計(jì)42.1 程序結(jié)構(gòu)43 程序代碼實(shí)現(xiàn)93.1 代碼結(jié)構(gòu)93.2.1 globals.h93.2.2 scan.c113.2.3 parser.c183.4 util.c303.5 test.cpp364 測(cè)試結(jié)果384.1.給出標(biāo)準(zhǔn)測(cè)試程序的詞法和語(yǔ)法分析結(jié)果：

2、384.1.1 詞法分析結(jié)果384.1.2 語(yǔ)法分析結(jié)果414.2.修改代碼后的結(jié)果：424.2.1修改425. 本課程設(shè)計(jì)我的獨(dú)創(chuàng)工作446.總結(jié)44 1 課程設(shè)計(jì)目標(biāo)學(xué)生在學(xué)習(xí)編譯原理課程過(guò)程中，結(jié)合各章節(jié)的構(gòu)造編譯程序的基本理論，要求用C或C+語(yǔ)言描述及上機(jī)調(diào)試，實(shí)現(xiàn)一個(gè) C-Minus 小編譯程序（包括詞法分析，語(yǔ)法分析等重要子程序），使學(xué)生將理論與實(shí)際應(yīng)用結(jié)合起來(lái)，受到軟件設(shè)計(jì)等開(kāi)發(fā)過(guò)程的全面訓(xùn)練，從而提高學(xué)生軟件開(kāi)發(fā)的能力。 要求：（1）設(shè)計(jì)詞法分析器設(shè)計(jì)各單詞的狀態(tài)轉(zhuǎn)換圖，并為不同的單詞設(shè)計(jì)種別碼。將詞法分析器設(shè)計(jì)成供語(yǔ)法分析器調(diào)用的子程序。功能包括：a. 具備預(yù)處理功能。將不翻

3、譯的注釋等符號(hào)先濾掉，只保留要翻譯的符號(hào)串，即要求設(shè)計(jì)一個(gè)供詞法分析調(diào)用的預(yù)處理子程序；b. 能夠拼出語(yǔ)言中的各個(gè)單詞；c. 返回（種別碼， 屬性值）。（2）語(yǔ)法分析要求用學(xué)習(xí)過(guò)的自底向上或自頂向下的分析方法等，實(shí)現(xiàn)對(duì)表達(dá)式、各種說(shuō)明語(yǔ)句、控制語(yǔ)句進(jìn)行語(yǔ)法分析。若語(yǔ)法正確，則用語(yǔ)法制導(dǎo)翻譯法進(jìn)行語(yǔ)義翻譯；生成并打印出語(yǔ)法樹(shù)；若語(yǔ)法錯(cuò)誤，要求指出出錯(cuò)性質(zhì)和出錯(cuò)位置（行號(hào)）。2 分析與設(shè)計(jì)2.1 程序結(jié)構(gòu)本程序采用C+語(yǔ)言以面向?qū)ο蟮乃枷刖帉?xiě)，程序分為兩部分：詞法分析（Scanner）和語(yǔ)法分析（Parser）。掃描程序執(zhí)行詞法分析，并將字符序列收集到token中，并將每一行的Token打印出來(lái)。

4、l 實(shí)現(xiàn)方法：Scanner：手工實(shí)現(xiàn)Parser：遞歸下降l 系統(tǒng)總圖:ParserScannersource code Token Syntax Tree 程序流程：在程序中，詞法分析獲取所有Token，并將獲取的Token存儲(chǔ)在scanner對(duì)象的tokenString中。然后Parser類(lèi)的語(yǔ)法分析程序就根據(jù)tokenString中的Token進(jìn)行語(yǔ)法分析，生成語(yǔ)法樹(shù)，最后打印語(yǔ)法樹(shù)。同時(shí)，這也是程序的流程。整體程序流程圖開(kāi)始刪除注釋出錯(cuò)詞法分析TF結(jié)束語(yǔ)法分析出錯(cuò)出錯(cuò)輸出出錯(cuò)信息TFFT從文件獲取源代碼l 掃描器：C慣用的詞法1、語(yǔ)言的關(guān)鍵字：else if int return v

5、oid while2、專(zhuān)用符號(hào)：+ - * / < <= > >= = != = ; , ( ) /* */3、其他標(biāo)記是ID和NUM，通過(guò)下列正則表達(dá)式定義：ID = letter letter*NUM = digit digit*letter = a|.|z|A|.|Zdigit = 0|.|94、空格由空白、換行符和制表符組成。空格通常被忽略，除了它必須分開(kāi)ID、NUM關(guān)鍵字。5. 注釋用通常的C語(yǔ)言符號(hào)/ * . . . * /圍起來(lái)。注釋可以放在任何空白出現(xiàn)的位置(即注釋不能放在標(biāo)記內(nèi))上，且可以超過(guò)一行。注釋不能嵌套。DFA圖如下：初始狀態(tài)設(shè)置為START，

6、當(dāng)需要得到下一個(gè)token時(shí)，取得此token的第一個(gè)字符，并且按照DFA與對(duì)此字符的類(lèi)型的分析，轉(zhuǎn)換狀態(tài)。重復(fù)此步驟，直到DONE為止，輸出token類(lèi)型。參考課本中所給的TINY掃描程序的DFA，C-的DFA不同之處在于注釋以及(= =,! = ,< =, > =)的處理：1. 注釋部分參考了課本中C風(fēng)格注釋的DFA； Other other *54321 / * * / Other other 2. 處理(= =,! = ,< =, > =)時(shí)并沒(méi)有新建狀態(tài)，而是在狀態(tài)INASSIGN中分了4種情況，通過(guò)一個(gè)變量分別處理。INENDCOMMENTl 語(yǔ)法分析： C

7、-語(yǔ)法與語(yǔ)義：1. program declaration-list 2. declaration-list declaration-list declaration | declaration 3. declaration var-declaration | fun-declaration 4. var-declaration type-specifier ID ; | type-specifier ID NUM ; 5. type-specifier int | void 6. fun-declaration type-specifier ID ( params ) compound-st

8、mt （在課后解釋中compound-stmt前面沒(méi)有“|”符號(hào)） 7. params params-list | void 8. param-list param-list , param | param 9. param type-specifier ID | type-specifier ID 10. compound-stmt local-declarations statement-list 11. local-declarations local-declarations var-declaration | empty 12. statement-list statement-li

9、st statement | empty 13. statement expression-stmt | compound-stmt | selection-stmt | iteration-stmt | return-stmt 14. expression-stmt expression ; | ; 15. selection-stmt if ( expression ) statement | if ( expression ) statement else statement 16. iteration-stmt while(expression) statement 17. retur

10、n-stmt return ; | return expression ; 18. expression var = expression | simple-expression 19. var ID | ID expression 20. simple-expression additive-expression relop additive-expression | additive-expression 21. relop <= | < | > | >= | = | != 22. additive-expression additive-expression ad

11、dop term | term 23. addop + | - 24. term term mulop factor | factor 25. mulop * | / 26. factor ( expression ) | var | call | NUM 27. call ID ( args ) 28. args arg-list | empty 29. arg-list arg-list , expression | expressionl 代碼設(shè)計(jì)格式：程序結(jié)構(gòu)：語(yǔ)法分析函數(shù)parser通過(guò)調(diào)用詞法分析函數(shù)getToken實(shí)現(xiàn)語(yǔ)法分析。文件和函數(shù)的設(shè)計(jì)說(shuō)明：文件main.c包含相應(yīng)頭文件

12、，及main函數(shù)的實(shí)現(xiàn)；文件golbals.h包含符號(hào)表和分析數(shù)的數(shù)據(jù)結(jié)構(gòu)及在其它文件中使用的變量；文件util.h 和util.c實(shí)現(xiàn)與詞法分析和語(yǔ)法分析輸出相關(guān)的函數(shù)printToken和printTree，以及分析樹(shù)節(jié)點(diǎn)初始化相關(guān)的函數(shù)newStmtNode，newExpNode(Expkind)和copyString；文件scan.h 和scan.c實(shí)現(xiàn)詞法分析，主要函數(shù)為getToken；文件parser.h 和parser.c實(shí)現(xiàn)語(yǔ)法分析，函數(shù)為與文法規(guī)則對(duì)應(yīng)的函數(shù)。3 程序代碼實(shí)現(xiàn)3.1 代碼結(jié)構(gòu)3.2.1 globals.h#ifndef _GLOBALS_H_#define

13、_GLOBALS_H_#include <STDIO.H>#include <STDLIB.H>#include <CTYPE.H>#include <STRING.H>#include<iostream>#include <direct.h>using namespace std;#ifndef FALSE#define FALSE 0#endif#ifndef TRUE#define TRUE 1;#endif/ 保留字的個(gè)數(shù)#define MAXRESERVED 6typedef enum/book-keeping

14、tokensENDFILE,ERROR,/reserved wordsIF,ELSE,INT,RETURN,VOID,WHILE,/multicharacter tokensID,NUM,/special symbols/LBRACKET/RBRACKET - LBRACE/RBRACE ASSIGN,EQ,LT,RT,PLUS,MINUS,TIMES,OVER,LPAREN,RPAREN,SEMI,LBRACKET,RBRACKET,NEQ,COMMA,LBRACE,RBRACE,LTEQ,RTEQTokenType;extern FILE* source; /源代碼文件extern FIL

15、E* listing; /listing output text fileextern FILE* code; /extern int lineno; /語(yǔ)法分析樹(shù)typedef enum NodeKindStmtK, ExpK, DeclarationK,;typedef enum StmtKindIfK, WhileK, AssignK, ReturnK, VoidK,;typedef enum ExpKindOpK, ConstK, IdK, Array_expK,Function_expK,;typedef enum ExpTypeVoid,Integer;typedef enum D

16、eclarationKind FunctionK, VarK, ArrayK, ParaMeterK ;#define MAXCHILDREN 3typedef struct treeNodestruct treeNode *declaration;struct treeNode *childMAXCHILDREN;struct treeNode *sibling;struct treeNode *params;int lineno;NodeKind nodekind;union kindStmtKind stmt;ExpKind exp;DeclarationKind declaration

17、;kind;struct MyStruct TokenType op;int val;char *name;int size;attr;ExpType type;TreeNode;extern int EchoSource;extern int TraceScan;extern int TraceParse;extern int TraceAnalyze;extern int TraceCode;extern int Error;#endif3.2.2 scan.c最主要的過(guò)程是GetToken,它消耗輸入字符并根據(jù)DFA圖返回下一個(gè)被識(shí)別的記號(hào)。這個(gè)實(shí)現(xiàn)利用了雙重嵌套情況分析，以及一個(gè)有關(guān)狀

18、態(tài)的大型情況列表，在大列表中是基于當(dāng)前輸入字符的單獨(dú)列表。記號(hào)本身被定義成globals.h中的枚舉類(lèi)型。掃描程序的狀態(tài)也被定義為一個(gè)枚舉類(lèi)型，位于掃描程序之中。掃描程序使用了三個(gè)全局變量，文件變量source，listing.在globals.h中聲明且在main.c中被分配和初始化的整型變量lineno。由getToken過(guò)程完成的額外的簿記如下所述：表reserved和過(guò)程reservedlookup完成位于由getToken的主要循環(huán)識(shí)別的標(biāo)識(shí)符之后的保留字的查找，currentToken的值也隨之改變。標(biāo)志變量save被用作指示是否將第一個(gè)字符增加到tokenString之上；由于需

19、要包括空白格和非消耗的先行，因此這些東西都是必要的。到掃描程序的字符輸入由getNextChar函數(shù)提供，該函數(shù)將一個(gè)256字符緩沖區(qū)內(nèi)部的lineBuf中的字符取到掃描程序中，如果已經(jīng)耗盡了這個(gè)緩沖區(qū)，且假設(shè)每一次都獲取了一個(gè)新的源代碼行，那么getNextChar就利用fget從source文件更新該緩沖區(qū)。最后由于從INUM和INID到最終狀態(tài)的轉(zhuǎn)換是非消耗的。可以通過(guò)一個(gè)ungetnextChar的過(guò)程在輸入緩沖區(qū)的反填一個(gè)字符來(lái)完成這一任務(wù)。#include "globals.h"#include "util.h"#include "

20、scan.h"typedef enum START, INASSIGN, INCOMMENT,INENDCOMMENT,INNUM, INID, DONE,INLT,INRT ,INNEQStateType;char tokenStringMAXTOKENLEN+1;#define BUFLEN 1024static char lineBufBUFLEN;static int linepos=0;static int bufsize=0;/getNextChar 從lineBuf獲得下一個(gè)非空字符，讀入新的一行如果lineBuf 耗盡static char getNextCHar(v

21、oid)if (!(linepos<bufsize)lineno+;if (fgets(lineBuf,BUFLEN-1,source)if (EchoSource)fprintf(listing,"%4d,%s",lineno,lineBuf);bufsize=strlen(lineBuf);linepos=0;return lineBuflinepos+;else return EOF;else return lineBuflinepos+; / backtracks one character in lineBufstatic void ungetNextCha

22、r(void)linepos-;/查找表的保留字static struct char *str;TokenType tok;reservedWordsMAXRESERVED="if",IF,"int",INT,"else",ELSE,"return",RETURN,"void",VOID,"while",WHILE;/查找一個(gè)標(biāo)識(shí)符,看看這是否是一個(gè)保留字/使用線(xiàn)性搜索static TokenType reservedLookup(char *s)int i;for(i=0;

23、i<MAXRESERVED;+i)if(!strcmp(s,reservedWordsi.str)return reservedWordsi.tok;return ID;TokenType getToken(void)/索引存儲(chǔ)為“tokenString”int tokenStringIndex = 0;/返回當(dāng)前tokenTokenType currentToken;/當(dāng)前狀態(tài)總是以START開(kāi)始StateType state = START;/指示存放至tokenString的標(biāo)志int save=TRUE;while (state != DONE)int c = getNextCH

24、ar();save = TRUE;switch (state)case START:if (isdigit(c)state = INNUM;else if (isalpha(c)state =INID;else if (c = '=')state =INASSIGN;else if (c = ' ') |(c = 't') | (c = 'n')save = FALSE;else if (c = '/') /遇到"/"則假定就是注釋?zhuān)M(jìn)入注釋狀態(tài)state = INCOMMENT;save =

25、FALSE;else if (c = '>')state = INRT;else if (c = '<')state = INLT;else if (c = '!')state = INNEQ;elsestate = DONE;switch (c)default:currentToken = ERROR;break;case EOF:save = FALSE;currentToken = ENDFILE;break;case '':currentToken = LBRACKET;break;case ''

26、;:currentToken = RBRACKET;break;case ',':currentToken = COMMA;break;case '+':currentToken = PLUS;break;case '-':currentToken = MINUS;break;case '*':currentToken = TIMES;break;case '':currentToken = LBRACE;break;case '':currentToken = RBRACE;break;case

27、'(':currentToken = LPAREN;break;case ')':currentToken = RPAREN;break;case '':currentToken = SEMI;break;break;/*=注釋改動(dòng)=*/case INENDCOMMENT:if (c = '*'&&getNextCHar() = '/') /token是*且下一個(gè)token是/則結(jié)束注釋save = FALSE;state = START; /注釋不作處理，直到注釋結(jié)束，elsesave = FAL

28、SE;state = INENDCOMMENT;break;case INCOMMENT:if (c = '*') /“/”后是*則進(jìn)入注釋/ungetNextChar();save = FALSE; state = INENDCOMMENT; /若是注釋開(kāi)頭，轉(zhuǎn)入注釋結(jié)尾處理else if (c = EOF)state = DONE;currentToken = ENDFILE;else / “/”后不是*則為除法ungetNextChar();save = FALSE;currentToken = OVER;state = DONE;break;/*=賦值改動(dòng)=*/case

29、 INASSIGN:state = DONE;if (c = '=') /下一個(gè)仍是等號(hào)就為相等currentToken = EQ;else /否則回退，ungetNextChar();save = FALSE;currentToken = ASSIGN;break;/*=小于等于=*/case INLT:state = DONE;if (c = '=')currentToken = LTEQ;elseungetNextChar();save = FALSE;currentToken = LT;break;/*=大于等于=*/case INRT:state =

30、DONE;if (c = '=')currentToken = RTEQ;elseungetNextChar();save = FALSE;currentToken = RT;break;/*=不等于=*/case INNEQ:state = DONE;if (c = '=')currentToken = NEQ;elseungetNextChar();save = FALSE;currentToken = ERROR;break;case INNUM:if (! isdigit(c)ungetNextChar();save = FALSE;state = DO

31、NE;currentToken = NUM;break;case INID:if (!isalpha(c)ungetNextChar();save = FALSE;state = DONE;currentToken = ID;break;case DONE:default:fprintf(listing, "Scanner Bug:state=%dn", state);state = DONE;currentToken = ERROR;break;if (save) && (tokenStringIndex <= MAXTOKENLEN)tokenSt

32、ringtokenStringIndex+ = (char)c;if (state = DONE)tokenStringtokenStringIndex = '0'if (currentToken = ID)currentToken = reservedLookup(tokenString);if (TraceScan)fprintf(listing, "t%d:", lineno);printToken(currentToken, tokenString);return currentToken;3.2.3 parser.c#include"gl

33、obals.h"#include"util.h"#include"scan.h"#include"parse.h"static TokenType token; /holds current tokenstatic TreeNode * stmt_sequence(void);static TreeNode * statement(void);static TreeNode * if_stmt(void);static TreeNode * while_stmt(void);/自定義函數(shù)static TreeNode * r

34、eturn_stmt(void);static TreeNode * int_stmt(void);static TreeNode * void_stmt(void);static TreeNode * declaration(void);static TreeNode * parameter(void);static TreeNode * exp(void);static TreeNode * simple_exp(void);static TreeNode * term(void);static TreeNode * factor(void);static void inFunction(

35、void);static void syntaxError(char *message)fprintf(listing, "n>>>");fprintf(listing, "Syntax error at line %d: %s", lineno, message);Error = TRUE;static void match(TokenType expected)if (token = expected)token = getToken();elsesyntaxError("unexpected token->&quo

36、t;);printToken(token, tokenString);fprintf(listing, " ");TreeNode * stmt_sequence(void)TreeNode *t = statement();TreeNode *p = t;while (token != ENDFILE) && (token != RBRACE) )TreeNode *q;/match(SEMI);q = statement();if (token != RBRACE)if (q != NULL)if (t = NULL)t = p = q;elsep-&g

37、t;sibling = q;p = q;return t;TreeNode * statement(void)TreeNode *t = NULL;switch (token)case IF:t = if_stmt();break;case WHILE:t = while_stmt();break;case ID: /若為ID不直接進(jìn)行賦值，進(jìn)入 exp()至factor()，將賦值加入至factor()t = exp();break;case RETURN:t = return_stmt();break;case INT:t = int_stmt();break;case VOID:t =

38、void_stmt();break;case SEMI:match(SEMI);break;default:syntaxError("unexpected token -> ");printToken(token, tokenString);token = getToken();break;return t;TreeNode *declaration(void)TreeNode *t = newDeclarationNode(token);if (t != NULL)/名稱(chēng)token = getToken();/ID賦給t->

39、 = copyString(tokenString);token = getToken();/token是"(" 則為函數(shù)聲明if (token = LPAREN)t->kind.declaration = FunctionK;match(LPAREN);TreeNode *paramets = parameter(); /函數(shù)參數(shù)匹配if (paramets != NULL)t->child0 = paramets;match(RPAREN);match(LBRACE);/匹配函數(shù)體TreeNode *body = stmt_sequence();if (bo

40、dy = NULL)syntaxError("error body！ ");elset->child1 = body;match(RBRACE);/token后不是（則是變量聲明else/普通變量類(lèi)型t->kind.declaration = VarK;/進(jìn)入數(shù)組類(lèi)型if (token = LBRACKET)match(LBRACKET);t->kind.declaration = ArrayK;t->attr.size = atoi(tokenString);token = getToken();match(RBRACKET);match(SEMI

41、);return t;TreeNode *parameter(void)TreeNode *t = NULL;if (token = VOID)match(VOID);/結(jié)束else if (token = RBRACE)return t;/有參數(shù)elset = newDeclarationNode(token);token = getToken();t-> = copyString(tokenString);token = getToken();/數(shù)組if (token = LBRACKET)t->kind.declaration = ArrayK;match(

42、LBRACKET);t->attr.size = 0;match(RBRACKET);/IDelset->kind.declaration = VarK;if (token = COMMA)match(COMMA);t->sibling = parameter();return t;TreeNode *if_stmt(void)TreeNode *t = newStmtNode(IfK);match(IF);if (t != NULL)match(LPAREN);t->child0 = exp();match(RPAREN);if (t != NULL)/如果帶大括號(hào)i

43、f (token = LBRACE)match(LBRACE);while (token != RBRACE)t->child1 = statement();match(RBRACE);/無(wú)括號(hào)elset->child1 = statement();if (token = ELSE)match(ELSE);if (t != NULL)/如果帶大括號(hào)if (token = LBRACE)match(LBRACE);while (token != RBRACE)t->child2 = statement();match(RBRACE);/無(wú)括號(hào)elset->child2 =

44、 statement();return t;TreeNode *while_stmt(void)TreeNode *t = newStmtNode(WhileK);match(WHILE);TreeNode *p = NULL;match(LPAREN);while (token != RPAREN)if (t->child0 = NULL)p = exp();t->child0 = p;elsep = p->child0;p = exp();match(RPAREN);/while函數(shù)大括號(hào)可有可無(wú)/如果有括號(hào)if (token = LBRACE)match(LBRACE)

45、;while (token!=RBRACE)t->child1 = statement();match(RBRACE);/無(wú)elset->child1 = statement();if (token = SEMI)match(SEMI);return t;TreeNode *return_stmt(void)TreeNode *t = newStmtNode(ReturnK);t-> = copyString(tokenString);match(RETURN);if (token != SEMI)t->child0 = exp();match(SEM

46、I);return t;TreeNode *int_stmt(void)TreeNode *t = newDeclarationNode(INT);match(INT);t-> = copyString(tokenString);match(ID);/函數(shù)if (token = LPAREN)match(LPAREN);t->kind.declaration = FunctionK;if (token = VOID)match(VOID);else/函數(shù)參數(shù)匹配inFunction();match(RPAREN);/函數(shù)體match(LBRACE);while (token != RBRACE)statement();match(RBRACE);/數(shù)組else if (token = LBRACKET)t->kind.exp = Array_expK;match(LBRACKET);t->child0 = exp();match(RBRACKET);/變量if (token = SEMI)t->kind.declaration = VarK;match(SEMI);return t;/void 函數(shù)TreeNode *void_stmt(void)TreeNode *t =