臺灣博碩士論文加值系統

近年來網際網路的蓬勃發展，網路已成為人們日常生活中重要的一環，依據西元2000年到2012年的全球網際網路統計上網人口已突破24億，成長率達566.4%，以及Netcraft統計2013年10月全球已經有超過七億個網站運行，因此網路安全維護的問題也變得日益重要。
隨著網路環境以及攻擊方法日益複雜，入侵偵測系統需要更多方面的稽核資訊，達到更完整而詳實的分析，以執行資訊整合的比對減少誤報，並希望找到漏報。本研究利用Hadoop雲端運算技術，可提供大量資料的分散式運算環境，有可靠度高、可擴展性強的特性，以增加入侵偵測系統的運作效能。
本論文提出一種基於MapReduce平行運算模組的雲端網路資訊處理平台，分析Snort入侵警訊、主機上的防火牆及網路防火牆等的網路安全設備資訊，執行資訊正規化、資訊整合及合併，並將資訊關聯報表的資料匯入網頁查詢列表，進行資訊查詢事件分析及資訊關聯圖的產生，達到強化且彈性的資訊查詢分析功能，透過資訊關聯合併後，發現警示縮減率為95.1%，有效降低資安專家人員稽核事件的時間與資料量，並作出正確的判斷。

The Internet has flourished in recent years and has become a vital component of everyday life. According to the Internet World Stats from 2000 to 2012, the number of Internet users worldwide was over 2.4 billion with a growth rate of 566.4%. Based on the Netcraft statistics reported in October 2013, over 700 million websites were in operation. Consequently, the importance of Internet safety maintenance is increasing.
As attack methods become more complicated, network defence experts require an increasing number of information auditing dimensions to improve the comprehensive and explicit qualities of analyses used to conduct information integration comparisons. Hadoop cloud computing technology was employed in this study to provide distributed computing environments for a massive amount of data. This computing approach exhibits high reliability and scalability, which can enhance the operational performance in security incident detection.
This thesis proposes a processing platform with cloud network technologies based on the MapReduce parallel computing model in order to analyze the information for internet security devices, such as Snort, the intrusion detection, and ASA5505, the network firewalls on the host computer and the Internet. Moreover, this thesis explores the application of this platform to the data normalization, integration and consolidation. Apart from that, the researcher imports the data-related statements into a list of web queries to analyze data query events and to produce data connection maps in order to strengthen the analysis function in security investigation cases. By consolidating the data-related statements, it is found that the reduction rate for detection data is 95.1%. This consolidation effectively reduces the time and information volume of security experts in audit events and also allows the experts to make correct judgments.
Keywords: Intrusion Detection System, Hadoop, Mapreduce, Data Integration, Data Consolidation

誌謝 ii
摘要 iii
ABSTRACT iv
目錄 vi
表目錄 viii
圖目錄 ix
附錄 xii
1. 緒論 1
1.1. 研究背景與動機 1
1.2. 研究目的 3
1.3. 研究步驟與流程 4
1.4. 論文架構 5
2. 背景知識與相關文獻探討 6
2.1. 雲端運算簡介 6
2.1.1. 雲端運算的重要特徵 7
2.1.2. 雲端服務的類型 8
2.2. Hadoop 12
2.2.1. HDFS分散式檔案系統 14
2.2.2. MapReduce運算模式 16
2.3. 入侵偵測系統 20
2.3.1. 入侵偵測系統簡介 20
2.3.2. Snort 24
2.4. Graphviz介紹 26
2.5. Hadoop雲端入侵偵測系統相關文獻探討 27
2.5.1. IDS警報關聯整合 27
2.5.2. 入侵警報合併 29
2.5.3. 當前系統的優劣分析 31
3. 雲端網路資訊處理平台 34
3.1. 資訊資料正規化 35
3.2. MapReduce運算模組介紹 38
3.3. 雲端網路資訊處理平台功能介紹 41
3.3.1. 資訊即時傳送分析功能 41
3.3.2. 雲端資安資訊查詢功能 42
4. 實驗成果與分析 43
4.1. 實驗環境建置 43
4.2. 警示與事件正確性驗證 46
4.3. 資訊關聯與分析查詢 53
5. 結論與未來研究 56
5.1. 結論 56
5.2. 未來研究 56
參考文獻 58
附錄A Hadoop雲端運算平台建置流程 61
附錄B Eclipse程式開發流程 72
自傳 79

[1]http://www.internetworldstats.com/stats.htm(2013.10.19)
[2]http://news.netcraft.com/archives/2013/10/02/october-2013-web-server-survey.html(2013.10.19)
[3]http://www.akamai.com/dl/documents/akamai_soti_q213.pdf?WT.mc_id=soti_Q213(2013.10.19)
[4]蔡碧展，“基於Hadoop平台的雲端基因架構”，碩士論文，國立高雄第一科技大學資訊管理系研究所，高雄，第30-37頁， 2010。
[5]http://www.ithome.com.tw/itadm/article.php?c=61024(2013.10.19)
[6]http://www.emc.com/collateral/analyst-reports/idc-digital-universe-are-you-ready.pdf(2013.10.19)
[7]http://www.syscom.com.tw/ePaper_Content_EPArticledetail.aspx?id=140&EPID=168&j=5&HeaderName=%E7%A0%94%E7%99%BC%E6%96%B0%E8%A6%96%E7%95%8C(2013.10.19)
[8]http://www.ithome.com.tw/itadm/article.php?c=49410&s=4(2013.10.19)
[9]http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf(2012.12.28)
[10]http://www.gartner.com/it-glossary/cloud-computing/(2013.10.19)
[11]http://blogs.idc.com/ie/?p=190(2013.10.19)
[12]http://blogs.forrester.com/james_staten/09-10-14-cloud_defined_now_stop_cloudwashing(2013.10.19)
[13]https://developers.google.com/appengine/training/intro/whatiscc(2013.10.19)
[14]http://www-935.ibm.com/services/tw/gts/fue/article/2_1/2.html(2013.10.19)
[15]http://www.microsoft.com/taiwan/Mcloud/oacloud/default.aspx(2013.10.19)
[16]http://zh.wikipedia.org/wiki/%E9%9B%B2%E7%AB%AF%E9%81%8B%E7%AE%97(2013.10.19)
[17]http://www.cloudopenlab.org.tw/ccipo_industryDefinition.do(2013.10.19)
[18]http://zh.wikipedia.org/wiki/Windows_Azure_Platform_AppFabric(2013.10.19)
[19]http://www.openfoundry.org/tw/resourcecatalog/Cloud-Computing/Infrastructure-Layer/Computing/Amazon-EC2(2013.10.19)
[20]http://www.moneydj.com/kmdj/wiki/wikiviewer.aspx?keyid=b2a16b54-77ee-4a1d-8feb-a3d0366e55c8(2013.10.20)
[21]http://hadoop.apache.org/(2013.10.20)
[22]http://hadoop.apache.org/docs/r0.18.0/hdfs_design.pdf(2013.10.19)
[23]Chang Fay, Dean Jeffrey, Ghemawat Sanjay, Hsieh Wilson C., Wallach Deborah A., Burrows Mike, Chandra Tushar, Fikes Andrew, Gruber Robert E., “Bigtable: a Distributed Storage System for Structured Data,” Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation, Seattle, Vol. 26, No. 2, pp. 205-218, 2006.
[24]Ghemawat Sanjay, Gobioff Howard, Leung Shun-Tak, “The Google File System,” Proceedings of the nineteenth ACM symposium on Operating systems principles, New York, Vol. 37, No. 5, pp. 29-43, 2003.
[25]Dean Jeffrey, Ghemawat Sanjay, “MapReduce: Simplified Data Processing on Large Clusters,” Proceedings of the 6th Symposium on Operating System Design and Implementation, San Francisco, Vol. 51, No. 1, pp.137-149, 2004.
[26]http://www.sans.org/resources/idfaq/(2012.1.7)
[27]Denning Dorothy E., “An Intrusion Detection Model” , Proceeding of the IEEE Transactions On Software Engineering, Piscataway, Vol. 13, No. 2, pp. 222-232, 1987.
[28]黃培生、楊中皇，“結合入侵偵測和蜜罐之分散式預警系統的設計與實現”，碩士論文，師範大學資訊教育研究所，高雄，第10-13頁，2009。
[29]Frédéric Cuppens, and Alexandre Miège, “Alert Correlation in a Cooperative Intrusion Detection Framework,” Proceeding of the IEEE Symposium on Research in Security and Privacy, Oakland, Vol. 26, No. 2, pp. 202-215, 2002.
[30]http://www.cert.org.tw/document/column/show.php?key=85(2012.1.7)
[31]楊景隆，“運用網路誘捕系統對入侵行為之分析與實作”，碩士論文，大葉大學資訊管理學系碩士班，彰化，第5-8頁，2006。
[32]http://www.snort.org(2012.1.7)
[33]吳育庭，“網路安全防護之研究”，碩士論文，國防大學理工學院資訊工程學系碩士班，桃園，第9-19頁，2012。
[34]http://www.graphviz.org/(2013.10.8)
[35]http://www.netadmin.com.tw/article_content.aspx?sn=1305300001(2013.10.8)
[36]Yang Shun-Fa, Chen Wei-Yu, Wang Yao-Tsung, “ICAS: an Inter-VM IDS Log Cloud Analysis System”, Proceeding of the IEEE International Conference on Cloud Computing and Intelligence Systems, Beijing, pp. 285-289, 2011.
[37]白浩泉，姚立紅，陸松年，“基於MapReduce平行運算模型的警報聚合演算法”，信息技術，第四期，第85-88頁，2011。
[38]http://trac.nchc.org.tw/cloud/wiki/Talk(2013.10.22)
[39]http://trac.nchc.org.tw/cloud(2013.10.23)
[40]Xu Guanghui, Xu Feng, Ma Hongxu, “ Deploying and Researching Hadoop in Virtual Machines,” Proceeding of the IEEE International Conference on Automation and Logistics, Zhengzhou, pp. 395-399, 2012.