ASP.NET抓取網(wǎng)頁內(nèi)容的實現(xiàn)方法_

1、ASP.NET抓取網(wǎng)頁內(nèi)容的實現(xiàn)方法_ 這篇文章主要介紹了ASP.NET抓取網(wǎng)頁內(nèi)容的實現(xiàn)方法,涉及用法HttpWebRequest及WebResponse抓取網(wǎng)頁內(nèi)容的技巧,需要的伴侶可以參考下 代碼如下: 代碼如下: 本文實例講解并描述了ASP.NET抓取網(wǎng)頁內(nèi)容的實現(xiàn)方法。分享給大家供大家參考。具體實現(xiàn)方法如下： 一、ASP.NET 用法HttpWebRequest抓取網(wǎng)頁內(nèi)容 代碼如下: / summary方法一：比較推舉 / 用HttpWebRequest取得網(wǎng)頁源碼 / 對于帶BOM的網(wǎng)頁很有效，不管是什么編碼都能正確識別 / /summary / param name=url網(wǎng)頁

2、地址 /param / returns返回網(wǎng)頁源文件/returns public static string GetHtmlSource2(string url) /處理內(nèi)容 string html = ; HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url); request.Accept = */*; /接受任意文件 request.UserAgent = Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322); / request

3、.AllowAutoRedirect = true;/是否允許302 /request.CookieContainer = new CookieContainer();/cookie容器， request.Referer = url; /當(dāng)前頁面的引用 HttpWebResponse response = (HttpWebResponse)request.GetResponse(); Stream stream = response.GetResponseStream(); StreamReader reader = new StreamReader(stream, Encoding.Defa

4、ult); html = reader.ReadToEnd(); stream.Close(); return html; 代碼如下: 二、ASP.NET 用法 WebResponse 抓取網(wǎng)頁內(nèi)容 代碼如下: public static string GetHttpData2(string Url) string sException = null; string sRslt = null; WebResponse oWebRps = null; WebRequest oWebRqst = WebRequest.Create(Url); oWebRqst.Timeout = 50000; t

5、ry oWebRps = oWebRqst.GetResponse(); catch (WebException e) sException = e.Message.ToString(); catch (Exception e) sException = e.ToString(); finally if (oWebRps != null) StreamReader oStreamRd = new StreamReader(oWebRps.GetResponseStream(), Encoding.GetEncoding(utf-8); sRslt = oStreamRd.ReadToEnd(); oStreamRd.Clos