施主，你的漏洞采集秘笈到了
最近斗哥在逛某论坛的时候，看到有人提出一个问题：“如何快速获取第一手漏洞信息，可否介绍一些第一手漏洞信息的获取渠道？”so...雷厉风行的斗哥这就为大家带来了介绍。
漏洞信息的获取渠道
1.网站 https://seclists.org/fulldisclosure/
任何黑客都会告诉你，在任何网站上都找不到最新的新闻和漏洞，甚至没有。而在这里，一个公共的，与供应商无关的论坛，最新的漏洞有时会在Bugtraq审核通过之前数小时或数天出现在此列表中。
2.网站 https://www.exploit-db.com/
Exploit-DB是一个CVE兼容的公共漏洞和相应易受攻击软件的存档，开发供渗透测试人员和漏洞研究人员使用。目标是通过直接提交，邮件列表以及其他公共资源提供最全面的漏洞利用集合，并将其呈现在一个免费且易于导航的数据库中。漏洞利用数据库是漏洞利用和概念验证的存储库，而不是建议，使其成为那些需要立即采取可操作数据的人的宝贵资源。
3.网站 https://www.openssl.org/news/vulnerabilities.html
Struts2是apache项目下的一个web 框架，普遍应用于阿里巴巴、京东等互联网、政府、企业门户网站。在过去的几年内，几乎每年Struts2都会爆出存在严重安全漏洞，苹果、中国移动、中国联通、百度、腾讯、淘宝、京东、Sohu、民生银行等大型企业的网站均遭毒手，运维工程师苦不堪言。故该框架的官方安全公告也是斗哥关注的一个对象。
4.网站 https://www.openssl.org/news/vulnerabilities.html
OpenSSL是为网络通信提供安全及数据完整性的一种安全协议，囊括了主要的密码算法、常用的密钥和证书封装管理功能以及SSL协议，目前正在各大网银、在线支付、电商网站、门户网站、电子邮件等重要网站上广泛使用，使用了存在漏洞的OpenSSL版本，用户登录该网站时就可能被黑客实时监控到登录账号和密码等敏感的信息。所以该协议漏洞应该特别值得关注。
自动化漏洞信息采集
1. 漏洞信息采集
接下来，我们可以写一段python代码，对上述四个网站进行漏洞信息的采集：
对Seclists网站的采集代码如下：
# 获取seclists完全漏洞列表
def seclists(time):
result = []
time = time.split("-")
# 漏洞列表网址 https://seclists.org/fulldisclosure/2018/Jan/date.html
url = 'https://seclists.org/fulldisclosure/%s/%s/date.html' % (time[0], time[1])
# 获取GET响应html
html = get(url).text
try:
vulnerablelist = \
re.findall('<h4>%s, %s %s</h4>\n<blockquote>(.*?)</blockquote>' % (time[4], time[3], time[2]), html, re.S)[
0].split('\n')
except IndexError:
vulnerablelist = []
for vulnerable in vulnerablelist:
vulnerabledict = {}
if vulnerable == "":
continue
vulnerable_href = re.findall('href="(.*?)"', vulnerable, re.S)[0]
vulnerabledict["name"] = re.findall('">(.*?)</a>', vulnerable, re.S)[0]
# 漏洞详情页面网址 https://seclists.org/fulldisclosure/2018/Nov/0
vulnerabledict["url"] = "https://seclists.org/fulldisclosure/%s/%s/%s" % (time[0], time[1], vulnerable_href)
result.append(vulnerabledict)
return result对Exploit-DB网站的采集代码如下：
# 获取exploit_db漏洞列表
def exploit_db(time):
result = []
url = "https://www.exploit-db.com/?draw=1&columns%5B0%5D%5Bdata%5D=date_published&columns%5B0%5D%5Bname%5D=date_published&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=true&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D=download&columns%5B1%5D%5Bname%5D=download&columns%5B1%5D%5Bsearchable%5D=false&columns%5B1%5D%5Borderable%5D=false&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=application_md5&columns%5B2%5D%5Bname%5D=application_md5&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%5Borderable%5D=false&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=verified&columns%5B3%5D%5Bname%5D=verified&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=false&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B4%5D%5Bdata%5D=description&columns%5B4%5D%5Bname%5D=description&columns%5B4%5D%5Bsearchable%5D=true&columns%5B4%5D%5Borderable%5D=false&columns%5B4%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B4%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B5%5D%5Bdata%5D=type_id&columns%5B5%5D%5Bname%5D=type_id&columns%5B5%5D%5Bsearchable%5D=true&columns%5B5%5D%5Borderable%5D=false&columns%5B5%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B5%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B6%5D%5Bdata%5D=platform_id&columns%5B6%5D%5Bname%5D=platform_id&columns%5B6%5D%5Bsearchable%5D=true&columns%5B6%5D%5Borderable%5D=false&columns%5B6%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B6%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B7%5D%5Bdata%5D=author_id&columns%5B7%5D%5Bname%5D=author_id&columns%5B7%5D%5Bsearchable%5D=false&columns%5B7%5D%5Borderable%5D=false&columns%5B7%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B7%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B8%5D%5Bdata%5D=code&columns%5B8%5D%5Bname%5D=code.code&columns%5B8%5D%5Bsearchable%5D=true&columns%5B8%5D%5Borderable%5D=true&columns%5B8%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B8%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B9%5D%5Bdata%5D=id&columns%5B9%5D%5Bname%5D=id&columns%5B9%5D%5Bsearchable%5D=false&columns%5B9%5D%5Borderable%5D=true&columns%5B9%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B9%5D%5Bsearch%5D%5Bregex%5D=false&order%5B0%5D%5Bcolumn%5D=9&order%5B0%5D%5Bdir%5D=desc&start=0&length=120&search%5Bvalue%5D=&search%5Bregex%5D=false&author=&port=&type=&tag=&platform=&_=1543280866340"
my_headers = [
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.75 Safari/537.36"]
random_header = random.choice(my_headers)
req = urllib2.Request(url)
req.add_header("User-Agent", random_header)
req.add_header("GET", url)
req.add_header("Host", "www.exploit-db.com")
req.add_header("Referer", "https://www.exploit-db.com/")
req.add_header("X-Requested-With", "XMLHttpRequest")对Struts 2安全公告的采集代码如下：
# 获取Struts 2安全公告
def struts(time):
result = []
time = time.split("-")
# time = 14 November 2018
# 漏洞公告网址 https://struts.apache.org/announce.html
url = 'https://struts.apache.org/announce.html'
# 获取GET响应html
html = get(url).text
vulnerablelist = re.findall('<ul id="markdown-toc">\n(.*?)\n</ul>', html, re.S)[0].split('\n')
for vulnerable in vulnerablelist:
vulnerabledict = {}
vulnerable_name = re.findall('">(.*?)</a></li>', vulnerable, re.S)[0]
if "%s %s %s" % (time[3], time[2], time[0]) in vulnerable_name:
vulnerable_href = re.findall('href="(.*?)"', vulnerable, re.S)[0]
vulnerabledict["name"] = vulnerable_name
vulnerabledict["url"] = url + vulnerable_href
result.append(vulnerabledict)
else:
break
return result对Openssl安全公告的采集代码如下：
# 获取Openssl安全公告
def openssl(time):
result = []
time = time.replace("-", "")
# 漏洞公告网址 https://www.openssl.org/news/vulnerabilities.html#
url = 'https://www.openssl.org/news/vulnerabilities.html'
# 获取GET响应html
html = get(url).text
bs = BeautifulSoup(html, "lxml")
vulnerablelist = bs.find_all('dt')
for vulnerable in vulnerablelist:
vulnerabledict = {}
vulnerable = str(vulnerable)
try:
vulnerable_time = re.findall('<a href="/news/secadv/(.*?).txt">', vulnerable, re.S)[0]
if vulnerable_time == time:
vulnerabledict["name"] = \
re.findall('">(.*?)</a> <a href="/news/secadv/%s.txt">' % vulnerable_time, vulnerable, re.S)[0]
vulnerabledict["url"] = re.findall('<dt><a href="(.*?)" name="', vulnerable, re.S)[0]
result.append(vulnerabledict)
else:
break
except IndexError:
continue
return result将上述采集到的信息结果记录到文件，poython代码如下：
# 获取漏洞信息并写入文件
def get_information(t):
# 格式化时间
time = t.strftime('%Y-%m-%d') # time = "2018-11-26"
TIME = t.strftime('%Y-%b-%B-%d-%A') # TIME = "2018-Nov-November-26-Tuesday"
# 创建结果文件
f = open('%s.txt' % time, 'a+') # 若是'wb'就表示写二进制文件
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]开始收集今日安全漏洞公告' % now
# 获取各大网站安全公告,并写入文件
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]开始获取Seclists安全公告' % now
vulnerable_seclists = seclists(TIME)
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]获取结果写入文件: %s.txt' % (now, time)
f.write(u'<Seclists安全公告>\n')
for vulnerable in vulnerable_seclists:
f.write(u'安全公告:%s, URL:%s\n' % (vulnerable["name"], vulnerable["url"]))
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]Exploit_DB安全公告' % now
vulnerable_exploit_db = exploit_db(time)
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]获取结果写入文件: %s.txt' % (now, time)
f.write(u'<Exploit_DB安全公告>\n')
for vulnerable in vulnerable_exploit_db:
f.write(u'安全公告:%s, URL:%s\n' % (vulnerable["name"], vulnerable["url"]))
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]Struts安全公告' % now
vulnerable_struts = struts(TIME)
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]获取结果写入文件: %s.txt' % (now, time)
f.write(u'<Struts安全公告>\n')
for vulnerable in vulnerable_struts:
f.write(u'安全公告:%s, URL:%s\n' % (vulnerable["name"], vulnerable["url"]))
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]Openssl安全公告' % now
vulnerable_openssl = openssl(time)
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]获取结果写入文件: %s.txt' % (now, time)
f.write(u'<Openssl安全公告>\n')
for vulnerable in vulnerable_openssl:
f.write(u'安全公告:%s, URL:%s\n' % (vulnerable["name"], vulnerable["url"]))
f.close()
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print u'[%s]今日安全漏洞公告收集完成,收集结果请查看: %s.txt' % (now, time)2.自动化任务
接下来，我们只需要设置一个简单的定时器即可，例如默认每天晚上23时脚本开始自动采集漏洞信息。当然，也可以自定义采集时间。代码如下：
# 每天h时m分开始执行任务,默认23时
def main(h=23, m=0):
while True:
now = datetime.datetime.now() # print(now.hour, now.minute)
if now.hour == h and now.minute == m:
get_information(now)
# 每隔60秒检测一次
time.sleep(60)
if __name__ == '__main__':
main(h=9, m=12)总结
本期漏洞信息自动化获取就介绍到这里啦！
本文参与 腾讯云自媒体分享计划，分享自微信公众号。
原始发表：2018-11-30，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
推荐阅读