前两篇文章主要讲了Joern相关技术的设计原理,以及CPG的实际表现
在研究Joern和Neo4j的过程中,我遇到了一个相当大的问题,就是由于我对OverflowDB包括scala和cypher语言都不熟。Joern和Neo4j分别支持这几种冷门语言,而相应的文档其实没有解决我的问题。
所以在继续研究Joern之前,先花时间简单记录一些Joern和Neo4j实用的语法和范例,给自己当个字典随时可以查阅。
寻找对应名字的方法定义的位置
joern> cpg.method.name("getRequestBody").l
val res4: List[io.shiftleft.codepropertygraph.generated.nodes.Method] = List(
Method(
id = 15337L,
astParentFullName = "<empty>",
astParentType = "<empty>",
code = "public static String getRequestBody(HttpServletRequest request) throws IOException",
columnNumber = Some(value = 5),
columnNumberEnd = Some(value = 5),
filename = "src\\main\\java\\org\\joychou\\util\\WebUtils.java",
fullName = "org.joychou.util.WebUtils.getRequestBody:<unresolvedSignature>(1)",
hash = None,
isExternal = false,
lineNumber = Some(value = 13),
lineNumberEnd = Some(value = 16),
name = "getRequestBody",
order = 1,
signature = "<unresolvedSignature>(1)"
)
)
寻找对应方法/函数调用的位置
joern> cpg.call("getRequestBody").take(1).l
val res7: List[io.shiftleft.codepropertygraph.generated.nodes.Call] = List(
Call(
id = 8485L,
argumentIndex = 2,
argumentName = None,
code = "getRequestBody(request)",
columnNumber = Some(value = 22),
dispatchType = "DYNAMIC_DISPATCH",
dynamicTypeHintFullName = ArraySeq(),
lineNumber = Some(value = 25),
methodFullName = "org.joychou.util.WebUtils.getRequestBody:<unresolvedSignature>(1)",
name = "getRequestBody",
order = 2,
possibleTypes = ArraySeq(),
signature = "<unresolvedSignature>(1)",
typeFullName = "java.lang.String"
)
)
寻找对应名字注解的节点
joern> cpg.annotation.name(".*Mapping").take(1).l
val res10: List[io.shiftleft.codepropertygraph.generated.nodes.Annotation] = List(
Annotation(
id = 2532L,
argumentIndex = -1,
argumentName = None,
code = "@RequestMapping(\"/safecode\")",
columnNumber = Some(value = 5),
fullName = "org.springframework.web.bind.annotation.RequestMapping",
lineNumber = Some(value = 20),
name = "RequestMapping",
order = 7
)
)
在joern的节点你都可以非常简单的用点连接来获取对应节点连接的其他节点
joern> cpg.annotation.name(".*Mapping").method.take(1).l
val res21: List[io.shiftleft.codepropertygraph.generated.nodes.Method] = List(
Method(
id = 2498L,
astParentFullName = "<empty>",
astParentType = "<empty>",
code = "public void crlf(HttpServletRequest request, HttpServletResponse response)",
columnNumber = Some(value = 5),
columnNumberEnd = Some(value = 5),
filename = "src\\main\\java\\org\\joychou\\controller\\CRLFInjection.java",
fullName = "org.joychou.controller.CRLFInjection.crlf:<unresolvedSignature>(2)",
hash = None,
isExternal = false,
lineNumber = Some(value = 20),
lineNumberEnd = Some(value = 28),
name = "crlf",
order = 1,
signature = "<unresolvedSignature>(2)"
)
)
除了常规的method,annotation,call这种以外,比较常见的节点类型还有
https://docs.joern.io/cpgql/node-type-steps/
当然除了上面的这些节点以外,还有一些调用关系的通用节点
返回节点列表对应节点的被调用节点,也就是父节点
返回节点列表对应节点的调用节点,也就是子节点
返回节点列表对应父节点的所有节点
凡是节点连接的都是作为结果传到下一级的,如果是想筛选符合条件的节点则需要用where或者属性过滤器,比如说
查询名字为getRequestBody,这个name就是属性过滤器,向下一级返回的是符合属性过滤器的method节点
或者用where也行,where语句内容会作为筛选条件影响返回的method内容
joern> cpg.method.name.l
val res36: List[String] = List(
"configure",
"main",
...
如果不是使用()作为属性过滤器,那么返回内容就会直接变成name属性列表。
当然除了where以外,也支持很多种过滤器
在处理结果返回的时候也有一些方式改变返回的内容。一般来说查询结果会是一个字典列表。
joern> cpg.method.name("getRequestBody").l
val res64: List[io.shiftleft.codepropertygraph.generated.nodes.Method] = List(
Method(
id = 15337L,
astParentFullName = "<empty>",
astParentType = "<empty>",
code = "public static String getRequestBody(HttpServletRequest request) throws IOException",
columnNumber = Some(value = 5),
columnNumberEnd = Some(value = 5),
filename = "src\\main\\java\\org\\joychou\\util\\WebUtils.java",
fullName = "org.joychou.util.WebUtils.getRequestBody:<unresolvedSignature>(1)",
hash = None,
isExternal = false,
lineNumber = Some(value = 13),
lineNumberEnd = Some(value = 16),
name = "getRequestBody",
order = 1,
signature = "<unresolvedSignature>(1)"
)
)
可以用map来改变返回的结构,这是一个类似于lambda的语法,会遍历列表的所有节点然后生成结果.
joern> cpg.method.name("getRequestBody").map(n=>List(n.filename, n.lineNumber, n.fullName, n.code)).l
val res66: List[List[String | Option[Integer]]] = List(
List(
"src\\main\\java\\org\\joychou\\util\\WebUtils.java",
Some(value = 13),
"org.joychou.util.WebUtils.getRequestBody:<unresolvedSignature>(1)",
"public static String getRequestBody(HttpServletRequest request) throws IOException"
)
)
除了map以外,另外还有两个实用的
既然需要寻找两个节点之间的路径,那么就少不了重复,重复获取父级节点就是最简单的一种数据流分析。
重复获取caller共5次,如果找不到结果就会停止
重复调用 caller 查询,直到找到一个方法名为 foo 的方法,找不到就返回空。
emit的意思是会将查询的过程节点作为返回的列表中的一员。
上面这句语句就是指,重复5次获取当前节点的caller的节点属性,除此之外还会带上路径上所有满足isMethod的节点。
Joern对于返回结果提供了公式化的输出格式,而且如果不指定输出直接就没有返回
def source = cpg.method.where(_.annotation.name(".*Mapping")).parameter
def sink = cpg.call.name("exec")
Neo4j的语法在我看来要比Joern的语法别扭多了,但有些问题其实在Neo4j会更容易得到答案,可视化的图结构在某些情况下会有非常明显的优势。
create (n:Person)-[:LOVES]->(m:Dog)
create (z:ziduan{name:"f_name",table:"dianlibiao"}) return count(*)
create (n:Person{name:"李四"})-[:FEAR{level:1}]->(t:Tiger{type:"东北虎"})
match (n:Person{name:""王五""}), (m:Person{name:"赵六"}) create (n)-[k:KNOW]->(m) return k
match (n:Person{name:"李四"})-[f:FEAR]->(t:Tiger) delete f
match(m)-[b:bian]-(n) delete b
match (n:Person{name:"李四"}) delete n
match(n) detach delete n
match (n) delete n
match (n) detach delete n
MATCH (r:Loc) DETACH DELETE r
match
(node)-[relationship]->(node)
where
(node | relationship)
return
(node | relationship)
match (n:Persion)-[:HAS_PHONE]->(p:Phone) where n.name="姓名6" return n, p limit 10
match(n) return n
match (t:Tiger) where id(t)=1837 return t
match (t:Tiger) where id(t)=1837 delete t
match (n:Persion)-[:HAS_PHONE]->(p:Phone)-[:CALL]->(p1: Persion) where n.name=“姓名6” return n, p,p1 limit 10
match p=()-[c: CALL]->() return p limit 10
match (n:USERS) where n.name=~'Jack.*' return n limit 10
match (n:USERS) where n.name= contains 'J' return n limit 10
match (n:Person{name:"王五"}), (m:Person{name:"赵六"}) return n,m
match(n) where n:标签1 or n:标签B return distinct n;
match(n) where not n:标签1 and not n:标签B return distinct n;
match (t:Tiger) where id(t)=1837 set t:A return t
本质上是给实体增加一个标签,一个实体可以有多个标签
match (a:A) where id(t)=1837 set a.年龄=10 return a
match (n:Person)-[l:LOVE]->(:Person) set l.date="1990" return n, l
match(n) set n:table return n
match (p1:Person{name:"姓名2"}),(p2:Person{name:"姓名10"}), p=shortestpath((p1)-[*..10]-(p2)) return p
shortestpath()用于查询最短路径 _..10 表示关系中*_不超过10度关系**
match (p1:Person{name:"姓名2"}),(p2:Person{name:"姓名10"}), p=allshortestpaths((p1)-[*..10]-(p2)) return p
match (n:Person{name:"王五"}), (m:Person{name:"赵六"}) merge (n)-[l:LOVE]->(m) return l
match (n),(m) where n=m merge (n)-[t:TABLE{table_name:n.table}]-(m) return t
OPTIONAL MATCH (n)-[r]->(m) RETURN m
匹配结果集中如果有丢的部分,则会用null来补充
MATCH (n)
WHERE n.name STARTS WITH '张'
RETURN n
MATCH (n)
WHERE n.name ENDS WITH '三'
RETURN n