networking-sfc deep dive 4

惠伟

发布于 2021-02-24 11:22:09

4430

发布于 2021-02-24 11:22:09

前面的文章介绍的流量路径比较简单，就是一条chain，从头走到尾巴，最多LB一下tap一下。有些需求是把一条chain的流量分叉到不同chain或者把不同chain上的流量合并到一条chain上，上networking-sfc中叫做service graph，这儿简单介绍一下。

拓扑

逻辑拓扑

流量从src出来到sf1，分叉到sf2和sf3，sf4合并sf2和sf3出来的流量，最后送给dst

物理拓扑

物理上两说有两台计算节点，sf1和sf4运行在test25g05上，src,sf2,sf3和dst运行上test25g06上

配置命令

openstack network create sfc-net-test5

openstack subnet create --network sfc-net-test5 --subnet-range 5.0.0.0/24 sfc-subnet-test5


openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_src_route
openstack port create --no-security-group --disable-port-security --enable --network provider port_in_mgmt_route

openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_p1_route
openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_p2_route
openstack port create --no-security-group --disable-port-security --enable --network provider port_sf1_mgmt_route

openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_p3_route
openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_p4_route
openstack port create --no-security-group --disable-port-security --enable --network provider port_sf2_mgmt_route

openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_p5_route
openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_p6_route
openstack port create --no-security-group --disable-port-security --enable --network provider port_sf3_mgmt_route

openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_p7_route
openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_p8_route
openstack port create --no-security-group --disable-port-security --enable --network provider port_sf4_mgmt_route

openstack port create --no-security-group --disable-port-security --enable --network sfc-net-test5 port_dst_route
openstack port create --no-security-group --disable-port-security --enable --network provider port_out_mgmt_route


openstack server create --image centos7-hw --flavor centos7-flavor --port port_in_mgmt_route  --port port_src_route src_route
openstack server create --image centos7-hw --flavor centos7-flavor --port port_sf1_mgmt_route --port port_p1_route --port port_p2_route sf1_route
openstack server create --image centos7-hw --flavor centos7-flavor --port port_sf2_mgmt_route --port port_p3_route --port port_p4_route sf2_route
openstack server create --image centos7-hw --flavor centos7-flavor --port port_sf3_mgmt_route --port port_p5_route --port port_p6_route sf3_route
openstack server create --image centos7-hw --flavor centos7-flavor --port port_sf4_mgmt_route --port port_p7_route --port port_p8_route sf4_route
openstack server create --image centos7-hw --flavor centos7-flavor --port port_out_mgmt_route --port port_dst_route dst_route


openstack sfc port pair create --service-function-parameters correlation=mpls --ingress port_p1_route --egress port_p2_route port-pair-test1_route
openstack sfc port pair create --service-function-parameters correlation=mpls --ingress port_p3_route --egress port_p4_route port-pair-test2_route
openstack sfc port pair create --service-function-parameters correlation=mpls --ingress port_p5_route --egress port_p6_route port-pair-test3_route
openstack sfc port pair create --service-function-parameters correlation=mpls --ingress port_p7_route --egress port_p8_route port-pair-test4_route

openstack sfc port pair group create --port-pair port-pair-test1_route port-pair-group-test1_route
openstack sfc port pair group create --port-pair port-pair-test2_route port-pair-group-test2_route
openstack sfc port pair group create --port-pair port-pair-test3_route port-pair-group-test3_route
openstack sfc port pair group create --port-pair port-pair-test4_route port-pair-group-test4_route



openstack sfc flow classifier create --logical-source-port port_src_route --source-ip-prefix 5.0.0.150/32 --destination-ip-prefix 5.0.0.105/32 --protocol tcp --destination-port 80:81 flowclassifier-test1_route
openstack sfc flow classifier create --logical-source-port port_p2_route --source-ip-prefix 5.0.0.150/32 --destination-ip-prefix 5.0.0.105/32 --protocol tcp --destination-port 8080:8080 flowclassifier-test2_route
openstack sfc flow classifier create --logical-source-port port_p2_route --source-ip-prefix 5.0.0.150/32 --destination-ip-prefix 5.0.0.105/32 --protocol tcp --destination-port 8181:8181 flowclassifier-test3_route
openstack sfc flow classifier create --logical-source-port port_p4_route --source-ip-prefix 5.0.0.150/32 --destination-ip-prefix 5.0.0.105/32 --protocol tcp --destination-port 8888:8888 flowclassifier-test4_1_route
openstack sfc flow classifier create --logical-source-port port_p6_route --source-ip-prefix 5.0.0.150/32 --destination-ip-prefix 5.0.0.105/32 --protocol tcp --destination-port 9999:9999 flowclassifier-test4_2_route




openstack sfc port chain create --chain-parameters correlation=mpls --flow-classifier flowclassifier-test1_route --port-pair-group port-pair-group-test1_route  sfc-port-chain-test1_route
openstack sfc port chain create --chain-parameters correlation=mpls --flow-classifier flowclassifier-test2_route --port-pair-group port-pair-group-test2_route  sfc-port-chain-test2_route
openstack sfc port chain create --chain-parameters correlation=mpls --flow-classifier flowclassifier-test3_route --port-pair-group port-pair-group-test3_route  sfc-port-chain-test3_route
openstack sfc port chain create --chain-parameters correlation=mpls --flow-classifier flowclassifier-test4_1_route --flow-classifier flowclassifier-test4_2_route --port-pair-group port-pair-group-test4_route  sfc-port-chain-test4_route

openstack sfc service graph  create --branching-point sfc-port-chain-test1_route:sfc-port-chain-test2_route,sfc-port-chain-test3_route --branching-point sfc-port-chain-test2_route:sfc-port-chain-test4_route --branching-point sfc-port-chain-test3_route:sfc-port-chain-test4_route service-graph-test1_route

流表

最大的特点就是分叉点或者合并点引进了reg0。

原来的流表match in_port=xxx和 flowclassifier，然后就到group table。

现在变成in_port前一个port chain的最后一个sf的egress port，把label暂存在reg0中，resubmit(,0)，然后match reg0和flowclassifier送去group table，把原来的一条流表变成了多条。

group table相同就合并，group table不同就分叉了。

test25g06上的流表

[root@test25g06 neutron]# ovs-ofctl dump-flows br-int |grep  priority=30
 cookie=0xfc571aa5284c5479, duration=1394.536s, table=0, n_packets=0, n_bytes=0, idle_age=2952, priority=30,tcp,in_port=794,nw_src=5.0.0.150,nw_dst=5.0.0.105,tp_dst=0x50/0xfffe actions=push_mpls:0x8847,load:0x1ff->OXM_OF_MPLS_LABEL[],set_mpls_ttl(255),group:1
 cookie=0xfc571aa5284c5479, duration=70.783s, table=0, n_packets=0, n_bytes=0, idle_age=70, priority=30,mpls,in_port=806,mpls_label=1022 actions=load:0x3fe->NXM_NX_REG0[],pop_mpls:0x0800,resubmit(,0)
 cookie=0xfc571aa5284c5479, duration=69.793s, table=0, n_packets=0, n_bytes=0, idle_age=69, priority=30,mpls,in_port=805,mpls_label=766 actions=load:0x2fe->NXM_NX_REG0[],pop_mpls:0x0800,resubmit(,0)
 cookie=0xfc571aa5284c5479, duration=69.436s, table=0, n_packets=0, n_bytes=0, idle_age=69, priority=30,tcp,reg0=0x3fe,nw_src=5.0.0.150,nw_dst=5.0.0.105,tp_dst=9999 actions=push_mpls:0x8847,load:0x4ff->OXM_OF_MPLS_LABEL[],set_mpls_ttl(255),group:4
 cookie=0xfc571aa5284c5479, duration=69.397s, table=0, n_packets=0, n_bytes=0, idle_age=69, priority=30,tcp,reg0=0x3fe,nw_src=5.0.0.150,nw_dst=5.0.0.105,tp_dst=8888 actions=push_mpls:0x8847,load:0x4ff->OXM_OF_MPLS_LABEL[],set_mpls_ttl(255),group:4
 cookie=0xfc571aa5284c5479, duration=69.357s, table=0, n_packets=0, n_bytes=0, idle_age=69, priority=30,tcp,reg0=0x2fe,nw_src=5.0.0.150,nw_dst=5.0.0.105,tp_dst=9999 actions=push_mpls:0x8847,load:0x4ff->OXM_OF_MPLS_LABEL[],set_mpls_ttl(255),group:4
 cookie=0xfc571aa5284c5479, duration=69.315s, table=0, n_packets=0, n_bytes=0, idle_age=69, priority=30,tcp,reg0=0x2fe,nw_src=5.0.0.150,nw_dst=5.0.0.105,tp_dst=8888 actions=push_mpls:0x8847,load:0x4ff->OXM_OF_MPLS_LABEL[],set_mpls_ttl(255),group:4
[root@test25g06 neutron]#
[root@test25g06 neutron]#
[root@test25g06 neutron]# ovs-ofctl dump-flows br-int |grep  priority=20
 cookie=0xfc571aa5284c5479, duration=31448.101s, table=0, n_packets=0, n_bytes=0, idle_age=65534, priority=20,dl_type=0x894f actions=resubmit(,10)
[root@test25g06 neutron]#
[root@test25g06 neutron]#
[root@test25g06 neutron]# ovs-ofctl dump-flows br-int table=5
 cookie=0xfc571aa5284c5479, duration=1405.098s, table=5, n_packets=0, n_bytes=0, priority=0,mpls,dl_dst=fa:16:3e:75:e8:3f actions=mod_vlan_vid:15,output:"patch-tun"
 cookie=0xfc571aa5284c5479, duration=80.257s, table=5, n_packets=0, n_bytes=0, priority=0,mpls,dl_dst=fa:16:3e:02:9c:e2 actions=mod_vlan_vid:15,output:"patch-tun"
[root@test25g06 neutron]#
[root@test25g06 neutron]# ovs-ofctl dump-flows br-int table=10
 cookie=0xfc571aa5284c5479, duration=84.004s, table=10, n_packets=0, n_bytes=0, priority=1,mpls,dl_vlan=15,dl_dst=fa:16:3e:d5:99:ae,mpls_label=1023 actions=strip_vlan,output:"qvoe49aa5d2-a5"
 cookie=0xfc571aa5284c5479, duration=82.999s, table=10, n_packets=0, n_bytes=0, priority=1,mpls,dl_vlan=15,dl_dst=fa:16:3e:f0:d7:2d,mpls_label=767 actions=strip_vlan,output:"qvodd1e56e7-7c"
 cookie=0xfc571aa5284c5479, duration=31457.905s, table=10, n_packets=0, n_bytes=0, priority=0 actions=drop

[root@test25g06 neutron]# ovs-ofctl dump-groups br-int
NXST_GROUP_DESC reply (xid=0x2):
 group_id=1,type=select,bucket=bucket_id:0,actions=mod_dl_dst:fa:16:3e:75:e8:3f,resubmit(,5)
 group_id=4,type=select,bucket=bucket_id:0,actions=mod_dl_dst:fa:16:3e:02:9c:e2,resubmit(,5)

test25g05上的流表

[root@test25g05 /home/huiwei]# ovs-ofctl dump-flows br-int |grep  priority=30
cookie=0x6ecffb0baaf418f8, duration=71.117s, table=0, n_packets=0, n_bytes=0, idle_age=77, priority=30,mpls,in_port=1222,mpls_label=1278 actions=pop_mpls:0x0800,NORMAL
 cookie=0x6ecffb0baaf418f8, duration=59.338s, table=0, n_packets=0, n_bytes=0, idle_age=66, priority=30,mpls,in_port=1224,mpls_label=510 actions=load:0x1fe->NXM_NX_REG0[],pop_mpls:0x0800,resubmit(,0)
 cookie=0xb0b9906cddc30223, duration=141.038s, table=0, n_packets=0, n_bytes=0, idle_age=141, priority=30,tcp,reg0=0x1fe,nw_src=5.0.0.150,nw_dst=5.0.0.105,tp_dst=8181 actions=push_mpls:0x8847,load:0x3ff->OXM_OF_MPLS_LABEL[],set_mpls_ttl(255),group:3
 cookie=0xb0b9906cddc30223, duration=140.992s, table=0, n_packets=0, n_bytes=0, idle_age=140, priority=30,tcp,reg0=0x1fe,nw_src=5.0.0.150,nw_dst=5.0.0.105,tp_dst=8080 actions=push_mpls:0x8847,load:0x2ff->OXM_OF_MPLS_LABEL[],set_mpls_ttl(255),group:2
[root@test25g05 /home/huiwei]# ovs-ofctl dump-flows br-int |grep  priority=20
 cookie=0xb0b9906cddc30223, duration=91277.675s, table=0, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=20,dl_type=0x894f actions=resubmit(,10)
[root@test25g05 /home/huiwei]#
[root@test25g05 /home/huiwei]# ovs-ofctl dump-flows br-int table=5
 cookie=0xb0b9906cddc30223, duration=151.410s, table=5, n_packets=0, n_bytes=0, priority=0,mpls,dl_dst=fa:16:3e:d5:99:ae actions=mod_vlan_vid:31,output:"patch-tun"
 cookie=0xb0b9906cddc30223, duration=151.366s, table=5, n_packets=0, n_bytes=0, priority=0,mpls,dl_dst=fa:16:3e:f0:d7:2d actions=mod_vlan_vid:31,output:"patch-tun"
[root@test25g05 /home/huiwei]# ovs-ofctl dump-flows br-int table=10
 cookie=0xb0b9906cddc30223, duration=507.935s, table=10, n_packets=0, n_bytes=0, priority=1,mpls,dl_vlan=31,dl_dst=fa:16:3e:02:9c:e2,mpls_label=1279 actions=strip_vlan,output:"qvo539d7116-00"
 cookie=0xb0b9906cddc30223, duration=154.068s, table=10, n_packets=0, n_bytes=0, priority=1,mpls,dl_vlan=31,dl_dst=fa:16:3e:75:e8:3f,mpls_label=511 actions=strip_vlan,output:"qvoc2502a93-27"
 cookie=0xb0b9906cddc30223, duration=91286.558s, table=10, n_packets=0, n_bytes=0, priority=0 actions=drop

[root@test25g05 /home/huiwei]# ovs-ofctl dump-groups br-int
NXST_GROUP_DESC reply (xid=0x2):
 group_id=2,type=select,bucket=bucket_id:0,actions=mod_dl_dst:fa:16:3e:f0:d7:2d,resubmit(,5)
 group_id=3,type=select,bucket=bucket_id:0,actions=mod_dl_dst:fa:16:3e:d5:99:ae,resubmit(,5)

问题记录

没有配置service graph时就是四条port chain的流表，配置service graph后要做流表做修改，但测试发现有问题

定位发现是controller rpc消息没有发送给正确的节点

https://review.opendev.org/#/c/714242/review.opendev.org

sf要处理label和修改报文特征，如tcp port

sf的port pair要有correlation(mpls or nsh)，否则报错，而且sf每一跳都要把label值减1 Service Graphs require source (branching) and destination port pair groups (their PPs) to have correlation enabled.

sf迁移DB中port pair detail中host_id不变

暂未解决

flowclassifier logical source port要设置对，虽然不在match in_port了，但是reg0流表要下在logical source port所在的物理机

flowclassifier Conflict 或者 InUse

我想配置的是sf1 match tcp port 80和81，然后sf2 match tcp port 80，sf3 match tcp port 81，相当于分叉了，最后sf4再match tcp port 80和81，实现合并

问题是总是提示flowclassifier Conflict或者InUse，flowclassifier中不指定默认全match，很容易冲突，ovs只有in_port=1没有in_port!=1。

个人想法是把logical-source-port移到port chain命令行中，egress+correlation+flowclassifier一样才算冲突，egress(in_port or reg0)，correation(None,mpls,nsh) logical-source-port所在的节点算是port chain中第一个节点，它的ingress是空，egress是logical source port

本文参与腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。

如有侵权请联系 cloudcommunity@tencent.com 删除

数据结构

本文分享自作者个人站点/博客前往查看

如有侵权，请联系 cloudcommunity@tencent.com 删除。

本文参与腾讯云自媒体同步曝光计划，欢迎热爱写作的你一起参与！

数据结构

登录后参与评论

0 条评论

热度