如何对kubernetes scheduler进行二次开发

通过新增Predicates&Priorities Policies来扩展default scheduler

新增Predicate Policy

  • predicate Interface
plugin/pkg/scheduler/algorithm/types.go:31

// FitPredicate is a function that indicates if a pod fits into an existing node.
// The failure information is given by the error.

type FitPredicate func(pod *v1.Pod, meta interface{}, nodeInfo *schedulercache.NodeInfo) (bool, []PredicateFailureReason, error)
  • Implement a predicate func
func PodFitsHostNew(pod *v1.Pod, meta interface{}, nodeInfo *schedulercache.NodeInfo) (bool, []algorithm.PredicateFailureReason, error) {
	if len(pod.Spec.NodeName) == 0 {
		return true, nil, nil
	}
	node := nodeInfo.Node()
	if node == nil {
		return false, nil, fmt.Errorf("node not found")
	}
	if pod.Spec.NodeName == node.Name {
		return true, nil, nil
	}
	return false, []algorithm.PredicateFailureReason{ErrPodNotMatchHostName}, nil
}
  • register the custom predicate policy with a custom name
plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go:47

func init() {
	...
	
	factory.RegisterAlgorithmProvider(factory.DefaultProvider, defaultPredicates(), defaultPriorities())
	// Cluster autoscaler friendly scheduling algorithm.
	factory.RegisterAlgorithmProvider(ClusterAutoscalerProvider, defaultPredicates(),
		copyAndReplace(defaultPriorities(), "LeastRequestedPriority", "MostRequestedPriority"))
	...
	
	factory.RegisterFitPredicate("CustomPredicatePolicy", predicates.PodFitsHostNew)
	
	...
}	
  • rebuild kube-scheduler and restart with flag of --policy-config-file

kube-scheduler xxxx --policy-config-file=/var/lib/kube-scheduler/policy.config

  • the content of --policy-config-file specified file
/var/lib/kube-scheduler/policy.config

{
"kind" : "Policy",
"apiVersion" : "v1",
"predicates" : [
    {"name" : "CustomPredicatePolicy"}
    ],
"priorities" : [
    ]
}

新增Priority Policy

  • Priority Interface
/Users/garnett/workspace/go/src/k8s.io/kubernetes/plugin/pkg/scheduler/algorithm/types.go

// PriorityMapFunction is a function that computes per-node results for a given node.

type PriorityMapFunction func(pod *v1.Pod, meta interface{}, nodeInfo *schedulercache.NodeInfo) (schedulerapi.HostPriority, error)
  • Implement a predicate func
  • register the custom predicate policy with a custom name
  • rebuild kube-scheduler and restart with flag of --policy-config-file
  • the content of --policy-config-file specified file
/var/lib/kube-scheduler/policy.config

{
"kind" : "Policy",
"apiVersion" : "v1",
"predicates" : [
    ],
"priorities" : [
    {"name" : "CumtomPriorityPolicy", "weight" : 1}
    ]
}

新增custom scheduler,pod指定scheduler-name进行调度

  • A custom scheduler can be written in any language and can be as simple or complex as you need.
  • Specify the “scheduleName” in pod.spec
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  schedulerName: my-scheduler
  containers:
  - name: nginx
    image: nginx:1.10

Here is a very simple example of a custom scheduler written in Bash that assigns a node randomly. Note that you need to run this along with kubectl proxy for it to work.

kubectl proxy --port=8001

#!/bin/bash
SERVER='localhost:8001'
while true;
do
    for PODNAME in $(kubectl --server $SERVER get pods -o json | jq '.items[] | select(.spec.schedulerName == "my-scheduler") | select(.spec.nodeName == null) | .metadata.name' | tr -d '"')
;
    do
        NODES=($(kubectl --server $SERVER get nodes -o json | jq '.items[].metadata.name' | tr -d '"'))
        NUMNODES=${#NODES[@]}
        CHOSEN=${NODES[$[ $RANDOM % $NUMNODES ]]}
        curl --header "Content-Type:application/json" --request POST --data '{"apiVersion":"v1", "kind": "Binding", "metadata": {"name": "'$PODNAME'"}, "target": {"apiVersion": "v1", "kind"
: "Node", "name": "'$CHOSEN'"}}' http://$SERVER/api/v1/namespaces/default/pods/$PODNAME/binding/
        echo "Assigned $PODNAME to $CHOSEN"
    done
    sleep 1
done

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏JetpropelledSnake

RESTful源码笔记之RESTful Framework的基本组件

开发我们的Web API的第一件事是为我们的Web API提供一种将代码片段实例序列化和反序列化为诸如json之类的表示形式的方式。我们可以通过声明与Djang...

1102
来自专栏10km的专栏

cmake:vs2015/MinGW静态编译leveldb

leveldb是google的开源项目(https://github.com/google/leveldb), 在linux下编译很方便,然而官方版本却没有提供...

5826
来自专栏技术沉淀

Rails里应用Devise

1196
来自专栏杨建荣的学习笔记

PCIE的简单配置(r8笔记第82天)

最近测试了一下PCIE-SSD在数据库环境的迁移 和加压情况,IOPS无可置疑,比起机械硬盘确实是高了很多个量级,在数据环境中的IO方面确实有很稳定的提升,目...

3788
来自专栏后台及大数据开发

CentOS下redis集群安装

环境: 一台CentOS虚拟机上部署六个节点,创建3个master,3个slave节点

902
来自专栏技术沉淀

Rails应用分页: Will Paginate

1113
来自专栏数据库新发现

Apache的一个错误:[error] an unknown filter was not added: PHP

在apache的日志里经常看到这个错误: [Fri Oct 08 14:49:18 2004] [error] an unknown filter was n...

1612
来自专栏Golang语言社区

一个用Go编写的WIP最小文本编辑器

Phi is a minimal code editor designed to look pretty, run fast, and be easy to c...

1292
来自专栏杨建荣的学习笔记

rac节点无法启动ORA-29702的问题及分析(70天)

今天在虚拟机上启动rac,发现有一个节点怎么都起不了。另外一个节点没问题。 SQL> startup nomount ORA-29702: error occ...

6046
来自专栏Java学习网

在Java EE7框架中使用MongoDB

中心点创建应用程序的执行在企业环境中,应用程序必须安全、便携和高可用性。它还必须能够与不同的系统交互,但可控的从一个最好的位置。JEE7合并是一个重要的框架的所...

3076

扫码关注云+社区

领取腾讯云代金券