文章/答案/技术大牛

发布

社区首页 >问答首页 >使用NodeGroup时，CloudFormation不会加入EKS集群

问使用NodeGroup时，CloudFormation不会加入EKS集群
EN

Stack Overflow用户

提问于 2020-04-27 18:57:51

回答 2查看 4.8K关注 0票数 0

我一直在跟踪这个指南，以便通过CloudFormation创建一个Kubernetes集群，但是NodeGroup从来没有加入过这个集群，我也没有收到任何关于为什么不加入的错误或解释。

我可以看到自动标度组和EC2机器是创建的，但是EKS报告说没有节点组。

如果我通过web管理工具手动创建一个新的节点组，它可以工作，但是它分配了不同的security groups。它有一个launch template而不是一个launch configuration。

相同的AMI，相同的IAM角色，相同的机器类型.

我在CloudFormation和EKS方面都是新手，我不知道现在该如何着手找出问题所在。

下面是模板：

Description: >
    Kubernetes cluster

Parameters:

  EnvironmentName:
    Description: An environment name that will be prefixed to resource names
    Type: String

  KeyName:
    Description: The EC2 Key Pair to allow SSH access to the instances
    Type: AWS::EC2::KeyPair::KeyName

  VpcBlock:
    Type: String
    Default: 192.168.0.0/16
    Description: The CIDR range for the VPC. This should be a valid private (RFC 1918) CIDR range.

  Subnet01Block:
    Type: String
    Default: 192.168.64.0/18
    Description: CidrBlock for subnet 01 within the VPC

  Subnet02Block:
    Type: String
    Default: 192.168.128.0/18
    Description: CidrBlock for subnet 02 within the VPC

  Subnet03Block:
    Type: String
    Default: 192.168.192.0/18
    Description: CidrBlock for subnet 03 within the VPC. This is used only if the region has more than 2 AZs.

  NodeInstanceType:
    Description: EC2 instance type for the node instances
    Type: String

  NodeImageId:
    Type: AWS::EC2::Image::Id
    Description: AMI id for the node instances.

  NodeAutoScalingGroupMinSize:
    Type: Number
    Description: Minimum size of Node Group ASG.
    Default: 1

  NodeAutoScalingGroupMaxSize:
    Type: Number
    Description: Maximum size of Node Group ASG. Set to at least 1 greater than NodeAutoScalingGroupDesiredCapacity.
    Default: 3

  NodeAutoScalingGroupDesiredCapacity:
    Type: Number
    Description: Desired capacity of Node Group ASG.
    Default: 3

  BootstrapArguments:
    Description: Arguments to pass to the bootstrap script. See files/bootstrap.sh in https://github.com/awslabs/amazon-eks-ami
    Default: ""
    Type: String

Resources:

  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock:  !Ref VpcBlock
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Environment 
          Value: !Ref EnvironmentName

  InternetGateway:
    Type: "AWS::EC2::InternetGateway"
    Properties:
      Tags:
        - Key: Environment 
          Value: !Ref EnvironmentName

  VPCGatewayAttachment:
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

  RouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Environment 
          Value: !Ref EnvironmentName

  Route:
    DependsOn: VPCGatewayAttachment
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref RouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway

  Subnet01:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select [ 0, !GetAZs '' ]
      CidrBlock: !Ref Subnet01Block
      VpcId: !Ref VPC
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Environment 
          Value: !Ref EnvironmentName

  Subnet02:
    Type: AWS::EC2::Subnet
    Metadata:
      Comment: Subnet 02
    Properties:
      AvailabilityZone: !Select [ 1, !GetAZs '' ]
      CidrBlock: !Ref Subnet02Block
      VpcId: !Ref VPC
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Environment 
          Value: !Ref EnvironmentName

  Subnet03:
    Type: AWS::EC2::Subnet
    Metadata:
      Comment: Subnet 03
    Properties:
      AvailabilityZone: !Select [ 2, !GetAZs '' ]
      CidrBlock: !Ref Subnet03Block
      VpcId: !Ref VPC
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Environment 
          Value: !Ref EnvironmentName

  Subnet01RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref Subnet01
      RouteTableId: !Ref RouteTable

  Subnet02RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref Subnet02
      RouteTableId: !Ref RouteTable

  Subnet03RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref Subnet03
      RouteTableId: !Ref RouteTable

  ControlPlaneSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Cluster communication with worker nodes
      VpcId: !Ref VPC

  ClusterRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${EnvironmentName}KubernetesClusterRole
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: eks.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonEKSServicePolicy
        - arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
      Tags:
        - Key: Environment 
          Value: !Ref EnvironmentName

  Cluster:
    Type: AWS::EKS::Cluster
    Properties:
      Name: !Sub ${EnvironmentName}KubernetesCluster
      RoleArn: !GetAtt ClusterRole.Arn
      ResourcesVpcConfig:
        SecurityGroupIds:
          - !Ref ControlPlaneSecurityGroup
        SubnetIds:
          - !Ref Subnet01
          - !Ref Subnet02
          - !Ref Subnet03

  NodeRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${EnvironmentName}KubernetesNodeRole
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
      Path: /
      Tags:
        - Key: Environment 
          Value: !Ref EnvironmentName

  NodeInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: "/"
      Roles:
      - !Ref NodeRole

  NodeSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for all nodes in the cluster
      VpcId: !Ref VPC
      Tags:
      - Key: !Sub "kubernetes.io/cluster/${EnvironmentName}KubernetesCluster"
        Value: 'owned'
      - Key: Environment 
        Value: !Ref EnvironmentName

  NodeSecurityGroupIngress:
    Type: AWS::EC2::SecurityGroupIngress
    DependsOn: NodeSecurityGroup
    Properties:
      Description: Allow node to communicate with each other
      GroupId: !Ref NodeSecurityGroup
      SourceSecurityGroupId: !Ref NodeSecurityGroup
      IpProtocol: '-1'
      FromPort: 0
      ToPort: 65535

  NodeSecurityGroupFromControlPlaneIngress:
    Type: AWS::EC2::SecurityGroupIngress
    DependsOn: NodeSecurityGroup
    Properties:
      Description: Allow worker Kubelets and pods to receive communication from the cluster control plane
      GroupId: !Ref NodeSecurityGroup
      SourceSecurityGroupId: !Ref ControlPlaneSecurityGroup
      IpProtocol: tcp
      FromPort: 1025
      ToPort: 65535

  ControlPlaneEgressToNodeSecurityGroup:
    Type: AWS::EC2::SecurityGroupEgress
    DependsOn: NodeSecurityGroup
    Properties:
      Description: Allow the cluster control plane to communicate with worker Kubelet and pods
      GroupId: !Ref ControlPlaneSecurityGroup
      DestinationSecurityGroupId: !Ref NodeSecurityGroup
      IpProtocol: tcp
      FromPort: 1025
      ToPort: 65535

  NodeSecurityGroupFromControlPlaneOn443Ingress:
    Type: AWS::EC2::SecurityGroupIngress
    DependsOn: NodeSecurityGroup
    Properties:
      Description: Allow pods running extension API servers on port 443 to receive communication from cluster control plane
      GroupId: !Ref NodeSecurityGroup
      SourceSecurityGroupId: !Ref ControlPlaneSecurityGroup
      IpProtocol: tcp
      FromPort: 443
      ToPort: 443

  ControlPlaneEgressToNodeSecurityGroupOn443:
    Type: AWS::EC2::SecurityGroupEgress
    DependsOn: NodeSecurityGroup
    Properties:
      Description: Allow the cluster control plane to communicate with pods running extension API servers on port 443
      GroupId: !Ref ControlPlaneSecurityGroup
      DestinationSecurityGroupId: !Ref NodeSecurityGroup
      IpProtocol: tcp
      FromPort: 443
      ToPort: 443

  ClusterControlPlaneSecurityGroupIngress:
    Type: AWS::EC2::SecurityGroupIngress
    DependsOn: NodeSecurityGroup
    Properties:
      Description: Allow pods to communicate with the cluster API Server
      GroupId: !Ref ControlPlaneSecurityGroup
      SourceSecurityGroupId: !Ref NodeSecurityGroup
      IpProtocol: tcp
      ToPort: 443
      FromPort: 443

  NodeGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      DesiredCapacity: !Ref NodeAutoScalingGroupDesiredCapacity
      LaunchConfigurationName: !Ref NodeLaunchConfig
      MinSize: !Ref NodeAutoScalingGroupMinSize
      MaxSize: !Ref NodeAutoScalingGroupMaxSize
      VPCZoneIdentifier:
        - !Ref Subnet01
        - !Ref Subnet02
        - !Ref Subnet03
      Tags:
      - Key: Name
        Value: !Sub "${EnvironmentName}KubernetesCluster-Node"
        PropagateAtLaunch: 'true'
      - Key: !Sub 'kubernetes.io/cluster/${EnvironmentName}KubernetesCluster'
        Value: 'owned'
        PropagateAtLaunch: 'true'
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MaxBatchSize: '1'
        MinInstancesInService: !Ref NodeAutoScalingGroupDesiredCapacity
        PauseTime: 'PT5M'

  NodeLaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Properties:
      AssociatePublicIpAddress: 'true'
      IamInstanceProfile: !Ref NodeInstanceProfile
      ImageId: !Ref NodeImageId
      InstanceType: !Ref NodeInstanceType
      KeyName: !Ref KeyName
      SecurityGroups:
      - !Ref NodeSecurityGroup
      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            VolumeSize: 20
            VolumeType: gp2
            DeleteOnTermination: true
      UserData:
        Fn::Base64:
          !Sub |
            #!/bin/bash
            set -o xtrace
            /etc/eks/bootstrap.sh ${EnvironmentName}KubernetesCluster ${BootstrapArguments}
            /opt/aws/bin/cfn-signal --exit-code $? \
                     --stack  ${AWS::StackName} \
                     --resource NodeGroup  \
                     --region ${AWS::Region}

Outputs:

    KubernetesClusterName:
      Description: Cluster name
      Value: !Ref Cluster
      Export:
        Name: KubernetesClusterName

    KubernetesClusterEndpoint:
      Description: Cluster endpoint
      Value: !GetAtt Cluster.Endpoint
      Export:
        Name: KubernetesClusterEndpoint

    KubernetesNodeInstanceProfile:
      Description: The name of the IAM profile for K8
      Value: !GetAtt NodeInstanceProfile.Arn
      Export:
        Name: KubernetesNodeInstanceProfileArn

kubernetes

amazon-cloudformation

aws-security-group

amazon-eks

amazon-web-services

回答 2

Stack Overflow用户

回答已采纳

发布于 2020-04-29 13:29:02

将辅助节点添加到EKS集群有两种方法：

独立启动和注册员工(https://docs.aws.amazon.com/eks/latest/userguide/launch-workers.html)
使用托管节点组(https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html)

正如我从您的模板中看到的，您现在正在使用第一种方法。重要的是，在启动工作节点之前，您需要等待EKS集群就绪并处于活动状态。您可以通过使用DependsOn属性来实现这一点。如果这不能解决您的问题，请查看云init日志(/var/log/ cloud -init-output.log)，以检查加入集群时发生了什么。

如果您希望使用托管节点组，只需删除AutoScaling组和LaunchConfiguration并使用此类型：https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-eks-nodegroup.html的好处是，AWS负责在您的帐户中为您创建所需的资源(AutoScaling组和LaunchTemplate)，并且您可以在AWS控制台中看到Node。

票数 1

Stack Overflow用户

发布于 2021-03-31 11:06:57

我使用了托管节点组(https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html)选项。它起作用了。但是如何定义自动标度策略。它只允许给出最大节点数和最小节点数，甚至连名称都不允许。

票数 -1

页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持

原文链接：

https://stackoverflow.com/questions/61466360

复制

相似问题

问使用NodeGroup时，CloudFormation不会加入EKS集群
EN

回答 2

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问使用NodeGroup时，CloudFormation不会加入EKS集群EN

回答 2

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问使用NodeGroup时，CloudFormation不会加入EKS集群
EN