dshimizu/blog/alpha

とりとめのないITブログ

EC2 Spot Instance, Fargate Spot が停止したりECSタスクが異常終了した時に Slack に通知する仕組みを CloudFormation で作ってみた

AWS AWS CloudFormation Amazon ECS Amazon EC2

はじめに

ECS on EC2 の Spot Instance や Fargate Spot を結構使っているのですが、停止した時に気づくための仕組みが Lambda だと管理が面倒です。 AWS Chatbot が EventBridge をサポートしているので、EventBridge でイベントを拾って SNS を経由して Slack に通知させるための仕組みを CFn で作ってみました。

目次

サンプル CFn テンプレート

早速ですがサンプルの CFn テンプレートです。 ここでは面倒なので各イベントを全て1つのテンプレートにまとめてます。

Slack と AWS Chatbot の連携設定については割愛します。以下あたりのドキュメントに記載があります。

以下のテンプレートで、AWS Chatbotの定義のSlackのワークスペースとチャンネルIDを環境に合わせて設定し、スタックを作成します。 ECS クラスターは環境に応じて書き換えます。

AWSTemplateFormatVersion: "2010-09-09"
Description: SNS Topic and EventBridge for SpotInstance Intteruption Notify

Resources:
  # ----------------------------------------------------------------
  # Define of IAM
  # ----------------------------------------------------------------
  SpotInstanceInterruptSlackNotifyIAMRole:
    Type: AWS::IAM::Role
    Properties:
      #RoleName: !Sub ${AWS::StackName}-iam-role
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: chatbot.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess

  # ----------------------------------------------------------------
  # Define of EventBridge For Spot Instance Stopped
  # ----------------------------------------------------------------
  SpotInstanceInterruptEventRule:
    Type: AWS::Events::Rule
    DependsOn:
      - SpotInstanceInterruptSlackNotifySNSTopic
      - SpotInstanceInterruptSlackNotifySNSTopicPolicy
    Properties:
      Description: "EventRule"
      EventPattern:
        source:
          - "aws.ec2"
        detail-type:
          - "EC2 Spot Instance Interruption Warning"
      State: "ENABLED"
      Targets:
        -
          Arn:
            Ref: "SpotInstanceInterruptSlackNotifySNSTopic"
          Id: "SpotInstanceInterruptSlackNotifySNSTopic"

  # ----------------------------------------------------------------
  # Define of EventBridge For Fargate Spot Stopped
  # ----------------------------------------------------------------
  FargateSpotStoppedEventRule:
    Type: AWS::Events::Rule
    DependsOn:
      - SpotInstanceInterruptSlackNotifySNSTopic
      - SpotInstanceInterruptSlackNotifySNSTopicPolicy
    Properties:
      Description: "EventRule"
      EventPattern:
        source:
          - "aws.ecs"
        detail-type:
          - "ECS Task State Change"
        detail:
          stopCode:
            - "SpotInterruption"
        clusterArn:
          - "arn:aws:ecs:exampleregion:1111222233334444:cluster/examplecluster"
      State: "ENABLED"
      Targets:
        -
          Arn:
            Ref: "SpotInstanceInterruptSlackNotifySNSTopic"
          Id: "SpotInstanceInterruptSlackNotifySNSTopic"

  # ----------------------------------------------------------------
  # Define of EventBridge For ECS Task Failed
  # ----------------------------------------------------------------
  ECSTaskStoppedEventRule:
    Type: AWS::Events::Rule
    DependsOn:
      - SpotInstanceInterruptSlackNotifySNSTopic
      - SpotInstanceInterruptSlackNotifySNSTopicPolicy
    Properties:
      Description: "EventRule"
      EventPattern:
        source:
          - "aws.ecs"
        detail-type:
          - "ECS Task State Change"
        detail:
          lastStatus: 
            - "STOPPED"
          stoppedReason:
            - anything-but:
                "prefix": "Scaling activity initiated by"
          clusterArn:
            - "arn:aws:ecs:exampleregion:1111222233334444:cluster/examplecluster"
          containers:
            exitCode:
              - "anything-but": 0
      State: "ENABLED"
      Targets:
        -
          Arn:
            Ref: "SpotInstanceInterruptSlackNotifySNSTopic"
          Id: "SpotInstanceInterruptSlackNotifySNSTopic"

  # ----------------------------------------------------------------
  # Define of SNS
  # ----------------------------------------------------------------
  SpotInstanceInterruptSlackNotifySNSTopic:
    Type: AWS::SNS::Topic
    Properties:
      TopicName: !Sub ${AWS::StackName}-sns-topic

  SpotInstanceInterruptSlackNotifySNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: events.amazonaws.com
            Action: sns:Publish
            Resource: '*'
      Topics:
        - !Ref SpotInstanceInterruptSlackNotifySNSTopic

  # ----------------------------------------------------------------
  # Define of Chatbot Slack Channel
  # ----------------------------------------------------------------
  SpotInstanceInterruptSlackNotifySlackChannelConfiguration:
    Type: AWS::Chatbot::SlackChannelConfiguration
    DependsOn:
      - SpotInstanceInterruptSlackNotifySNSTopic
      - SpotInstanceInterruptSlackNotifySNSTopicPolicy
    Properties:
      ConfigurationName: !Sub ${AWS::StackName}-chatbot
      IamRoleArn: !GetAtt SpotInstanceInterruptSlackNotifyIAMRole.Arn
      LoggingLevel: ERROR
      SlackChannelId: ********
      SlackWorkspaceId: ********
      SnsTopicArns:
        - !Ref SpotInstanceInterruptSlackNotifySNSTopic

まとめ

AWS Chatbot が EventBridge をサポートしてくれたおかげで、余計なLambda等を作ることなくいろんな通知を楽に Slack に集約できて良いです。

参考