AWS EventBridge for scheduling

Amazon’s EventBridge is one of the three main messaging infrastructure components on the AWS platform along with the SimpleQueueService (SQS) and the SimpleNotificationService (SNS). EventBridge’s architecture is essentially that of an event bus – it is a serverless event bus that operates at massive scale and can be used to architect applications work on the concept of publishing and subscriptions. Messages published to EventBridge can be subscribed to and EventBridge is able to deliver them to a variety of targets in real time. A key benefit of this kind of architecture is the disconnected nature of the subscribers – which makes it very easy to add or replace component services with minimal disruption to the overall platform.

Eventbridge’s features and integrations go way beyond simple publish and subscribe and it provides direct integrations to a variety of external sources and can also be configured to do cross AWS account message delivery. However for the scope of this article – we will focus on working with the core feature of EventBridge to build our own custom serverless event driven application. In doing so we will demonstrate how to both setup an EventBridge event bus and work with the EventBridge event bus using Lambdas and Python with the boto3 SDK and also demonstrate how to define the serverless infrastructure for our application using AWS Cloudformation.

The example application is going to be an event driven notification system to send emails, texts, push notifications with different handlers picking up the appropriate event – this example should be able to serve as a baseline reference implementation that can be used for your own specific functional requirements that can benefit from EventBridge. The final integrated serverless notification system will be as shown in the architecture diagram that is the feature image above – we will describe the workflow in more detail in the next section.

Outlining the Notification System

The API Gateway is the entry endpoint to the system where notification requests can be sent from upstream systems that may be client side UIs or other systems. While this example will use an Http API Gateway for simplicity, there are several options available including the excellent AppSync service which I use in this article . And if your component is not going to be invoked by any direct client side input – perhaps on the basis of other kind of data triggers – then the API endpoint can be skipped entirely and the Lambda that is the target of the endpoint can be invoked directly.

The Event Publisher Lambda will publish specific event types – texts, emails or push event types based on the incoming request to the EventBridge event bus. The publisher is agnostic to the subscribers and will simply publish a payload with a specific event type – the subscribers will register with EventBridge to request specific events. In this case, we will have a Lambda handler for each event type.

Emails will be sent out using AWS SES – SimpleEmailService, texts will be sent out using SNS – SimpleNotificationService and mobile push notifications via Google Firebase.

The entire architecture will be defined as infrastructure-as-code using a Cloudformation template – where we will also be defining the actual EventBridge event bus. It is worth noting here that Cloudformation makes the wiring of Lambda targets from EventBridge exceedingly trivial compared to manually configuring stuff in the AWS console which I am not a fan of doing. By simply defining EventBridge Event types to the Cloudformation Function Resources, all the magic gets done for you when you deploy the stack. We will be demonstrating this in yaml the following sections.

We will define and code out all of this piece by piece in the rest of the sections below

Defining the EventBridge Event Bus

Just before we start defining our Cloudformation stack – a note to point out a couple of articles I have written than might be of use to you in case you are not familiar with all this. If you are – please ignore these two links below and review the yaml code directly.

To see how to setup AWS SAM to work with Cloudformation – please see this article.

For a detailed write up of working with Lambdas – specifically Python based Lambdas – and Cloudformation – take a look at this post.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Serverless EventBridge Notification System Example

Globals:
  Function:
    Architectures:
      - arm64
    Runtime: python3.9
    Timeout: 300

Resources:

  EventBridgeEventBus:
    Type: AWS::Events::EventBus
    Properties:
      Name: "notification-eventbus"

A resource of type AWS::Events::EventBus with a custom name and we have our EventBridge Event Bus all set and ready to go. Well not till we deploy it, but we can now use this reference in the template when we define subscription filters in the next few sections. The globals section defines the common properties for our Lambdas – since all our Lambdas are going to be Python and we want them to use arm64 – it makes sense to define this globally instead of individually.

Defining the API Gateway backed up by the Event Publisher Lambda

First lets define the Lambda yaml before we code out the Lambda in Python where will be using the boto3 SDK to publish messages to the event bus we defined in the previous section.

EventPublisherLambda:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ./event-publisher/
      Handler: event-publisher.event_handler
      Description: Publish notifications to the event bus with their type
      FunctionName: event-publisher
      Policies:
        - AmazonEventBridgeFullAccess
      Environment:
        Variables:
          EVENTBUS_NAME: !Ref EventBridgeEventBus
      Events:
        HttpAPIEvent:
          Type: HttpApi
          Properties:
            Path: /notifications
            Method: post
            PayloadFormatVersion: "2.0"

If you are using your own names and definitions, ensure the codeuri and handler properties match the directory and Python file structure. The Lambda is fronted by an Http API Gateway endpoint which is defined in the events section. Cloudformation will generate a random url and define a route to “/notifications” which will send traffic to our Lambda. We are passing the name of the event bus to our Lambda as an environment variable since we will need to publish to it.

Lastly we are providing an IAM policy based access to EventBridge for this Lambda. It is required otherwise the publish actions will fail – however it is possible to tailor access more restrictively to the Lambda by using more fine grained policy definitions, but it isn’t really needed as long as your Lambda limits the actions it does on the bus and it is fine to use the provided AWS SAM utility policy.

Coding the Event Publisher Lambda in Python

We are about to code one of the key components of the notification system, so lets review how the contract is going to behave.

The goal is for the Lambda to publish event payloads based on incoming requests. Keeping it simple we just need three inputs from the upstream systems – recipient – email address, phone number or push topic, message – simple text, and destination – email, text or push as the valid values.

The event publisher Lambda is agnostic to all the implementation details and is simply going to publish it to the event bus for a handler to pick up and handle as needed. In order to allow subscribers to do so – we will make use of the EventBridge payload structure that looks like this:

eventbridge_event = {
            'Time': start_time, #time of event
            'Source': "event-publisher",
            'Detail': json.dumps(event), 
            'DetailType': destination, #email, text, push
            'EventBusName': eventbus
        }

Specifically subscribers can use the the Source and DetailType properties of EventBridge to set their specific subscription filters. In our case – we will use the contract and simply pass the destination from the API Gateway input.

EventBridge allows for fairly complex subscription definitions – but my personal advice – keep things simple in your design. I mean that is always good advice isn’t it, and DetailType should be sufficient for filtering out the vast majority of use cases except in very complex payloads – where simplicity is already lost.

So to summarize – the Lambda will accept recipient, message and destination as inputs from the API Gateway – create a payload event with these values – set the fields above in the eventbridge event – and publish it.

from datetime import datetime
import boto3
import json
import os

eventbus = os.environ["EVENTBUS_NAME"]
eventbridge = boto3.client("events")


def event_handler(event, context):

    # Expecting destination, recipient, message in event
    event = json.loads(event["body"])
    destination = event["destination"]

    start_time = datetime.now()

    eventbridge_event = {
        'Time': start_time,
        'Source': "event-publisher",
        'Detail': json.dumps(event),
        'DetailType': destination,
        'EventBusName': eventbus
    }

    response = eventbridge.put_events(
        Entries=[
            eventbridge_event
        ]
    )
    
    print(response)

As you can see from the contract, EventBridge supports publishing multiple events, in a single call – but our design will simply publish one event at a time.

Subscribing the Text Handler Lambda to the Event Bus

Just like the above section – lets first define the Lambda in the cloudformation yaml before coding it in Python to demonstrate how to handle the EventBridge invocation.

As mentioned earlier – requesting a subscription filter from EventBridge for DetailType “text” from Source “event-publisher” – as defined in the previous section is exceedingly simple with cloudformation – all we need to do is define an EventBridgeRule Event type and provide the subscription pattern as demonstrated below.

TextEventHandlerLambda:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ./text-handler/
      Handler: text-handler.event_handler
      Description: Subscribe and send texts via SNS
      FunctionName: text-handler
      Policies:
        - AmazonSNSFullAccess
      Events:
        EventBridgeTextEvents:
          Type: EventBridgeRule
          DependsOn: EventBridgEventBus
          Properties:
            EventBusName: !Ref EventBridgEventBus
            Pattern:
              source:
                - event-publisher
              detail-type:
                - text

Coding the Text Handler to send texts via SNS

Take a look at this post for more details about the setting up and send SMS Texts via SNS.

Below I am simply going to demonstrate the code without the specifics of verifying numbers and such which I cover in that article I linked above. When EventBridge invokes our text-handler Lambda for “text”, DetailType, we will have access to the message we published from the publisher in the event payload.

eventbridge_event = {
        'Time': start_time,
        'Source': "event-publisher",
        'Detail': json.dumps(event),
        'DetailType': destination,
        'EventBusName': eventbus
    }

So it can simply be extracted very easily from the “Detail” object as shown below and a text message sent to the recipient using SNS.

import boto3

sns = boto3.resource('sns')


def event_handler(event, context):
	
    text_event = event["detail"]
    recipient = text_event("recipient")
    message = text_event("message")
    
	response = sns.publish(
            PhoneNumber=recipient,
            Message=message
        )

    message_id = response["MessageId"]
    print(message_id)

Subscribing the Email & Push Handlers

As you probably surmised – the yaml definition for the email and push subscription Lambdas are going to be almost exactly the same as the yaml for the text-handler – only the EventBridgeRule filter varies and of course the function names and code locations.

EmailEventHandlerLambda:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ./email-handler/
      Handler: email-handler.event_handler
      Description: Subscribe and send emails via SES
      FunctionName: email-handler
      Policies:
        - Statement:
            - Effect: Allow
              Action:
                - "ses:SendEmail"
              Resource: "*"
      Events:
        EventBridgeEmailEvents:
          Type: EventBridgeRule
          DependsOn: EventBridgEventBus
          Properties:
            EventBusName: !Ref EventBridgEventBus
            Pattern:
              source:
                - event-publisher
              detail-type:
                - email

PushEventHandlerLambda:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ./push-handler/
      Handler: push-handler.event_handler
      Description: Subscribe and send push notifications via Firebase
      FunctionName: push-handler
      Events:
        EventBridgePushEvents:
          Type: EventBridgeRule
          DependsOn: EventBridgEventBus
          Properties:
            EventBusName: !Ref EventBridgEventBus
            Pattern:
              source:
                - event-publisher
              detail-type:
                - push

Coding the Email Handler Lambda to send emails via SES

Once again, for more details and caveats of working with SES and verified domains and config sets and all – please look at this article here.

Just focusing on the Python code – see below – we are extracting the recipient details from the EventBridge payload as we did for the text handler Lambda

import os
import boto3
import json
from datetime import datetime

from_email = "email@mydomain.com"
client = boto3.client('ses')


def event_handler(event, context):
    
    email_event = event["detail"]
    recipient = email_event("recipient")
    message = email_event("message")

    body_html = f"""<html>
        <head></head>
        <body>
          <p>{{message}}</p> 
        </body>
        </html>
                    """
    
    body_html = body_html.replace("{{message}}", message)

    email_message = {
        'Body': {
            'Html': {
                'Charset': 'utf-8',
                'Data': body_html,
            },
        },
        'Subject': {
            'Charset': 'utf-8',
            'Data': message,
        },
    }

    client.send_email(
        Destination={
            'ToAddresses': [recipient],
        },
        Message=email_message,
        Source=from_email,
    )

I am skipping the Python code demonstration of push notifications – this is a less common use case and requires a lot of prerequisites outside of AWS like having your own App and a Firebase account. If you want to see how to send push notifications using the Firebase Python SDK – I am going to ask you to ask you to please refer to this article and repeat what was demonstrated for the email and text Lambdas – same as the case for the other two Lambdas, extract the details and modify the code demonstrated in the linked article to handle this specific case.

Concluding

I want to reiterate some of the key benefits of using EventBridge – as an architecture design – the decoupled nature of the components make for seamless extensibility or removal of components – change the publisher implementation and simply ensure subscribers continue to get their DetailType from the new component. Add or remove subscribers as you need to seamlessly, the publishers are agnostic to this. For example, should you need to send another type of text message handler using a vendor separate from SNS – then simply implement a new handler and add a new type for this. Outside of the design pattern benefits, EventBridge has other feature benefits like cross account event delivery and message replays and the like that might be of use to your specific use case. The EventBridge Schema Registry is another cool feature that might be of use if you work in a large organization and would like to discover schemas that EventBridge is aware of that you could subscribe to seamlessly.

So hopefully this article and project helped a little with implementing your use case. If you are extending from this baseline design, some other obvious next steps to consider are incorporating a database like Dynamodb for record keeping of your message flow and also spending time designing a possibly more complex event driven flow than this relatively liner design here. This could also be where you might need to make more use of the EventBridge rule pattern matching features beyond the example used in this article. In any case, good luck with whatever you plan to work on with EventBridge.

Search This Blog

shabab tech