On-Premises Broker

Overview

To analyze your on-prem code and container repositories, Heeler requires deploying its on-prem Broker. Deploy the Broker using the Heeler-provided container image and configure it to enable access to your on-prem repositories and connect to the Heeler platform.

Aside from deploying the broker, set up is a two-step process:

1. Obtain the unique Heeler application credentials for your broker

2. Configure the Broker to connect to the Heeler platform

Heeler Application Credentials

To obtain the necessary Heeler application credentials, navigate to Settings and Brokers. Once there, click Add Broker and you will see a modal like below:

At this time, the Heeler Broker supports on-prem analysis of GitLab code repositories and JFrog Artifactory container images.

• Assign the Broker a name

• Add the repositories under Connections

• Click Save

Once saved, the modal will display the Heeler application credentials your Broker will use to communicate with Heeler. The credentials are labeled Key and ID as in the modal below with ID mapping to BROKER_KEY_ID and Key mapping to BROKER_SECRET_KEY:

Save the credentials securely for future use. If lost, you'll have to Generate New credentials and update your Broker configuration accordingly.

Click Done to view the Broker Models listing with your newly added broker. Note that its status will remain Healthy only after connecting with Heeler and starting processing. Once connected and active, broker metrics can be accessed via a dropdown.

Deploy and Configure Broker

Broker Image Location

Heeler provides an image for you to deploy a Broker on-premise. The image is located at:

// Heeler Broker image location
654654247928.dkr.ecr.us-east-1.amazonaws.com/broker:latest

Broker Requirements

You can deploy the image in multiple ways. The Broker has been verified on AWS using ECS and under an ASG. The key requirements are:

1. The Broker must have a routable URL available to your Gitlab repository. Gitlab will use the URL to send commit notifications to the Broker via webhooks.

2. The Broker should be able to scale in response to workload. When scaled up, the Broker instances run independently. They pull jobs from Heeler, perform their jobs in isolation, and push their work product when complete. Typical jobs include harvesting data, analyzing a commit, or backfilling data.

3. Each Broker instance should have a minimum of 4Gb of memory.

Broker Configuration

The remaining instructions assume you are using ECS to deploy your broker.

1. Store Broker secrets, i.e., BROKER_KEY_ID and BROKER_SECRET_KEY

AWS allows for ECS env variables to come from AWS Secrets without exposing sensitive values in plain text. We highly recommend storing the BROKER_SECRET_KEY and BROKER_KEY_ID using AWS Secrets and using the ARN of each secret as an environment variable.

If you use AWS Secrets to store these values, then update the ECS task definition with the following:

"secrets": [

    {
        "name": "BROKER_KEY_ID",
        "valueFrom": "arn:aws:secretsmanager:us-east-1:56565656565:broker_key_id_123"
    },
    {
        "name": "BROKER_SECRET_KEY",
        "valueFrom": "arn:aws:secretsmanager:us-east-1:56565656565:broker_secret_key_123"
    }
],

If you do not use AWS Secrets to store these values, then you will need to provide them as environment variables.

1. Define Environment Variables

The Broker instances rely upon some environment variables as part of their configuration. These variables can be defined as follows:

"environment": [
    {
        "name": "BROKER_HEELER_URL",
        "value": "https://app.heeler.com/api/internal/broker"
    },
    {
        "name": "SCA_COMMAND",
        "value": "heeler-sca"
    },
    {
        "name": "BROKER_KEY_ID",
        "value": {NOT RECOMMENDED see aws secrets}
    },
    {
        "name": "BROKER_SECRET_KEY",
        "value": {NOT RECOMMENDED see aws secrets}
    }
]

1. Command argument

The Broker instance must be activated via a the command argument:broker. The broker binary is under a specific folder /app/broker. To configure correctly, use:

"command": [
    "/app/broker",
    "broker"
],

Broker Confirmation

Once the Broker is operational, return to the Broker Models page in Heeler, where you should see your new Broker with updated status.