Calling Azure Service Bus from Azure API Management Service

Azure API Management Service (APIM) can seamlessly integrate with Azure Service Bus, allowing you to send messages to both queues and topics, including their respective subscriptions. To achieve this integration, a few configurations and authentication steps are necessary, ensuring a secure and efficient communication channel between your API Gateway and Service Bus resources.

Authentication

For secure message transmission between APIM and Service Bus, proper authentication is crucial. Here’s how you can establish the connection:

  1. Activate Managed Identity: Start by activating a system-assigned Managed Identity from the API Management Service. This identity will serve as the authentication mechanism.

  1. Assign Roles in Service Bus

    2a. In the Azure Service Bus resource, navigate to the Access Control menu, then select Role Assignments and click Add.

    2b. From the Role tab, assign roles such as Azure Service Bus Data Receiver and/or Azure Service Bus Data Sender.

    2c. In the Assign access to group, specify the Managed identity from the available options and select the relevant API Management service.

Api Management Configuration

To enable APIM to send messages to Service Bus, you must define a new operation and configure the inbound rules appropriately. The following inbound policy should be set for the operation:

<policies>
    <inbound>
        <base />
            <authentication-managed-identity resource="https://servicebus.azure.net" output-token-variable-name="msi-access-token" ignore-error="false" />
            <set-header name="Authorization" exists-action="override">
                    <value>@((string)context.Variables["msi-access-token"])</value>
            </set-header>
            <set-header name="BrokerProperties" exists-action="override">
                <value>@("{ \"CorrelationId\": \"TestCorelationId\" }")</value>
            </set-header>
            <set-backend-service base-url="https://YOUR-SERVICE-BUS-URL.servicebus.windows.net" />
            <rewrite-uri template="YOUR-TOPIC-OR-QUEUE/messages" />
    </inbound>
...
</policies>

Explanation of Inbound Policy:

This inbound policy performs authentication using managed identity, sets custom headers (“Authorization” and “BrokerProperties”), specifies the backend service URL, and rewrites the request URI before forwarding the request to the specified Service Bus endpoint.

  • Authentication (authentication-managed-identity): Requests a managed identity (MSI) against the specified Azure Service Bus resource, storing the resulting access token in the variable msi-access-token.

  • Setting Headers (set-header): Custom headers like Authorization and BrokerProperties are set for the request. The Authorization header contains the access token, and the BrokerProperties header includes additional message properties like correlation IDs.
    The full list properties that can be set looks like this:

BrokerProperties:  { "SessionId": "{27729E1-B37B-4D29-AA0A-E367906C206E}", "MessageId": "{701332E1-B37B-4D29-AA0A-E367906C206E}", "TimeToLive" : 90, "CorrelationId": "{701332F3-B37B-4D29-AA0A-E367906C206E}", "SequenceNumber" : 12345, "DeliveryCount" : 2, "To" : "http://contoso.com", "ReplyTo" : "http://fabrikam.com",  "EnqueuedTimeUtc" : " Sun, 06 Nov 1994 08:49:37 GMT", "ScheduledEnqueueTimeUtc" : " Sun, 06 Nov 1994 08:49:37 GMT"}
  • Backend Service Configuration (set-backend-service): Specifies the base URL of the Service Bus endpoint where requests will be forwarded.

  • URI Rewriting (rewrite-uri): Directs the request to a specific topic or queue by rewriting the URI. The template should be the topic/queue name followed by “/messages”.

Sources

Remove subtitles automatically from video files

Shell scripts still rule!

I have my personal PLEX setup at home running from a Raspberry Pi. It is easy to manage and is useful in many occasions.

My Samsung TV has issues when loading a video file that has over 30 tracks (video/audio/subtitles). This was not a general problem in the past, but I am glad that the PLEX documentation mentions it. (It was hard to find)

Removing subtitles

Trying to fix the issue I came across mkvtoolnix tool that helps re-muxing the video files and remove all the tracks that I do not need.

The command to do so is quite simple. First one selects 2 & 3 subtitle tracks (Subtitle Id can differ). The second command just removes all subtitles from the video file.

mkvmerge -o output.mkv -s 2,3 input.mkv
or
mkvmerge -o output.mkv --no-subtitles input.mkv

I quickly created a small shell script that iterates through my video files and creates new versions for me.

for file in *mkv; do
 sudo mkvmerge -o "${file%.mkv}".PLEX.mkv -s 2,3 "$file"
done

Super! My issue is fixed, but I don’t want to manually log on to my Pi every time and run the command. This lead me to automate this task even further.

Automating subtitles removal

Then I found inotifywait from inotify-tools. This monitors directory tree for changes and can call an action on events like file created, modified, etc.

This prompted me to write a script combining the two tools and also adding some logic to the script. Ended up with the following script:

This script activates on new files. Filters out files created after the subtitles have been removed and also files that do not meet a certain criteria. I also automated the removal of these files. If the original file is not present anymore then this script also removes the altered version.

Making it a service

The next step is to create a service out of this script that is started at system boot.

For this a file must be created at /etc/systemd/system/sub-remover.service

The contents should be:

[Unit]
Description=Subtitle Remover Service
After=network.target

[Service]
ExecStart=/mnt/subremover/subremover.sh /mnt
Restart=always
User=pi

[Install]
WantedBy=multi-user.target

The important part is the ExecStart this should point to the shell script file and should also have the folder it will be looking at. In this case the subremover.sh path with /mnt as the folder it should look at.

To enable the service run:

sudo systemctl daemon-reload
sudo systemctl enable sub-remover.service

For managing the service you can use:

sudo systemctl start sub-remover.service
sudo systemctl stop sub-remover.service
sudo systemctl status sub-remover.service

For checking the logs of the service:

sudo journalctl -u sub-remover.service -f 

Use -f to follow the logs live.