While working on our React Native app–Mender–we needed a simple way to upload and download images. Our backend is hosted on Azure using Node.js and Express, so Azure Blob Storage was a natural fit for our image store.

A lot of great articles exist explaining how to upload to Azure Blob Storage. Unfortunately, few do a good job explaining the details necessary for downloads. Even Azure's documentation leaves a lot to be desired. The majority of the articles provide steps to download blobs directly to the filesystem. While this works in some use cases, we needed something that could easily transport images without having to write it to disk or load the entire image in memory before sending it to our mobile app.

That is where streams come in...

Streams allow us to read and write data in chunks, passing them to another process or system as we receive or send them. This makes our application memory efficient (only handling a small portion of the data at a time) and speed efficient (since there is no disk I/O or waiting for the data to be fully collected in memory).

Passing the data to another process, or piping it, we are able to continue using the benefit of streams further down in our application process. In our case, Mender is able to read chunks of image data from our backend, which itself is reading data chunks from Azure.

For a more detailed introduction to streams, check out Liz Parody's article, Understanding Streams in Node.js.

Implementing the Download Process

Over the next few sections, we will take you through the components we use on our Express server and React Native application to download images.  The application process flow is:

• React Native app sends an HTTP get request to an endpoint on our backend server (requesting the specific image file)
• Server uses the request parameters to start the image file download from Azure
• Server pipes the download back to the HTTP response of the original get
• React Native app processes the image file

Let's start with the code we use on our Express server and then move to the client side, our React Native application.

Connecting to Azure

To interface with Azure, we use their @azure/storage-blob client. We created an object with properties that handle the interactions we want to perform: delete, download, upload. These properties store asynchronous functions that manage details needed by the storage-blob client. Our object is used on the Express server, so we are able to securely load our Azure connection string as an environment variable.

The download function takes in the container and filename for the specific image. It returns a Promise that resolves to a data object that contains a Readable stream: readableStreamBody.

const { BlobServiceClient } = require('@azure/storage-blob');

const azureBlob = {
    const AZURE_STORAGE_CONNECTION_STRING = process.env.AZURE_STORAGE_CONNECTION_STRING;
    delete: async (containerName, fileName) => {
    },
    download: async (containerName, fileName) => {
        // Create the BlobServiceClient object which will be used to create a container client
        const blobServiceClient = await BlobServiceClient.fromConnectionString(AZURE_STORAGE_CONNECTION_STRING);
        // Get a reference to a container
        const containerClient = await blobServiceClient.getContainerClient(containerName);
        // Get a block blob client
        const blockBlobClient = containerClient.getBlockBlobClient(fileName);
        return blockBlobClient.download(0);
    },
    upload: async (containerName, fileName) => {
    },
}

module.exports = {
    azureBlob
};

Triggering the Download Function

To trigger the download, we use Express Router's get method. Our React Native application sends a get request to our /api/avatar/:container/:imgUrl endpoint on our backend server. We use the route parameters container and imgUrl, of the request URL, to specify the blob container and filename. These string variables are fed in to our azureBlob.download(container, fileName) function we presented in the previous section.

const router = require('express').Router();
const { azureBlob } = require('../azure');

// Get route for avatars
router.get('/avatar/:container/:imgUrl', (req, res, next) => {
    // Start blob download from Azure
    azureBlob.download(req.params.container, req.params.imgUrl)
        // Pipe download stream to response
        .then(downloadBlockBlobResponse => 
            downloadBlockBlobResponse.readableStreamBody.pipe(res))
        .catch(error => {
            next(error);
        })
})

module.exports = router;

The Trick

This line of code is the how we are able to benefit from streams:

azureBlob.download(req.params.container, req.params.imgUrl)
        .then(downloadBlockBlobResponse =>
            downloadBlockBlobResponse.readableStreamBody.pipe(res))

When we receive the blockBlobClient.download(0) object from our azureBlob.download function, we are receiving a Promise that contains a readable stream of data. Using .pipe(destination), we are then able to send these incoming chunks of data from Azure to our React Native application using the HTTP response object, res.

– Node.js official documentation –
The readable.pipe() method attaches a Writable stream to the readable, causing it to switch automatically into flowing mode and push all of its data to the attached Writable. The flow of data will be automatically managed so that the destination Writable stream is not overwhelmed by a faster Readable stream.

Starting the Download from React Native

When the app needs to update a user image, for example when a user changes their avatar, it starts an asynchronous download. Using Expo for our React Native development, we are able to use their FileSystem component to handle the data stream coming from our Express server. (A side note on Expo: it is powerful development and build tool that vastly streamlines these processes across Android and iOS.) The application starts the download with two variables: the remote URI to download and the local URI of the file to download to.

1. remote URI example: "https://expressserver.company.com/api/img/avatar/avatars/photo-5ea10a1f3a7b30e38143ec00.jpeg"
2. local URI example: "file://avatars/photo-5ea10a1f3a7b30e38143ec00.jpeg"

During the image upload process, the application selects a random, unique filename and the respective container associated with the image as it appears on Azure Blob Storage. This URI string is stored in mongoDB as the imgUri. To download an image from Azure the application first queries mongoDB for the imgUri.

• imgUri example: "avatars/photo-5ea10a1f3a7b30e38143ec00.jpeg"

The following React Native function is used to start the download process. It takes two variables: the imgUri to download and the API endpoint on our Express server to use.

import Constants from 'expo-constants';
import * as FileSystem from 'expo-file-system';

// Express server URL
const SERVER_URL = Constants.manifest.extra.SERVER_URL;

// Image download function
const downloadImg = async (imgUri, imgEndpoint) => {
    return FileSystem.downloadAsync(
      SERVER_URL + imgEndpoint + imgUri,
      FileSystem.documentDirectory + imgUri
    )
    .then(({ uri }) => {
      console.log('Finished downloading to ', uri);
      return uri
    })
    .catch(error => console.log(error))
};

The FileSystem component handles the collection of the data chunks coming from our Express server stream, saving then to a specified location. By using the imgUri with the FileSystem.document directory, we are telling the downloadAsync function to use the container as the folder for the image. In the example above, the application would download to "file://avatars" folder with "photo-5ea10a1f3a7b30e38143ec00.jpeg" as the filename.

And that is it. We have used streams in Express to pipe data flowing from Azure Blog Storage to a local file on our mobile device. A lot more can be done with streams, so I highly recommend spending time learning more about them. If you have any questions or need some advice on your Node.js project, please feel free to reach out to us at info@appagetech.com.

Last year we partnered with a FinTech firm that wanted to implement a new natural language processing (NLP) model and transition to a new cloud provider. To accomplish this, we decided to use Azure Functions for cloud hosting and spaCy as the foundation for the NLP model. Our initial builds only ran into minor configuration issues. However, soon as our application began to grow, we hit some major issues, including: unstable builds, crashing CI/CD pipelines, failed deployments. With these issues hindering our progress, it was time to take the system apart to determine the cause of the problems. Everything was working perfectly—except spaCy. This was a major issue, however, as spaCy was key to this project: best results versus competing machine learning tools and calibrated to the client’s use case.

After many frustrating hours, and what proved to be numerous trials and, ultimately, errors, we discovered the best way to deploy a large spaCy model to Azure functions: manually use the model data directory as a part of the application’s repository.

TL;DR

• Download and extract: en_core_web_lg-2.2.5.tar.gz
• Copy model data directory to your app folder: __app__
• Import SpaCy into your function file: import spacy
• Load the model using: nlp = spacy.load('<filepath>/en_core_web_lg-2.2.5')

Background

Why did we choose spaCy? spaCy is a production ready, incredibly fast and accurate, NLP tool, especially when using their large general-purpose models. We were able to conduct accurate part-of-speech tagging and noun chunking out of the box. From there, we developed a propriety processing algorithm to consume spaCy’s Dependency Parser. This was tailored to the specific type of text documents that the client was analyzing. All in all, we ended up with a very clean and representative collection of data.

Why did we choose Azure Functions? Since spaCy is just one part of our Azure Function that handles data processing, we needed a system without cold start lag and had the ability to dynamically scale. The capabilities of Azure Functions and a dedicated App Service plan worked well with what we were trying to accomplish versus AWS Lambda. A dedicated App Service plan allowed us to run the Function application like a traditional web app. The plan uses dedicated scalable memory and compute. So, we were able to keep the application in memory, including the large spaCy model, allowing hot starts. By hosting on Azure Functions, our app is cached in memory, has unbounded timeout duration, flexible memory usage (up to 14 GB), and flexible compute.

Different Methods We Tried

Using the recommended download method

We tried to incorporate the model download as one of the build steps. The benefit of this would have been a smaller repo and access to the most recent model on each update. However, due to the size of the download, our builds began to randomly fail. At the same, our build time was exploding, making it impossible to iterate quickly. Builds were averaging 20+ minutes.

$ python -m spacy download en_core_web_sm

>>> import spacy
>>> nlp = spacy.load("en_core_web_sm")

Installation via pip

Similar to the previous method, this would give us a smaller repo, but we would have lost the ability to dynamically use the most recent model. Nonetheless, we needed a stable build and deployment pipeline. So, we incorporated the model's external URL as a part of our requirements.txt . Our team now had luck on the build step; however, the deployment started to randomly fail. Builds saw a minor improvement averaging less than 10 minutes, yet the deployments started taking 15 to 20 minutes, if they worked.

# With external URL
pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-2.2.5/en_core_web_lg-2.2.5.tar.gz

Our Final Implementation! Wooooo, it is working

Our last ditch effort was to manually download and extract the language model as a part of our application's repo. This meant drastically increasing the repo size. It also meant that we would have to manually replace the model when a new one became available. With nothing else working, we considered our options and decided these downsides were manageable for us.

We used the most recent large English model available at the time en_core_web_lg-2.2.5. After downloading and extracting the archive, we saved the model directory in our app folder.

To use the model in our function, we loaded the model directly. For us this was in the parent director of our current utility function. Below is our code.

import spacy

def get_spacy_path():     
    current_path = pathlib.Path(__file__).parent.parent    
    return str(current_path / 'en_core_web_lg-2.2.5')
print(get_spacy_path())
nlp = spacy.load(get_spacy_path())

Infrastructure

• App Service: B2 (3.5 GB memory; 1:1 vCPU:Core)
• Git repo: Azure Repos
• Azure Function: Python Function App version 2.0
• Python: version 3.7
• CI/CD: Azure DevOps Pipelines (YAML deployment)

Closing Thoughts

Our approach was not ideal. The expectations for how these technologies could/should be combined were not close to reality. We could have saved ourselves a lot of pain if we tested various deployment methods upfront. Our first priority was testing and building the application—DevOps was secondary. As we approach new projects, our testing framework will include production deployment, treating it as equally as important as the application code itself from day one.

Final Deployment YAML

# Production ----- Python Function App to Linux on Azure
# Build a Python function app and deploy it to Azure as a Linux function app.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/python

trigger:
- master

variables:
  # Azure Resource Manager connection created during pipeline creation
  azureSubscription: <subscription>

  # Function app name
  functionAppName: <appName>

  # Agent VM image name
  vmImageName: 'ubuntu-latest'

  # Working Directory
  workingDirectory: '$(System.DefaultWorkingDirectory)/__app__'

stages:
- stage: Build
  displayName: Build stage

  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)

    steps:
    - bash: |
        if [ -f extensions.csproj ]
        then
            dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
        fi
      workingDirectory: $(workingDirectory)
      displayName: 'Build extensions'

    - task: UsePythonVersion@0
      displayName: 'Use Python 3.7'
      inputs:
        versionSpec: 3.7

    - bash: |
        pip install --upgrade pip
        pip install -t .python_packages/lib/site-packages -r requirements.txt
      workingDirectory: $(workingDirectory)
      displayName: 'Install application dependencies'

    - task: ArchiveFiles@2
      displayName: 'Archive files'
      inputs:
        rootFolderOrFile: '$(workingDirectory)'
        includeRootFolder: false
        archiveType: zip
        archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
        replaceExistingArchive: true

    - publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
      artifact: drop

- stage: Deploy
  displayName: Deploy stage
  dependsOn: Build
  condition: succeeded()

  jobs:
  - deployment: Deploy
    displayName: Deploy
    environment: 'production'
    pool:
      vmImage: $(vmImageName)

    strategy:
      runOnce:
        deploy:

          steps:
          - task: AzureFunctionApp@1
            displayName: 'Azure functions app deploy'
            inputs:
              azureSubscription: '$(azureSubscription)'
              appType: functionAppLinux
              appName: $(functionAppName)
              package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'