Introduction

In today’s digital landscape, building scalable web applications is crucial for businesses and developers. Flask, a popular Python web framework, offers a lightweight and flexible solution for creating web applications. When combined with HarperDB, a powerful NoSQL database, and deployed on the Google Cloud Platform (GCP) using GitHub Actions, developers can harness the full potential of these technologies to create robust and scalable applications.

In this step-by-step guide, we will walk you through the process of creating a Flask app with HarperDB as the backend database and deploying it on the Google Cloud Platform. You will learn how to set up a HarperDB instance, create a CI/CD pipeline with GitHub Actions, and deploy the Flask app on Google Cloud Run. We’ll also address security considerations and troubleshooting common issues.

Prerequisites

The prerequisites for this project include:

1. Flask and Python: Familiarity with Flask, a Python web framework, and Python programming language is essential.

2. Google Cloud Platform (GCP) Account: You will need a GCP account to create a project, access the necessary services, and deploy your Flask app on GCP.

3. HarperDB: A basic understanding of HarperDB and its concepts is required as this project integrates HarperDB as the backend database for the Flask app.

4. Git and GitHub: Familiarity with the Git version control system and GitHub platform is necessary to clone the project, manage code changes, and set up the CI/CD pipeline using GitHub Actions.

5. Docker: Knowledge of Docker and containerization concepts is required to containerize your Flask app using Docker and create Docker images for deployment.

6. GitHub Actions: To configure the CI/CD pipeline for the deployment of the Flask app, one must be familiar with GitHub Actions, a workflow automation tool.

It is recommended to have a good understanding of these prerequisites before starting the project to ensure a smooth development and deployment process. If you are not familiar with any of these technologies, it is advisable to refer to the respective documentation and tutorials to gain the necessary knowledge.

Let’s jump in

Setting up a HarperDB Instance

The first step is to create a HarperDB account. The sign-up process is straightforward: visit the HarperDB sign-up page and fill in the details.

Once you’re inside the account, you can create a new DB instance.

When creating a DB instance, you can choose between two options.

1. Creating a HarperDB cloud instance that is managed and scaled by HarperDB.

2. Installing HarperDB on your own server.

For this tutorial, we’ll stick with the cloud instance. So, we provide an instance name, username, and password and hit create. It might take a few minutes to launch the instance.

HarperDB’s free tier offers 1 GB of free storage and 0.5 GB of RAM.

Creating a Schema and Database

Once the DB instance is ready, we can start creating tables for the application. For our movie journal app, we’ll need tables for movies and reviews.

In HarperDB, you can create a schema and a table. HarperDB also requires you to assign a field for a hash attribute, which is similar to a primary key in a relational database. This attribute is used as a unique identifier for each record. For the movie table, a good hash attribute would be “movieId”.

The user interface is pretty straightforward, so steps for explanation aren’t needed here.

Creating a Python Project

Now that we have the database set up, let’s start building the Python application. We’ll use Flask, a micro web framework for Python, to create the application.

To interact with the HarperDB instance, we’ll use the HarperDB Python SDK. This SDK simplifies CRUD operations, making it as easy as calling a function.

Creating and Exposing the REST APIs

The movie journal app will have several functionalities, each of which will be exposed as a REST API:

1. Adding a new movie: We’ll create an API that allows users to add a new movie to their journal. The API will accept a POST request with the movie details (title, director, release year, etc.) and add the movie to the “movies” table in the HarperDB instance.

2. Updating movie details: Users should be able to update the details of a movie they’ve added. We’ll create an API that accepts a PUT request with the updated movie details and the movieId of the movie to be updated.

3. Deleting a movie: If a user wants to remove a movie from their journal, they can do so using the delete API. This API will accept a DELETE request with the movieId of the movie to be deleted.

4. Adding a review for a movie: Users can add their reviews for a movie. We’ll create an API that accepts a POST request with the review text and the movieId of the movie being reviewed.

Code:

Remember to install the necessary dependencies by creating a requirements.txt file with the following content:

flask
harperdb
python-dotenv

And install them using pip:

pip install -r requirements.txt

Application:

from flask import Flask, request, jsonify, render_template
import harperdb

app = Flask(__name__, static_folder='static', template_folder='templates')

db = harperdb.HarperDB(
    url='YOUR_HARPERDB_URL',
    username='YOUR_HARPERDB_USERNAME',
    password='YOUR_HARPERDB_PASSWORD'
)

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/movie', methods=['POST'])
def add_movie():
    movie_data = request.get_json()
    db.insert(schema='movie_journal', table='movies', records=[movie_data])
    return jsonify(movie_data), 201

@app.route('/movie/<movie_id>', methods=['PUT'])
def update_movie(movie_id):
    movie_data = request.get_json()
    movie_data['id'] = movie_id
    db.update(schema='movie_journal', table='movies', records=[movie_data])
    return jsonify(movie_data), 200

@app.route('/movie/<movie_id>', methods=['DELETE'])
def delete_movie(movie_id):
    db.delete(schema='movie_journal', table='movies', hash_values=[movie_id])
    return '', 204

@app.route('/review/<movie_id>', methods=['POST'])
def add_review(movie_id):
    review_data = request.get_json()
    review_data['movie_id'] = movie_id
    db.insert(schema='movie_journal', table='reviews', records=[review_data])
    return jsonify(review_data), 201

@app.route('/movie/<movie_id>', methods=['GET'])
def get_movie(movie_id):
    movie = db.search_by_hash(schema='movie_journal', table='movies', hashes=[movie_id], get_attributes=['*'])
    return jsonify(movie), 200

@app.route('/review/<movie_id>', methods=['GET'])
def get_reviews(movie_id):
    reviews = db.sql(f"SELECT * FROM movie_journal.reviews WHERE movie_id = '{movie_id}'")
    return jsonify(reviews), 200

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0')

Please replace 'YOUR_HARPERDB_URL', 'YOUR_USERNAME', and 'YOUR_PASSWORD' with your actual HarperDB URL, username, and password.

This code includes a route to serve the index.html file and static files from the static directory. The host='0.0.0.0' argument in app.run() is necessary to make your server publicly accessible if it's running in a Docker container.

For the frontend, you can create a templates directory in the same directory as your Python script and put your index.html file in it. You can also create a static directory for your app.js file and any other static files you have.

This code creates a Flask application with six routes:

1. /movie (POST): Adds a new movie to the database.

2. /movie/<movie_id> (PUT): Updates the details of a movie.

3. /movie/<movie_id> (DELETE): Deletes a movie.

4. /review/<movie_id> (POST): Adds a review for a movie.

5. /movie/<movie_id> (GET): Fetches the details of a movie.

6. /review/<movie_id> (GET): Fetches all reviews for a movie.

Each route corresponds to a function that interacts with the HarperDB instance using the HarperDB Python SDK.

Error handling?

It’s important to handle potential errors in our application to ensure it runs smoothly. Flask provides a way to handle errors using error handlers. For example, you can create an error handler for a 404 error (page not found) like this:

@app.errorhandler(404)
def page_not_found(e):
    return render_template('404.html'), 404

In the context of HarperDB, you should handle errors that might occur during database operations. For instance, trying to insert a record with a duplicate ID will result in an error. You can handle this by using a try/except block:

Frontend?

Here’s a simple HTML page that could serve as the frontend for the Flask app:

<!DOCTYPE html>
<html>
<head>
    <title>Movie Journal App</title>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
</head>
<body>
    <h1>Movie Journal App</h1>

    <h2>Add a Movie</h2>
    <form id="add-movie-form">
        <label for="title">Title:</label><br>
        <input type="text" id="title" name="title"><br>
        <label for="director">Director:</label><br>
        <input type="text" id="director" name="director"><br>
        <label for="year">Year:</label><br>
        <input type="text" id="year" name="year"><br>
        <input type="submit" value="Add Movie">
    </form>

    <h2>Movies</h2>
    <div id="movies"></div>

    <script src="app.js"></script>
</body>
</html>

And here’s some JavaScript code that uses jQuery to interact with the Flask app:

$(document).ready(function() {
    // Fetch all movies when the page loads
    $.get('/movie', function(data) {
        data.forEach(function(movie) {
            $('#movies').append('<p>' + movie.title + ' (' + movie.year + '), directed by ' + movie.director + '</p>');
        });
    });

    // Add a new movie when the form is submitted
    $('#add-movie-form').submit(function(e) {
        e.preventDefault();

        var title = $('#title').val();
        var director = $('#director').val();
        var year = $('#year').val();

        $.post('/movie', { title: title, director: director, year: year }, function(data) {
            $('#movies').append('<p>' + data.title + ' (' + data.year + '), directed by ' + data.director + '</p>');
        });

        // Clear the form
        $('#title').val('');
        $('#director').val('');
        $('#year').val('');
    });
});

This JavaScript code uses jQuery to interact with your Flask app. It fetches all movies when the page loads and adds a new movie when the form is submitted.

Testing the APIs

Once the APIs are ready, we need to test them to ensure they’re working as expected. We can use tools like Postman to send requests to the APIs and check the responses.

For example, to test the ‘add movie’ API, you can set up a POST request in Postman with the URL ‘http://localhost:5000/movie', set the body to raw JSON, and input a movie object like this:

{
    "title": "Inception",
    "director": "Christopher Nolan",
    "year": "2010"
}

Click ‘Send’ and you should see the movie object returned in the response.

Dockerizing your Flask app is an important step in the CI/CD pipeline.

Here’s how you can dockerize your Flask app:

1. Create a file named Dockerfile .

2. Open the Dockerfile and add the following:

# Use an official Python runtime as a parent image
FROM python:3.7-slim

# Set the working directory in the container to /app
WORKDIR /app

# Add the current directory contents into the container at /app
ADD . /app

# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

# Make port 5000 available to the world outside this container
EXPOSE 5000

# Run app.py when the container launches
CMD ["python", "app.py"]

This Dockerfile will create a Docker image that contains your Flask application and all its dependencies.

4. Build the Docker image by running the docker build -t

5. Once the Docker image is built, you can run your Flask app in a Docker container using docker run

With these steps, you have dockerized your Flask app. You can now include the Docker image building and pushing steps in the GitHub Actions workflow.

CI/CD Workflow using GitHub Action:

Next, let’s create a CI/CD pipeline using GitHub Actions. This pipeline will build the Docker image, push it to Google Container Registry (GCR), and deploy it to Google Cloud Run.

1. Set up your GCP Project:

• Create a GCP project if you haven’t already.

• Enable the necessary APIs: Cloud Run API and Container Registry API.

• Create a service account and download the JSON key file. This service account will be used to authenticate the GitHub Actions workflow with GCP.

2. Configure the GitHub Actions workflow:

• In your GitHub repository, create a .github/workflows directory.

• Create a new YAML file, e.g., deploy.yml, in the .github/workflows directory.

• Configure the workflow with the necessary steps and actions.

name: Deploy to Google Cloud Run

on:
  push:
    branches:
      - main

env:
  PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
  GCR_LOCATION: us
  SERVICE_NAME: movie-journal-app
  GCR_PATH: gcr.io/$PROJECT_ID/$SERVICE_NAME

jobs:
  setup-build-publish-deploy:
    name: Setup, Build, Publish, and Deploy
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Set up Cloud SDK
      uses: google-github-actions/setup-gcloud@v0.2.1
      with:
        project_id: ${{ secrets.GCP_PROJECT_ID }}
        service_account_key: ${{ secrets.GCP_SA_KEY }}
        export_default_credentials: true

    - name: Configure Docker to use the gcloud command-line tool as a credential helper
      run: |
        gcloud --quiet auth configure-docker $GCR_LOCATION-docker.pkg.dev

    - name: Build
      run: |
        docker build \
          --tag "$GCR_PATH:$GITHUB_SHA" \
          --build-arg GITHUB_SHA="$GITHUB_SHA" \
          --build-arg GITHUB_REF="$GITHUB_REF" \
          .

    - name: Publish
      run: |
        docker push "$GCR_PATH:$GITHUB_SHA"

    - name: Deploy
      run: |
        gcloud run deploy "$SERVICE_NAME" \
          --image "$GCR_PATH:$GITHUB_SHA" \
          --platform managed \
          --region "$GCR_LOCATION" \
          --allow-unauthenticated

Make sure to replace the following placeholders in the YAML file:

• ${{ secrets.GCP_PROJECT_ID }} with your GCP project ID.

• ${{ secrets.GCP_SA_KEY }} with your GCP service account key. Store it as a secret in the GitHub repository.

4. Configure GitHub Secrets:

In your GitHub repository, go to “Settings” > “Secrets”.

Add the following secrets:

• GCP_PROJECT_ID: Your GCP project ID.

• GCP_SA_KEY: The contents of the service account JSON key file (base64-encoded). You can encode the JSON key file using the following command:

cat path/to/key-file.json | base64

5. Commit and push your changes to trigger the workflow.

• This GitHub Actions workflow will trigger on every push to the main branch. It will build the Docker image, push it to GCR, and deploy it to Cloud Run.

Security Considerations?

When deploying applications, it’s crucial to consider security. For HarperDB, ensure you keep your HarperDB instance secure by limiting who can access it. Use strong, unique passwords, and consider implementing IP whitelisting.

For the Flask app, consider using HTTPS to ensure the secure transmission of data between the client and the server. You can obtain an SSL certificate and configure your Flask app to use it.

When deploying on GCP, follow the principle of least privilege when setting up your service account. This means giving it only the permissions it needs to perform its tasks.

Troubleshooting? I got you covered.

If you encounter issues while following this tutorial, here are a few common problems and their solutions:

• Issue: Flask app not starting or showing an error message.
  Solution: Check your Flask app for syntax errors or missing dependencies. Make sure you’ve installed all the required packages listed in the requirements.txt file.

• Issue: HarperDB operations are not working as expected.
  Solution: Ensure that your HarperDB instance is running and that you’ve correctly configured the HarperDB Python SDK in your Flask app.

• Issue: CI/CD pipeline failing in GitHub Actions.
  Solution: Check the logs in GitHub Actions to see where the pipeline is failing. Make sure you’ve correctly set up the GCP project and service account, and that the Dockerfile is correctly building the Docker image.

• Issue: The app is not accessible after deploying to GCP.
  Solution: Check the logs in Google Cloud Run to see if the app is running. Make sure you’ve correctly configured the service in Cloud Run and that the Docker image is correctly pushed to the Google Container Registry.

Conclusion

And there you have it! We’ve built a movie journal app using HarperDB and Python. This application allows users to manage their data efficiently with HarperDB as the backend database. Explore further by enhancing the user interface, implementing authentication and authorization, or adding additional features to tailor the app to your specific requirements.

I hope you’ve enjoyed it and have learned something new. I’m always open to suggestions and discussions on LinkedIn. Hit me up with direct messages.

If you’ve enjoyed my writing and want to keep me motivated, consider leaving starts on GitHub and endorse me for relevant skills on LinkedIn.

Till the next one, happy coding!