=======Comprehensive Guide to Setting Up the Backend of the Application======= {{:ixc2024:tech:backend:screenshot_2024-05-23_103333.png?400|}} ==== Objective ==== The backend is the backbone of the application, handling data processing, storage, authentication, and communication between the mobile and web apps. It should be designed in a manner such that it is scalable, secure, and efficient to handle potentially large volumes of data and user interactions. This guideline will assist in setting up, configuring, and understanding the technologies required for the backend portion of this hackathon. ==== Components of the Backend ==== ^**Component** ^**Description** ^ |API Service |Exposes endpoints for the mobile and web apps to interact with the backend. | |Database |Stores user data, food waste records, recipes, tips, and other relevant information. | |Authentication |Manages user registration, login, and authorization. | |Data Processing|Handles business logic, data validation, and processing. | |Cloud Provider |Ensures scalability, reliability, and provides additional services like serverless computing, storage, and security.| ==== Technology Stack ==== The diagram below shows an overview of the cloud infrastructure. We will be using Google cloud and the two main services will be Google App Engine for deploying the web frontend and the API and Google Storage for handling data. Firebase will be used as the main database and GCS will be used for saving photos, pdfs, videos and other types of files. {{:ixc2024:tech:backend:screenshot_2024-05-23_103710.png?400|}} == Cloud Provider (Google Cloud) == We will be using Google Cloud because it has excellent machine learning and data analytics services, competitive pricing and a slightly easier learning curve than AWS and AZURE for beginners. Google also offers $300 USD credits to use on any of their services for newly created accounts. It has been verified with Google support that this offer will still be valid in August, in Malaysia. == API Frameworks == In the table below, we provide a list of popular frameworks that can be used to build the API service. You are free to use any framework of your choice however we recommend using express.js (node.js) for its simplicity, speed, and extensive middleware options. Our tutorial will be focused on express.js. ^**Framework** ^**Description** ^ |Express.JS (Node.JS)|Lightweight, fast, large community, easy to get started| |Django (Python) |Batteries included framework, built in admin panel, ORM| |SpringBoot (Java) |Enterprise grade, robust ecosystem, strong security | == Authentication == We will be using **Firebase** authentication. It is easy to implement and supports various authentication methods. The tutorial will show how to integrate Firebase Authentication into an Express.js backend. ==== Tutorial ==== ==Google Cloud Account Set Up== -> **Sign in and set Up A Billing Account** __Note:__ You have to link your bank card to the console for the purpose of setting up more parameters but you will **__NOT__** be charged anything for the free trial account. - Go to the [[https://console.cloud.google.com/|Google Cloud Console]] and sign in. - Click on the **Try For Free** button in the middle of the page to set up your 300$ free credits account - Select your country and click on **Agree & Continue** - Set your **Account Type** to individual, add your name and address, input your bank card information, then click on **Strat Free** - Click **Continue** on the window that appears to get prompted to a new identity verification tab. - Verify your identity by entering the code received on your bank application along with the temporary charge of 1eu __Note:__ Don't click on the button **Activate Full Account** as this will change your free trial account into a normal account that gets charged for it's usage -> **Create a New Project** 1. Go back to your main console page 2. Click on the box appearing at the left of the search bar {{:ixc2024:tech:tools:1.png?400|}} 3. Click on "Select a project" at the top right and then click on "New Project". 4. Choose a unique project name and select the desired organization if applicable. 5. Once the project is created, go to the notification icon and select the project you have just created {{:ixc2024:tech:tools:3.png?400|}} 6. Note down the Project ID that appears in the middle of the screen for future reference. {{:ixc2024:tech:tools:4.png?400|}} -> **Enable Required APIs** - Go to the search bar at the top of the screen and navigate to **Library APIs & Services**. (__Note:__ If the page fails to load, clear your browser's cache and cookies, try a different browser, or check if you have some enabled extensions interfering) - Search for **Cloud Storage API** and enable the Cloud Storage by Google Enterprise API. - Go back to the API library, search for **App Engine Admin API**, and enable the App Engine Admin API by Google Enterprise API. -> **Set Up Cloud Storage** - Click on **Google Cloud** at the top left of the page to go back to the main page of your project, go to the search bar again and navigate to **Cloud Storage**. - Click on **+ Create** to make a new bucket. - Choose a unique name and select the desired region. - Leave other settings as default and create the bucket. ==Create an API Service== -> **Install Node JS** We recommend using NVM (Node Version Manager). It makes it easy for managing multiple Node.js versions on the same machine. It works across macOS, Linux, and Windows (with nvm-windows). * Mac and Linux curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash source ~/.nvm/nvm.sh nvm install node * Windows - Download **‘nvm-setup.zip’** from the [[https://github.com/coreybutler/nvm-windows/releases|nvm-windows releases page]]. - Extract the contents and run the nvm-setup.exe installer. - Open your Command Prompt and install Node.js: nvm install latest nvm use latest * Verify the Installation (for all machine types) node -v npm -v -> **Create Express.js NodeJS Backend** * Initialize a new nodejs project Create a folder for your project - for this tutorial we will call it waste-backend and initialize a new node application. mkdir waste-backend cd waste-backend npm init -y * Install Initial Dependencies Install the basic dependencies to get started with your node js project npm install express axios cors body-parser - **Express.js**, or simply **Express**, is a back end web application framework for building RESTful APIs with Node.js - **Axios** is a node js client that helps you make http requests. You will be using this to send information to and from the frontend - **Cors** is It is used to enable cross-origin requests in web applications. This is essential for APIs that are accessed from a different domain than the one serving the client application. - **Body Parser** It is used to parse JSON, raw, text, and URL-encoded form data submitted in HTTP requests. This is essential for handling form submissions and JSON payloads in your APIs. -> **Install GCP CLI** - Make sure that you are in the root directoryyour Node app in the terminal, if not, run **'cd *name of your root directory*'**. - Install the Google Cloud SDK following the instructions at [[https://cloud.google.com/sdk/docs/install|Google Cloud SDK Installation Guide]]. (__Note:__When asked if you want to log in, input yes and log in to the google cloud account you are using) - Go back to your Command Prompt and enter the numeric choice associated with your project -> **Set-Up Service Account** - Go back to your cloud console through [[https://console.cloud.google.com/|Google Cloud Console]]. - Navigate through the search bar to **Service Accounts IAM & Admin**. - Click on **Create Service Account**. - Provide an appropriate name and description for the service account. For instance, use github-ci-cd as it will be utilized for Github CI/CD. - Click **Continue** on the 2nd step without needing to modify any default information - Click on **Done** - Scroll down to the service accounts available, click on the three dots of the one you just created and select Manage keys. - Scroll, Click **Add Key → Create New Key**. - Choose the JSON key type and securely download it. Remember, this key grants access to Google Cloud resources, so keep it safely saved. == Version Control == -> **Using GitHub** - Create a Github account or use an existing one. - Create a new repository, leave all the selected parameters at default - Initialize Git in API Service - Go to your Command Pronpmt an run **'git init'** - Go back to your created Github Repository, scroll down to **…or push an existing repository from the command line**, and copy the code displayed in your Command Prompt ==Integrating Firebase Authentication WIth Our API Service== -> **Install Firebase Admin SDK** Install the firebase-admin plugin from npm in your Command Prompt npm install firebase-admin -> **Initialize Firebase Admin SDK** In Visual Studio, open the folder you have been working on (In this example it is Waste-Backend), create a file named auth.js in your project’s root directory and enter the code below: const admin = require('firebase-admin'); const serviceAccount = require('./path/to/serviceAccountKey.json'); admin.initializeApp({ credential: admin.credential.cert(serviceAccount), }); async function verifyToken(token) { try { const decodedToken = await admin.auth().verifyIdToken(token); return decodedToken; } catch (error) { throw new Error('Authentication failed'); } } module.exports = { verifyToken }; -> **Creating Express.js Server with Authentication Endpoints** Create a file named index.js in your projects root directory and enter the code below: const express = require('express'); const { verifyToken } = require('./auth'); const admin = require('firebase-admin'); const app = express(); const PORT = process.env.PORT || 3000; app.use(express.json()); // Endpoint for user registration app.post('/register', async (req, res) => { const { email, password } = req.body; try { const user = await admin.auth().createUser({ email, password, }); res.status(201).send(user); } catch (error) { res.status(400).send(error.message); } }); // Endpoint for user login (generates custom token) app.post('/login', async (req, res) => { const { email, password } = req.body; try { const user = await admin.auth().getUserByEmail(email); // Normally, you would verify the password here // Since Firebase Admin SDK does not handle password verification, this is just for demonstration const token = await admin.auth().createCustomToken(user.uid); res.status(200).send({ token }); } catch (error) { res.status(400).send(error.message); } }); // Middleware to protect routes app.use(async (req, res, next) => { const token = req.headers.authorization?.split(' ')[1]; if (token) { try { req.user = await verifyToken(token); next(); } catch (error) { res.status(401).send('Unauthorized'); } } else { res.status(401).send('Unauthorized'); } }); // Protected route example app.get('/profile', (req, res) => { res.send(`Hello ${req.user.email}`); }); app.listen(PORT, () => { console.log(`Server is running on port ${PORT}`); }); This will allow you to create endpoints that can be used by the frontend to send or access data. The following URLs can be used to access the different endpoints. __Note:__ Additional resources on how to use firebase authentication with your backend and frontend can be found in the //Additional Resources// section. ==Deploying API Service to Google Cloud == -> **Create app.yaml** To deploy the app we need to create a configuration file in our project's root directory. This file has to be a yaml file as per Google’s recommendation. For the purpose of this tutorial we will create a file called app.yaml in our root directory and add the settings and routing for Google App Engine. - The app.yaml file configures settings and routing for Google App Engine applications. - Keep the app.yaml in your project's root directory alongside your source code. # [START app_yaml] runtime: nodejs20 entrypoint: node index.js # [END app_yaml] __Notes:__ * runtime: Specifies the runtime environment (e.g., nodejs20) * service: Defines the service name, typically a project-specific prefix or subdomain. * entrypoint: By default App Engine expects server.js as the entry point but if your app uses anything else then you specify that here. -> **Deploy your App Locally** - Close your command prompt and open it again - Run **'gcloud** to make sure that everything is connected - Navigate to your project folder **'cd waste-backend'** - Run the command **'gcloud init'** - Select **'1'** to re-initialize the configuration [default] with new settings - Select the email account that you are using in your Cloud Console to perform operations for this configuration - Pick the number allocated to your project - Deploy your Node app by executing the command in root directory of your project gcloud app deploy 5. Choose a region where you want your App Engine Application to be located ====Additional Resources==== ^Title ^Link ^ |How to deploy Node JS to Google Cloud |[[https://dev.to/rushi-patel/deploy-node-js-project-to-google-app-engine-with-github-actions-cicd-a-complete-guide-3od9|https://dev.to/rushi-patel/deploy-node-js-project-to-google-app-engine-with-github-actions-cicd-a-complete-guide-3od9]] | |Google Cloud tutorial |[[https://cloud.google.com/gcp?utm_source=bing&utm_medium=cpc&utm_campaign=emea-fi-all-fi-bkws-all-all-trial-e-gcp-1011340&utm_content=text-ad-none-any-DEV_c-CRE_-ADGP_Hybrid+%7C+BKWS+-+EXA+%7C+Txt+-+GCP+-+General+-+v3-KWID_43700061979378565-kwd-77378331205405:loc-65-userloc_148537&utm_term=KW_google%20cloud-NET_o-PLAC_&&refclickid=NNE_HotelDigitalMediaCampaign&pmed=DPM_CVG_TRV_HOT_NNE_SEM_GOG_&gclid=c643c2b3606910b67384a4ff2b66e033&gclsrc=3p.ds&|Cloud Computing, Hosting Services, and APIs | Google Cloud]]| |Python FastAPI tutorial |[[https://realpython.com/fastapi-python-web-apis/|https://realpython.com/fastapi-python-web-apis/]] | |Integrating cloud SQL into your Node JS app |[[https://github.com/GoogleCloudPlatform/cloud-sql-nodejs-connector|GitHub - GoogleCloudPlatform/cloud-sql-nodejs-connector: A JavaScript library for connecting securely to your Cloud SQL instances]] | |Getting started with NodeJS, Google Cloud, Firebase and Cloud Storage|[[https://cloud.google.com/nodejs/getting-started|https://cloud.google.com/nodejs/getting-started]] | ==== Partners ==== * [[https://www.lut.fi/en|Lappeenranta-Lahti University of Technology]] * [[https://sunwayuniversity.edu.my|Sunway University]] * [[https://www.utu.fi/en|Turku University]]