Contents

Setting up

This tutorial will show how to build a web scraper using JavaScript code.

Before we begin, we need to setup the work environment -

1. NodeJS - a back-end JavaScript runtime environment that runs on a JavaScript Engine and executes JavaScript code outside a web browser. Download and install it on your system.
2. VSCode - or any other code editor where we will write our code. Download and install it.
3. GitHub - this is where we will store our code for automation purposes. If you are reading this, chances are you already are familiar with GitHub and have an account there.

Download or clone the repository from GitHub to see the demo scraper in action.

Once you have NodeJS setup, you can demo the scraper as follows -

• Open up the folder in VSCode.
• Run yarn in the terminal to install the dependancy packages. If you don’t have yarn, install it from here.
• Run the scraper by running the command node ./index.js in the terminal.

You should see datasets being downloaded and saved in the database folder.