Page Scrape
Once you have determined your page style, you will open the Data Miner window. This is where you run the recipes to perform the scraping.
Page scrape is where basic scraping takes place. You must visit a website and run a recipe. In this tutorial, we will cover how to scrape a single page and how to scrape multiple pages. If you wanted to scrape inside links, this is covered in the Crawl tutorial.
New Scrape / Scrape and Append Video
In this video we will cover how to use New Scrape and Scrape & Append. Continue below for written instructions
Next Page Automation Video
In this video we will cover how run Next Page Automation. Continue below for written instructions
Written instructions
Step 1 - Visit the site you want to scrape
Data must be visible on the page in order for Data Miner to scrape it.
Click the Data Miner extension in the top right corner of the browser.
Click "Start Scraping".
Step 2 - Click Page Scrape
Page scrape is where you scrape data.
On The left, you will see a list of Recipes (recipes scrape data).
On the right, you will see three scraping methods.
- New Scrape
- Scrape and Append
- Next Page Automation
Step 3 - Preview a Recipe
Try a recipe by clicking "Preview".
After selecting a recipe, a preview of data will display on the right-hand side.
If the data is wrong or needs improvements, you can edit the recipe by clicking "Edit".
You can also learn how to write recipes from our
Recipe Creator
section of the documentation.
Step 4 - Select a Scrape Method
You can scrape data using 3 different methods.
New Scrape - Will only scrape once and replaces old data. Use case - a single table that you need to scrape daily.
Scrape and Append - Will only scrape once and adds to existing data. Use case - when you need to manually visit many pages.
Next Page Automation - Automatically click a "Next" button and scrape many pages, accumulating to one file. Use case - a list of products with numbered pages at the bottom.
Once you've chosen your method, click the blue Scrape button, the recipe will run and the data will be scraped.
Step 5 - Download
View your data. Download as a CSV, Excel file or Copy to your clipboard. Or use "Save As" to rename and keep the data in Data Miner.
Try on your own!
Using our Practice Sandbox try running through the above steps on your own!
Additional Documentation
Do you have to click into a Detail page to access the data? This is covered in the Crawl section.
Do you not see any Public Recipes? You can learn to make recipes yourself from our Recipe Creator section.