Once you have determined your page style, you will open the Data Miner window. This is where you run the recipes to perform the scraping.
Page scrape is where basic scraping takes place. You must visit a website and run a recipe. In this tutorial, we will cover how to scrape a single page and how to scrape multiple pages. If you wanted Data Miner to visit the website for you, this is covered in the Crawl tutorial.
Page Scrape Overview Video
Written Page Scrape Instructions:
- Navigate to the page you want to scrape, open up Data Miner, click "Start Scraping".
- Select Page Scrape form the left hand side.
- Click the "Select a Recipe" Tab and choose your recipe. You can choose from Public, Generic, or My Recipes. To test a recipe, simply click on the Recipe and a single scrape will run and data will display on the right hand side of Data Miner.
- If no data is extracted. Continue trying additional Recipes. You can refine the recipe list by filtering and searching recipes.
- If no additional Recipes are available, you can learn how to write your own by clicking "New Recipe" on the left hand side of Data Miner.
- Once you have a recipe that extracts the data you want. Continue to the Scrape tab. This is where you can continue to scrape additional pages using "Scrape Page Again" or "Next Page".
- Once you have finished scraping all the desired detail pages, continue to the download tab. From this tab you can download the data directly to your computer as a CSV file, Excel file or you can copy it to your clipboard. Data Miner also has the option to Save it directly in Data Miner. Just give the file a name and click "Save As". This data will be available from the Save Results folder on the left hand side. Once the data is saved or downloaded, you can close Data Miner.
The "Scrape Page again" button allows you to manually visit a new page of the same style and scrape new data into the same file. "Next Page Pagination" is when you want Data Miner to automatically click a Next button and scrape data. (Only available for List Recipes.)
Try on your own!
Using our Practice Sandbox try running through the above steps on your own!