Writing Recipes with Recipe Creator

Recipe Creator is a feature inside Data Miner that allows you to create your own recipes for any website. In 8 minutes you'll understand Recipe Creator basics, like hover & shift, selecting rows, selecting columns and Next Page Pagination. Watch the videos or written step by step instructions below.

Watch The Row Selection Video:

Written Row Selector Instructions:
  1. Navigate to the site you want, launch Data Miner, click New Recipe and pick your page type.
  2. List pages require rows and have multiple pages, while detail pages only have one page and only require columns.
  3. Starting with a list page, navigate to the Rows tab, click the "Find" button and using your mouse hover over the website data until a highlighted box encloses all the information you are looking to scrape.
  4. Once the Row is highlighted, press shift on your keyboard, then over on the tool select one of the suggested classes or html elements to lock in the selection.
  5. Your goal is to get a green outline wrapping around all the individual row containers on your list page. Once this is done you can move onto columns.

Watch The Column Selection Video:

Written Column Selector Instructions:
  1. To begin selecting the individual data. Click on the Column tab and select “column 1”.
  2. Give the column a name, click the "Find" button, and hover & shift over the data you wish to extract into this column.
  3. Pick the class or HTML element that highlights the data the best. (Helpful tip - use the "Select Parent" button for more options if the selector outlines too many other items)
  4. Once the data is highlighted correctly, lock in the selector by clicking "confirm" at the bottom of Recipe Creator and click the Eyeball to check your work.
  5. Continue creating by clicking "Add new Column".
  6. Once you have all your columns done, finish by clicking the Save tab at the top. Give the recipe a name, click save and then run to scrape.

Watch The Next Page Video:

Written Next Page Instructions:
  1. Similar to Rows and Columns, click on the Nav Tab.
  2. Click the "Find" button to activate the find tool.
  3. Locate the next page button on the web page.
  4. Hover your mouse over the next page button and hit shift on your keyboard
  5. Select one of the suggested selectors and hit confirm.
  6. Click "Test Navigation" to see if it works.
  7. If it doesn't work, Recipe Creator needs the next page URL, which is found in the <a> element. Click parent until you find it and choose those selectors.
  8. *An advanced selector that works for most cases is a:contains(Next)

Nav with No Find (Xpath / Auto-generated Recipe):

**Demonstrated with Data Miner 2017**

Written Next Page Instructions:
  1. Similar to Rows and Columns, click on the Nav Tab.
  2. Right click on page, select Inspect, select Elements in new window.
  3. Select the "arrow in a box" icon on the top left of the new window
  4. Locate the next page button on the web page and click it.
  5. Find the a element and identify the class associated with it.
  6. Now type .//a[@class="XXXX"] into your selector input box replacing the XXXX with your class.
  7. This selector will vary across sites, you may need to replace class with a different attribute. For example .//a[@title="Next""]
  8. *An advanced selector that works for most cases is .//a[contains(text(),"Next")]