How Data Miner Works

Below are some of the most common questions customers have about Data Miner technology. Feel free to ask our support team anything we have not covered here.

A note about your privacy and security:

When using Data Miner the data you scrape is always private (see "Where is my data stored?"). Your data or your credentials never leave your browser and never touch our server. Only you have access to the data you scrape whether you have a free Data Miner subscription plan or a paid one. Your data is never shared with anyone nor we can access to it in any way. Data Miner only generates revenue from subscription plans and not from collecting or selling your data.

What are Recipes?

Recipes are data extraction instructions that Data Miner uses to extract data from websites. Recipes contain name and position of HTML elements on a web page. Recipes do not contain actual data or private information about you. When you visit a website Data Miner automatically filters thousands of recipes that our users have created and shared and shows only the ones that are appropriate for the site you are currently viewing.

Can Data Miner mask my IP or rotate IP addresses?

No. Data Miner works on your own machine using your own browser. We do not hide your traffic going to the the website or intercept nor change your web traffic in any way. In fact you can always see every page that data miner opens in your browser. There is no magic happening behind the scene. Data Miner simply automates your browser visiting a set of pages and extracting the data that is visible on the page.

Will Data Miner need my password for a website?

No. You use Data Miner after you have signed into a website manually on your own. Therefore Data Miner tool never needs to know your account credentials. Your passwords or account credentials are never captured by Data Miner and ever stored anywhere.

Where is my data stored?

The data you scrape is stored on your own machine inside the temporary storage of your browser. Your data is not stored on Data Miner servers. When you click on the download button your data is extracted from your temporary storage in your browser and save for you as a file. THe Data you extract never leaves your machine. You can assure yourself that your data is not on Data Miner server by disconnecting your internet connection and then clicking on download button. You will see that even without internet connection your data will be downloaded and saved on your computer. Meaning that your data was never stored on our servers in the first place.

There is only one copy of your data and that is inside your Chrome browser (local storage) on your own computer. This effectively protects your privacy but also means that if you delete your data or if your browser crashes your data collections will be lost. So we recommend downloading your data out of Data Miner periodically when you are scraping large set of data.

Is scraping data legal?

Saving data that you have legally access to is generally allowed under the terms of use of many websites. However many sites simply don't provide an export functionality or make it easy to export the data that they have access to. That is where Data Miner comes handy. Data Miner helps you transform data in your browser from HTML into a clean spread sheet table. You should always check the terms of service of the site you are using to make sure exporting your data is allowed.
Data Miner simply helps you save the data that you see your browser. If you don't have access to see the data then Data Miner can not export it. Data Miner works similarly to the print functionality of your browser. It can only export the data that you can see.

Will I be blocked by a site if I use Data Miner?

Data Miner does not make any requests to the server in the background or modify your outgoing requests to the website in any way. So there is no way for a site to know that you have installed Data Miner on your computer. Similar to the printing functionality of your browser when you print a page from website no notification is sent to a website that you have printed a page. Many websites have limits for how many pages a day or a month a user can visit. You can consult the terms of service of your website and stay within the limits they grant you.

Will Data Miner slow my computer?

Data Miner does not run anything in the background or consume your memory when you are not interacting with it. Data Miner consumes minimal memory and CPU when running and extracting data form a page.

Can I control the speed of scraping?

Yes, you can control your scraping speed by changing the wait time between page scrapes. When you are doing Auto-pagination or Batch scraping you can instruct Data Miner to wait a specific number of seconds in between the each page scrape. The wait time you provide is randomized by %50.

Do you buy or sell data?

We don't. We neither buy any data you have nor sell you any data. And more importantly we don't have access to the data that you scrape on your local machine. We are simply in the business of providing a tool for you to easily export your data of a web page that you have already access to.

Can you run the scraping for me?

We can't. Data Miner team only develops the Data Miner tool. We do not operate scraping on behalf of Data Miner users. We only provide you a personal tool and you run it on your own computer with your own internet connection.

Can you help me circumvent a website's security or account?

Absolutely not. Data Miner tool only works when your have legitimate access to a website and are able to see the data in your browser. If you can't see the data in your browser because you don't have access to it then Data Miner can't extract it for you.