How to change WebDriver’s IP address

I have already written several articles on how to use WebDriver for web scraping, but I have never touched on the topic of changing WebDriver’s IP address. Nevertheless, this topic is quite crucial when you come to web scraping, and here I’d like to show you an example of using proxies with WebDriver in Python (and you can easily convert it into your language API).

First of all you need a proxy server. You can use any proxy service you like, but if you don’t have one, I would recommend that you consider BestProxyAndVPN (if you’re ready to pay for proxy services). I have tried many proxy services and ended up using this one as the most stable.

Now let’s look at two pieces of code displaying how to make WebDriver send its requests through a proxy.

Using Proxy with Chrome Driver

If you use Selenium WebDriver with the Chrome browser, you can tune it to use a proxy in the following way:

Don’t forget to change the proxy server address and ChromeDriverPath to your values.

Using Proxy with Firefox WebDriver

If you prefer scraping with the Firefox browser, you can cause it to use proxy with the following code:

And again, don’t forget to change the proxy address and port to your values.

EDIT: Here is another way to specify proxy servers for Firefox webdriver (thanks to Curro Pavon):

 

发布了430 篇原创文章 · 获赞 415 · 访问量 925万+
展开阅读全文

没有更多推荐了,返回首页

©️2019 CSDN 皮肤主题: 编程工作室 设计师: CSDN官方博客

分享到微信朋友圈

×

扫一扫,手机浏览