In this tutorial, I will guide you through the process of installing Selenium on a Mac and using it in Python to search for smartwatches on Amazon.
Selenium is a powerful tool used for web scraping and browser automation. It's like a robot that can interact with websites, extract information, and perform actions just like a human would.
Selenium allows you to automate browser actions, such as clicking buttons, filling out forms, and navigating between pages. This makes it an excellent tool for tasks like data extraction (web scraping), web testing, and automating repetitive tasks on websites.
Tutorial
Install Chrome & Chromedriver
To get started with Web Scraping on Mac OS, we need to install Google Chrome and Chromedriver. You can do this by running the following Bash-Script in your terminal.
Chromedriver is a tool that helps you automate and control Google Chrome for tasks like web scraping and testing.
1) Copy Bash Script
First, copy the following code and create a new file called: mac_update_chromedriver_and_chrome.sh
Registered members can directly download the Bash-Script-File for MacOS or Linux at the bottom of this blog post.
2) Run Bash Script
Now run your Bash-Script it from your Terminal app on Mac OS.
- Open the Terminal App and navigate to the directory where the script is located using the "cd" command
cd /Users/nick/Desktop
2. Make the script executable by running the command:
chmod +x update_chromedriver_and_chrome.sh
What does the 'chmod' command do?
chmod is a command in MacOs and Linux that stands for "change mode". It is used to change the permissions of files and directories.
The chmod command allows you to set permissions for three different types of users: the owner of the file, members of the group that the file belongs to, and all other users.
The permissions that can be set using chmod are read (r), write (w), and execute (x). These permissions can be set or removed for each of the three types of users.
For example, the command "chmod +x myscript.sh" would add the execute permission to the file "myscript.sh", allowing the user to run the script.
3. Run the script by typing the command:
./update_chromedriver_and_chrome.sh
When running the script for the first time, you may encounter the following error message:
If you encounter this error, navigate to:
"System Preferences" --> "Security & Privacy"
and click "Allow Anyway"
Now run your bash script again using the command:
./update_chromedriver_and_chrome.sh
3) Check if Installation was Successful
Run the following code in your terminal to check if the installation of Chrome & Chromedriver was successful:
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --version && chromedriver --version
4) Install Selenium in Python Environment
First, activate your Python-Environment. Replace 'nicks-env' with your own python environment you'd like to use:
source venv/nicks-env/bin/activate
And install selenium using
pip install selenium
5) Run a little test script
This script opens Chrome, goes to amazon.com and searches for 'smartwatch'.
Note that the script uses the 'time' library, so you might to 'pip install time' first.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
print("start")
driver = webdriver.Chrome()
url = 'https://www.amazon.com'
driver.get(url)
time.sleep(1)
driver.find_element(by=By.XPATH, value='//*[@id="twotabsearchtextbox"]').send_keys('smartwatch')
time.sleep(1)
driver.find_element(by=By.XPATH, value='//*[@id="nav-search-submit-button"]').click()
time.sleep(5)
driver.close()
print("end")
If you see 'start' and 'end' printed in the console, everything worked as expected.
Additional Resources
Subscribe to this Blog and my Free Newsletter to get Additional Resources
- For Mac: File Update Chromedriver and Chrome
- For Linux: File Update Chromedriver and Chrome