Web Scraping Basics with Python

Web scraping lets you pull data from websites automatically — prices, listings, headlines, anything on a page. Here is a clean, beginner-friendly starting point using Python’s most popular scraping tools.

1. Install the tools

requests downloads the page; beautifulsoup4 reads the HTML.

pip install requests beautifulsoup4

2. The script

This example fetches a page and prints the text of every heading on it. Swap the URL and the tag to grab whatever you need.

import requests
from bs4 import BeautifulSoup

URL = "https://example.com"

response = requests.get(URL, headers={"User-Agent": "Mozilla/5.0"})
response.raise_for_status()

soup = BeautifulSoup(response.text, "html.parser")

for heading in soup.select("h1, h2, h3"):
    text = heading.get_text(strip=True)
    if text:
        print(text)

3. Grab the data you actually want

• soup.select("a") — all links.
• soup.select(".price") — every element with the class price.
• soup.find("h1").get_text() — the first H1 on the page.

Scrape responsibly

• Check the site’s robots.txt and terms of service first.
• Add a short delay between requests — do not hammer a server.
• For pages that load data with JavaScript, you may need a tool like Playwright or Selenium.

Need data pulled, cleaned and delivered?

I build custom scrapers and data pipelines that collect, clean and deliver exactly the data your business needs — on a schedule, into a spreadsheet, dashboard or database.