Advanced Configuration

🎯 Selenium Advanced Configuration

Become a Browser Whisperer!

🌟 The Big Picture: Your Browser’s Secret Settings

Imagine you have a robot friend who can use any web browser for you. That robot is Selenium. But sometimes, the robot needs special instructions to handle tricky situations—like sneaking through a secret tunnel (proxy), showing a special ID card (SSL certificates), catching packages being delivered (downloads), or knowing the password to get into a locked room (authentication popups).

Today, you’ll learn how to give your robot friend these super powers!

🔮 The Universal Analogy: The Delivery Robot

Think of Selenium as a delivery robot going to different buildings (websites):

• Proxy = Taking a secret underground tunnel instead of the main road
• SSL Certificate = The security guard checking if you’re really who you say you are
• Downloads = Catching packages that buildings throw at you
• Authentication Popup = A door that asks “What’s the password?”

Let’s learn each one!

🚇 1. Proxy Configuration

Taking the Secret Tunnel

What is a Proxy?

A proxy is like a middleman. Instead of going directly to a website, your browser first goes to the proxy, and the proxy goes to the website for you.

Why use it?

• Your school/office might block certain websites
• You want to appear like you’re browsing from a different place
• You want to keep your real location private

graph LR
    A["🤖 Your Browser"] --> B["🚇 Proxy Server"]
    B --> C["🌐 Website"]
    C --> B
    B --> A

Simple Example: Setting Up a Proxy

from selenium import webdriver

# Create proxy settings
proxy = "123.45.67.89:8080"

# Chrome with proxy
options = webdriver.ChromeOptions()
options.add_argument(
    f'--proxy-server={proxy}'
)

driver = webdriver.Chrome(
    options=options
)

Proxy with Username & Password

Some proxies need a login. It’s like the secret tunnel having a guard.

from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType

# For authenticated proxy
proxy = Proxy()
proxy.proxy_type = ProxyType.MANUAL
proxy.http_proxy = "user:pass@proxy.com:8080"
proxy.ssl_proxy = "user:pass@proxy.com:8080"

options = webdriver.ChromeOptions()
options.add_argument(
    '--proxy-server=http://proxy.com:8080'
)

Different Types of Proxies

# SOCKS proxy example
options.add_argument(
    '--proxy-server=socks5://127.0.0.1:9050'
)

🔐 2. SSL Certificate Handling

The Security Guard’s ID Check

What is SSL?

When you see the little lock icon 🔒 in your browser, that’s SSL working! It means the website proved it’s real and not a fake.

Sometimes, test websites don’t have proper certificates (like a friend without their ID card). Selenium can be told to trust them anyway.

graph LR
    A["🤖 Browser"] -->|"Hello!"| B["🌐 Website"]
    B -->|"Here's my certificate"| A
    A -->|"Certificate OK ✓"| C["🔒 Secure Connection"]
    A -->|"Certificate BAD ✗"| D["⚠️ Warning!"]

Ignore SSL Errors (For Testing Only!)

⚠️ Warning: Only use this for testing! Never in real apps!

from selenium import webdriver

options = webdriver.ChromeOptions()

# Tell Chrome to ignore certificate errors
options.add_argument(
    '--ignore-certificate-errors'
)
options.add_argument(
    '--ignore-ssl-errors'
)

driver = webdriver.Chrome(
    options=options
)

Firefox SSL Settings

from selenium import webdriver

profile = webdriver.FirefoxProfile()

# Accept untrusted certificates
profile.accept_untrusted_certs = True
profile.assume_untrusted_cert_issuer = True

driver = webdriver.Firefox(
    firefox_profile=profile
)

Using Custom Certificates

Sometimes you need to use your own certificate:

from selenium import webdriver

options = webdriver.ChromeOptions()

# Add your custom certificate
options.add_argument(
    '--ignore-certificate-errors'
)
options.add_experimental_option(
    'prefs', {
        'ssl.certificate_errors': False
    }
)

📦 3. Download Handling

Catching the Packages!

The Problem

When a website wants to give you a file (like a PDF or image), Selenium doesn’t know where to put it! We need to tell it.

graph TD
    A["🌐 Website"] -->|"Here's a file!"| B["📦 File"]
    B --> C{Where to save?}
    C -->|"Default folder"| D["📁 Downloads"]
    C -->|"Custom folder"| E["📁 Your Choice"]

Chrome Download Settings

from selenium import webdriver
import os

# Set your download folder
download_path = "/path/to/downloads"

options = webdriver.ChromeOptions()
prefs = {
    # Where to save files
    "download.default_directory":
        download_path,
    # Don't ask where to save
    "download.prompt_for_download":
        False,
    # Allow automatic downloads
    "download.directory_upgrade":
        True,
    # Disable safe browsing for files
    "safebrowsing.enabled":
        True
}
options.add_experimental_option(
    "prefs", prefs
)

driver = webdriver.Chrome(
    options=options
)

Firefox Download Settings

from selenium import webdriver

profile = webdriver.FirefoxProfile()

# Download folder (2 = custom location)
profile.set_preference(
    "browser.download.folderList", 2
)
profile.set_preference(
    "browser.download.dir",
    "/path/to/downloads"
)

# Don't show download window
profile.set_preference(
    "browser.download.manager.showWhenStarting",
    False
)

# Auto-save these file types
profile.set_preference(
    "browser.helperApps.neverAsk.saveToDisk",
    "application/pdf,text/csv,application/zip"
)

driver = webdriver.Firefox(
    firefox_profile=profile
)

Waiting for Download to Complete

import time
import os

def wait_for_download(folder, timeout=30):
    """Wait until download finishes"""
    seconds = 0
    while seconds < timeout:
        time.sleep(1)
        # Check for temp files (.crdownload)
        downloading = False
        for file in os.listdir(folder):
            if file.endswith('.crdownload'):
                downloading = True
        if not downloading:
            return True
        seconds += 1
    return False

# Usage
wait_for_download("/path/to/downloads")

Handling Different File Types

🚪 4. Authentication Popup Handling

Knowing the Password

What’s an Authentication Popup?

Some websites show a popup box asking for username and password BEFORE showing the page. It’s like a bouncer at a club asking “What’s the password?”

graph TD
    A["🤖 Browser"] -->|"I want to enter"| B["🚪 Website Door"]
    B -->|"Password please!"| C["📝 Login Popup"]
    C -->|"user:pass"| D["✅ Welcome!"]

Method 1: URL with Credentials

The easiest way - put the username and password IN the URL!

from selenium import webdriver

# Format: https://username:password@website.com
url = "https://admin:secret123@example.com/secure"

driver = webdriver.Chrome()
driver.get(url)

Method 2: Using Alert Handling

from selenium import webdriver
from selenium.webdriver.common.alert import Alert
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("https://example.com/protected")

# Wait for the authentication popup
try:
    alert = WebDriverWait(driver, 10).until(
        EC.alert_is_present()
    )
    # Send credentials
    alert.send_keys("username")
    alert.send_keys("\t")  # Tab key
    alert.send_keys("password")
    alert.accept()
except:
    print("No popup appeared")

Method 3: Using AutoIT (Windows)

For complex popups, you might need AutoIT:

import subprocess

# After popup appears
subprocess.call([
    'autoit_script.exe',
    'username',
    'password'
])

Method 4: Browser Extension (Advanced)

You can create a Chrome extension that handles auth:

import zipfile
import os

# Create extension files
manifest = '''
{
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Auth Extension",
    "permissions": [
        "webRequest",
        "webRequestBlocking",
        "<all_urls>"
    ],
    "background": {
        "scripts": ["bg.js"]
    }
}
'''

background_js = '''
chrome.webRequest.onAuthRequired.addListener(
    function(details) {
        return {
            authCredentials: {
                username: "myuser",
                password: "mypass"
            }
        };
    },
    {urls: ["<all_urls>"]},
    ['blocking']
);
'''

# Save as extension
with zipfile.ZipFile('auth.zip', 'w') as z:
    z.writestr("manifest.json", manifest)
    z.writestr("bg.js", background_js)

🎁 Putting It All Together

Complete Configuration Example

from selenium import webdriver

def create_configured_driver():
    options = webdriver.ChromeOptions()

    # 1. PROXY
    options.add_argument(
        '--proxy-server=http://proxy:8080'
    )

    # 2. SSL
    options.add_argument(
        '--ignore-certificate-errors'
    )

    # 3. DOWNLOADS
    prefs = {
        "download.default_directory":
            "/downloads",
        "download.prompt_for_download":
            False,
    }
    options.add_experimental_option(
        "prefs", prefs
    )

    driver = webdriver.Chrome(
        options=options
    )

    # 4. AUTH via URL
    driver.get(
        "https://user:pass@site.com"
    )

    return driver

📝 Quick Reference Table

🌟 Key Takeaways

1. Proxy = Secret tunnel for your browser traffic
2. SSL = Trust or verify website certificates
3. Downloads = Tell browser where to save files
4. Auth Popup = Handle password doors automatically

You’re now a Browser Configuration Master! 🎉

These skills let you automate browsers in real-world situations where websites have security, need logins, or send files. Practice each one, and you’ll be unstoppable!

💡 Remember: With great power comes great responsibility. Only use SSL bypasses and proxies for legitimate testing purposes!