Configure Proxies for Selenium: Setup, Scale and Debugging

Selenium proxy configuration is more awkward than it should be, and most of the awkwardness comes from one design decision: Chrome's --proxy-server flag accepts a host and port but not credentials. Everything else follows from working around that. This covers the setup for both browsers, how to structure rotation across a Selenium fleet, running through Grid, and a debugging procedure for when it silently fails.

The Basic Setup

For a proxy that does not require authentication, Chrome is straightforward:

from selenium import webdriver
 
options = webdriver.ChromeOptions()
options.add_argument("--proxy-server=http://gateway.flameproxies.com:8080")
 
driver = webdriver.Chrome(options=options)
driver.get("https://ipcheck.flameproxies.com")

Firefox uses profile preferences rather than a flag:

from selenium import webdriver
 
options = webdriver.FirefoxOptions()
options.set_preference("network.proxy.type", 1)
options.set_preference("network.proxy.http", "gateway.flameproxies.com")
options.set_preference("network.proxy.http_port", 8080)
options.set_preference("network.proxy.ssl", "gateway.flameproxies.com")
options.set_preference("network.proxy.ssl_port", 8080)
# Resolve DNS through the proxy, not locally
options.set_preference("network.proxy.socks_remote_dns", True)
 
driver = webdriver.Firefox(options=options)

Two Firefox details worth noting. network.proxy.type = 1 means manual configuration — without it the other preferences are ignored. And you must set ssl separately from http, or HTTPS traffic bypasses the proxy entirely, which is the most common Firefox misconfiguration.

Authentication, Briefly

Credentials cannot go in the Chrome flag. The practical options are IP allowlisting (simplest where your egress is stable), a small generated Chrome extension that supplies credentials via webRequest, or selenium-wire, which proxies the browser through a local layer that handles auth.

Firefox can be made to work with a profile and an auth-handling extension, but in practice most teams either allowlist or route through a local forwarder.

If your infrastructure has a stable outbound address, allowlisting removes this entire category of problem and is worth preferring. Combining allowlisting with credentials where the provider supports both also means a leaked credential is useless from an unlisted address.

Targeting and Per-Session Rotation

The key structural point for Selenium: targeting lives in the username, and a driver instance holds one proxy configuration for its lifetime. You cannot change the proxy of a running Chrome driver. Rotation therefore happens at driver creation.

That makes the driver the unit of rotation:

import uuid
from selenium import webdriver
 
PROXY_HOST = "gateway.flameproxies.com"
PROXY_PORT = 8080
 
def make_driver(country=None, city=None, session=None):
    session = session or uuid.uuid4().hex[:12]
    parts = [PROXY_USER]
    if country: parts.append(f"country-{country}")
    if city:    parts.append(f"city-{city}")
    parts.append(f"session-{session}")
    username = "-".join(parts)
 
    options = webdriver.ChromeOptions()
    options.add_argument(f"--proxy-server=http://{PROXY_HOST}:{PROXY_PORT}")
    # plus whichever auth approach you use, carrying `username`
 
    driver = webdriver.Chrome(options=options)
    driver.session_label = session
    return driver

Because driver startup is expensive — a second or more of browser launch — do not create a fresh driver per request. Create one per logical session and run a batch of related work through it, then dispose and recreate when you want a new IP.

A reasonable pattern:

def run_batch(urls, country, per_driver=20):
    results = []
    driver = make_driver(country=country)
    try:
        for i, url in enumerate(urls):
            if i and i % per_driver == 0:
                driver.quit()
                driver = make_driver(country=country)   # new IP
            driver.get(url)
            results.append(extract(driver))
    finally:
        driver.quit()
    return results

Tune per_driver from the target's measured tolerance, and lower it if block rates rise.

Matching the Locale Bundle

A proxy IP alone does not make a coherent visitor. Selenium gives you control over the rest, and you should use it:

def make_driver(country, city=None):
    lang, tz = LOCALE[country]      # e.g. ("de-DE", "Europe/Berlin")
 
    options = webdriver.ChromeOptions()
    options.add_argument(f"--proxy-server=http://{PROXY_HOST}:{PROXY_PORT}")
    options.add_argument(f"--lang={lang}")
    options.add_experimental_option("prefs", {
        "intl.accept_languages": lang,
    })
 
    driver = webdriver.Chrome(options=options)
 
    # Timezone must be set via CDP, not via options
    driver.execute_cdp_cmd("Emulation.setTimezoneOverride",
                           {"timezoneId": tz})
    return driver

The CDP timezone override is the piece people miss. A German IP with a New York timezone is an incoherent profile, and some targets resolve that incoherence in ways that change what you are served — which defeats the point of targeting Germany at all.

Reducing Page Weight

Selenium loads everything a browser loads, which is expensive when you pay per gigabyte. For extraction work that does not need visuals, block what you do not use:

options.add_experimental_option("prefs", {
    "profile.managed_default_content_settings.images": 2,   # block images
})
options.add_argument("--blink-settings=imagesEnabled=false")

For finer control, use CDP to block by resource type:

driver.execute_cdp_cmd("Network.enable", {})
driver.execute_cdp_cmd("Network.setBlockedURLs", {
    "urls": ["*.png", "*.jpg", "*.jpeg", "*.gif", "*.webp",
             "*.woff", "*.woff2", "*.mp4", "*.css"]
})

A heavy product page can drop from a couple of megabytes to under two hundred kilobytes. Across a large crawl that is the difference between a manageable bandwidth bill and an unpleasant one.

Selenium Grid and Remote WebDriver

With Grid, the proxy is configured in the capabilities sent to the hub, and the browser node makes the outbound connection — so the proxy applies from the node's network position, not yours.

from selenium import webdriver
 
options = webdriver.ChromeOptions()
options.add_argument(f"--proxy-server=http://{PROXY_HOST}:{PROXY_PORT}")
 
driver = webdriver.Remote(
    command_executor="http://grid-hub:4444/wd/hub",
    options=options,
)

Two consequences worth planning for.

If you use IP allowlisting, allowlist the nodes, not the hub or your orchestrator. The outbound request originates at the node. This is a very common Grid misconfiguration — everything looks configured correctly and every request returns 407.

Node count caps your concurrency. Grid parallelism is bounded by available browser sessions across nodes, so scaling collection means scaling nodes, which is memory-bound. Browser-based collection is expensive per unit of parallelism; reserve it for targets that genuinely need rendering and use plain HTTP clients everywhere else.

Verify Before You Trust It

Never assume the configuration applied. Check it at driver creation:

import json
 
def verify_driver(driver, want_country, want_city=None):
    driver.get("https://ipcheck.flameproxies.com")
    body = driver.find_element("tag name", "body").text
    data = json.loads(body)
 
    if data.get("country_code") != want_country:
        return False, data
    if want_city and want_city.lower() not in (data.get("city") or "").lower():
        return False, data
    return True, data
 
driver = make_driver(country="DE", city="Berlin")
ok, info = verify_driver(driver, "DE", "Berlin")
if not ok:
    driver.quit()
    raise RuntimeError(f"Driver did not get expected location: {info}")

One lightweight page load per driver. Given that a driver may run twenty or more requests afterward, the overhead is negligible and it prevents an entire batch of wrong-market data.

Debugging When It Fails Silently

Selenium proxy problems tend to fail quietly rather than loudly. A procedure:

1. Is the proxy applying at all? Load ipcheck.flameproxies.com and read the result. If your own ISP appears, the proxy is not applying. For Chrome, check that the flag string is well-formed. For Firefox, check network.proxy.type is 1.

2. HTTP works but HTTPS does not? Firefox with network.proxy.http set but not network.proxy.ssl. Set both.

3. Every request returns 407? Authentication is not reaching the proxy. If allowlisting, confirm the browser node's egress address is listed, not your orchestrator's. If using an extension or selenium-wire, confirm it loaded.

4. Works manually but not in the fleet? Almost always an egress address difference. Print the observed IP from inside the failing environment rather than reasoning about what it should be.

5. Right country, wrong city? Either the provider fell back silently, or your parameter syntax is being accepted but partially ignored. Verify with a plain curl using the identical username string — if curl also shows the wrong city, it is the gateway, not Selenium.

6. Intermittent failures under load? Check whether driver instances are being reused past their intended session budget, and whether you are exhausting a narrow geographic pool at high concurrency.

7. Getting challenge pages that a normal browser does not? This is fingerprint, not proxy. Selenium has detectable automation signals. Check navigator.webdriver, and consider whether this target needs a stealth-configured setup.

When Selenium Is the Wrong Tool

Worth saying plainly: if the target returns usable HTML without JavaScript execution, a plain HTTP client is dramatically cheaper — no browser memory, far less bandwidth, far higher concurrency per machine, and no proxy-auth awkwardness at all.

Reserve Selenium for targets that genuinely require rendering or interaction. Many teams run everything through browsers out of habit and pay for it in both bandwidth and infrastructure.

FlameProxies supports both IP allowlisting and username/password authentication with username-encoded country, city, OS, ASN, and region targeting — so the allowlisting route that sidesteps Chrome's credential limitation is available, and with no concurrency cap your Grid parallelism is bounded by your node capacity rather than by a thread allowance.