How to archive websites with cookie banners and pop-ups

If you've ever tried to take automated screenshots of European websites, you already know where this is going. Cookie banners. Those consent popups that ask permission to use cookies and block half the page while they're at it.

For a regular visitor, it's a minor annoyance — click "Accept" and move on. But for automated archiving, it's a real problem. You set up monitoring on a competitor's pricing page, the first screenshot comes in, and instead of a pricing table you're looking at a full-screen consent dialog. The second capture — same thing. Third — cookie wall again. Your screenshot archive ends up being a collection of consent dialogs instead of useful business records.

We hit this problem early on while building Snapshot Archive. Our servers run in Helsinki, and European servers trigger GDPR banners on almost every site. We had to figure out reliable ways to handle them — not just for one or two sites, but for any URL a user might add to monitoring. Here's what we learned and what works in practice.

Why cookie banners break automated screenshots in ways you might not expect

The issue goes beyond the banner being annoying. Cookie consent popups behave differently from regular page elements, and that creates problems at several levels.

Most consent banners are overlay elements — they sit on top of the page content, often with a dimmed background behind them. A screenshot captures exactly what a browser would render, which means the banner covers half the page, sometimes all of it. Some sites take it further with a cookie wall (the entire page is locked until the visitor makes a consent choice), where you can't scroll, can't see the content, and the screenshot is completely useless.

But there's a subtler problem that catches people off guard: the page behind the banner may not fully load until consent is given. Some sites defer images, scripts, and dynamic content until the visitor clicks "Accept." Even if you somehow removed the banner from the screenshot with CSS, the page underneath might be half-empty — no hero image, no JavaScript-rendered pricing widget, no dynamically loaded testimonials. This is the difference between a banner that's merely annoying and one that actively blocks content from loading.

Three types of consent popups and how hard each one is to handle

Not all cookie banners work the same way, and the approach for handling them depends on which type you're dealing with.

Most European sites rely on one of the major consent management platforms (CMPs) — Cookiebot, OneTrust, TrustArc, Didomi, or Quantcast. These platforms generate banners from similar templates, which means automated tools can learn to recognize and dismiss them using standard CSS selectors. The tricky cases are custom-built implementations — smaller sites that rolled their own consent solution with unique HTML that doesn't match any known pattern.

Option 1: hide the banner element with CSS

The simplest approach. If you know which HTML element wraps the consent popup, you tell your screenshot tool to set its visibility to hidden before capturing the page. This works well for top/bottom strips that sit on top of otherwise fully-loaded content — the page renders normally behind the banner, and hiding the element produces a clean screenshot.

The catch: if the site uses a cookie wall or delays content loading until consent is given, hiding the element won't help. You'll end up with a screenshot of a blank or partially loaded page, which is arguably worse than a screenshot with a banner on it — at least the banner tells you something went wrong.

In Snapshot Archive, you configure hide selectors per monitor. If a competitor's site uses OneTrust, you'd add #onetrust-consent-sdk to the hide list. We also maintain a built-in list of common CMP selectors that gets applied automatically, so many banners disappear without any manual setup.

Option 2: click the "Accept" button before capturing

For modal overlays and cookie walls, the better approach is to simulate a click on the consent button before taking the screenshot. This tells the site "yes, I accept cookies," dismisses the banner, and lets the page load completely — all content, all scripts, final layout. The page behaves exactly as it would for a real visitor who clicked "Accept."

The challenge is that every site's "Accept" button has a different CSS selector. OneTrust uses #onetrust-accept-btn-handler. Cookiebot uses #CybotCookiebotDialogBodyLevelButtonLevelOptinAllowAll (not exactly memorable). Custom implementations could use anything. We covered how to find the right selector using DevTools — along with real examples from Hetzner, CWSpirits, and Fine Wine & Good Spirits — in our guide to click selectors.

In Snapshot Archive, you set a click selector per monitor and the system clicks that element before every capture. For the most common CMPs, this happens automatically without configuration.

Option 3: set cookies before the page even loads

A less obvious approach, but the cleanest one when it works. Most consent platforms store the visitor's preference in a cookie — if you set that cookie before loading the page, the CMP assumes the visitor already made a choice and skips the banner entirely.

OneTrust checks for a cookie called OptanonAlertBoxClosed. If it exists, the banner never appears. Cookiebot checks for CookieConsent. The result is no banner, no clicking, and the page loads as if a returning visitor opened it. We used this approach to handle age verification on jackdaniels.com, where the gate requires filling in three form fields (day, month, year) that click selector can't handle — setting a single cookie (STYXKEY_AGE_VERIFIED=true) skips the entire form. The full walkthrough is in our article on handling age verification with custom cookies.

The downside is that you need to know the exact cookie name and value for each CMP, and it doesn't work for every implementation. Most automated archiving tools don't support this out of the box, but it's worth knowing about — especially for sites where click selector alone can't solve the problem.

How to handle banners on sites you don't control

Handling cookie banners on your own site is straightforward — you know your CMP, you know the selectors, you can test everything. The real challenge is archiving sites that aren't yours: competitor websites, regulatory pages, client sites you're monitoring for compliance.

Start by taking a test screenshot with no banner handling configured. If a cookie banner covers the content, figure out which CMP the site uses — you can usually tell by inspecting the banner element in DevTools, or by searching the page source for cookiebot, onetrust, trustarc, didomi, or quantcast in the HTML.

Once you've identified the CMP, try the standard selectors for that platform. Here are the ones that typically work for the five most common CMPs:

If the site uses a custom consent implementation, you'll need to find the selector manually. In Chrome, right-click the banner or its button, choose "Inspect," then right-click the highlighted HTML element and select "Copy → Copy selector." That gives you a CSS selector you can paste straight into your monitoring tool's settings.

Real example: removing the cookie banner on theguardian.com

Let's walk through the entire process on a real site. We added theguardian.com to Snapshot Archive specifically to demonstrate the problem and the fix.

Step 1: create a project and add the URL

We created a separate project called "TheGuardian" to keep things organized — optional, but it helps when you're monitoring multiple sites.

We added the URL through "Add URL," entered https://theguardian.com, selected the project, set the frequency and viewport size. No cookie banner settings yet — we wanted to see the problem first.

Step 2: the problem — banner covers everything

The first screenshot came in, and half the screen was covered by The Guardian's "Personalised advertising — it's your choice" banner. The content behind it was dimmed and unreadable — exactly the kind of result that makes your screenshot archive useless.

Without configuring banner handling, every subsequent capture looks the same. The archive fills up with identical screenshots of a consent dialog, and visual diff compares two identical banners and reports 0% change — because technically, nothing did change. The actual page content behind the banner is invisible to both the screenshot and the comparison.

Step 3: find the banner's CSS class in DevTools

The key step: don't rush to close the banner. You need to find its CSS class first.

Open theguardian.com in an incognito window (so there are no saved cookies from previous visits). When the banner appears, right-click on it and choose "Inspect." In DevTools, you'll see the banner's HTML structure. For The Guardian, the banner is wrapped in a div.message-container.gu-overlay element — the class we need is .message-container.

Step 4: add the selector to the monitor settings

Back in Snapshot Archive, we opened the monitor settings (Edit Monitor) and entered .message-container in the "Hide selectors" field. If you need to hide multiple elements (say, a cookie banner and a chat widget), put each selector on a separate line. In our case, a single class was enough.

Step 5: clean screenshots from now on

After saving the settings, the next capture came in clean — no banner, no overlay, just The Guardian's homepage with headlines, navigation, and content fully visible.

The difference is visible in the archive: captures on the right still have the cookie banner, captures on the left (taken after adding the hide selector) are clean. The whole process — from "I see a banner" to "clean screenshot" — took less than a minute.

Edge cases that will trip you up even after setup

Even with banner handling configured, a few situations can still cause trouble.

Geo-targeted banners are the most common surprise. Some sites show different consent popups depending on the visitor's location — if your archiving server sits in Europe but the site targets US visitors, you might see a GDPR banner that American users never encounter. This matters for competitor monitoring, because you want to see what their actual visitors see, not a consent wall triggered by your server's geolocation.

Late-loading banners create a different kind of inconsistency. Some CMPs take a second or two to initialize, so if your screenshot fires before the banner appears, you get a clean shot — but the next capture might show the banner because the timing shifted slightly. This makes visual diff less reliable, because the system alternates between "banner present" and "banner absent" states that have nothing to do with actual content changes. Setting a consistent delay before capture helps, but it's a trade-off with speed.

CMP changes between captures happen more often than you'd expect. A site might switch from Cookiebot to OneTrust, or update their consent copy and class names during a redesign. When your hide/click selector stops matching the new HTML, banner screenshots start appearing again until you notice and update the selector. Worth checking your archive periodically to make sure things are still working — or rely on visual diff to catch it, since a screenshot full of cookie banner looks nothing like a clean page and will trigger an alert automatically.

Multiple popups stacked on top of each other are the most annoying edge case. Some sites layer a cookie banner, a newsletter signup, and a chat widget all at once. Handling just the cookie banner leaves the other overlays in your screenshot. You may need multiple hide selectors — one for the CMP, one for the chat widget (like #intercom-container or .drift-widget-container), and one for marketing popups.

Is it legal to accept cookies on someone else's site?

This question comes up from time to time, so let's address it directly. When your automated tool clicks "Accept" on a cookie banner, it's doing exactly what any visitor would do — accessing a publicly available web page and interacting with a consent mechanism the way it was designed to work.

Accepting cookies on a public page doesn't create legal issues. You're not scraping private data, not bypassing authentication, and not violating access restrictions. The consent banner exists to comply with privacy regulations for the site's visitors, and your screenshot tool is, technically, a visitor. That said, we're not lawyers — if your archiving involves regulated industries or sensitive compliance requirements, check with your legal team. We covered what makes screenshots legally valid in a separate guide, including chain of custody and timestamp requirements.

How Snapshot Archive handles cookie banners

We built cookie banner handling into Snapshot Archive because we needed it ourselves — our screenshot servers run in Helsinki, which means every EU site shows us a consent dialog.

When you add a URL to monitor, our system tries to detect and handle common CMPs automatically. For most sites running OneTrust, Cookiebot, Didomi, or similar platforms, the banner gets dismissed with no configuration on your end. For sites with custom implementations — like The Guardian example above — you can set two fields per monitor: hide_selectors (CSS selectors of elements to hide before capture, one per line) and click_selector (a CSS selector of an element to click before capture). The click happens first, then the hide, then the screenshot — this order matters, because clicking "Accept" loads the full page, and hiding removes any leftover visual clutter like a closing animation that didn't finish in time.

The free plan covers up to 3 monitored URLs, so you can test banner handling on a tricky site before committing. Give it a try — no credit card needed.

Quick checklist before archiving any new URL

Before you start archiving a new site, run through these steps: take a test screenshot to see if a cookie banner appears. If it does, identify the CMP by inspecting the banner element or searching the page source. Try the standard selector from the table above, then check that the page content loads fully after the banner is handled. If other overlays remain — chat widgets, newsletter popups — add their selectors to the hide list too. Review your first few captures to make sure everything looks right.

Once the selectors are set, you shouldn't need to touch them again unless the site changes its CMP. If something breaks, visual diff will catch it — a screenshot full of cookie banner looks nothing like a clean page, and that change gets flagged automatically. Cookie banners are a solvable problem, and they're the kind of thing worth handling upfront rather than discovering six months later that half your archive is consent dialogs.