Network Observation

🎭 Network Observation in Playwright: Becoming a Traffic Detective

Imagine you’re a detective watching cars pass through a highway checkpoint. You want to see who’s coming, who’s leaving, and sometimes even record everything for later. That’s exactly what Playwright’s Network Observation does for web traffic!

🌟 The Big Picture

When a website loads, it makes many requests (asking for things) and receives responses (getting things back). Playwright lets you:

• Wait for specific requests or responses
• Listen to network events
• Record all traffic into a file (HAR)
• Replay recorded traffic later

Think of it like having a security camera for your website’s conversations!

1️⃣ Waiting for Requests

What is it?

A request is when your browser asks the server: “Hey, can I have this data?”

Sometimes you need to wait until a specific request is made before continuing your test.

🚌 The School Bus Analogy

Imagine waiting at a bus stop. You don’t just jump on any bus—you wait for your specific bus (Bus #42 to school).

// Wait for a request to a specific URL
const requestPromise = page.waitForRequest(
  '**/api/users'
);

// Click button that triggers the request
await page.click('#load-users');

// Now wait for that request
const request = await requestPromise;
console.log(request.url());

🎯 Key Points

• waitForRequest() returns a Promise
• You can match by exact URL or pattern (**/api/*)
• You can also use a function to match:

const req = await page.waitForRequest(
  request => request.url().includes('users')
    && request.method() === 'POST'
);

2️⃣ Waiting for Responses

What is it?

A response is the server saying: “Here’s what you asked for!”

After a request, you often need to wait for the response to arrive.

📬 The Mailbox Analogy

You send a letter (request) and wait by the mailbox for a reply (response). You might wait for any letter, or specifically for one from Grandma!

// Wait for response from API
const responsePromise = page.waitForResponse(
  '**/api/products'
);

await page.click('#fetch-products');

const response = await responsePromise;
console.log(response.status()); // 200
console.log(await response.json());

🔍 Checking Response Details

const response = await page.waitForResponse(
  resp => resp.url().includes('/api/search')
    && resp.status() === 200
);

// Get the data
const data = await response.json();
console.log('Found:', data.results.length);

3️⃣ Network Events

What is it?

Events are like announcements whenever something happens on the network. Instead of waiting for one thing, you can listen to everything!

📻 The Radio Analogy

Network events are like tuning into a radio station. You hear every announcement—not just the ones you were waiting for.

graph TD
    A["🌐 Page"] -->|request| B["📤 Request Event"]
    A -->|response| C["📥 Response Event"]
    A -->|requestfinished| D["✅ Finished Event"]
    A -->|requestfailed| E["❌ Failed Event"]

🎧 Listening to Events

// Log every request
page.on('request', request => {
  console.log('➡️', request.method(),
    request.url());
});

// Log every response
page.on('response', response => {
  console.log('⬅️', response.status(),
    response.url());
});

📋 All Network Events

🛠️ Practical Example

// Track all API calls
const apiCalls = [];

page.on('request', req => {
  if (req.url().includes('/api/')) {
    apiCalls.push({
      url: req.url(),
      method: req.method()
    });
  }
});

await page.goto('https://mysite.com');
console.log('API calls made:', apiCalls);

4️⃣ HAR File Recording

What is it?

HAR (HTTP Archive) is like a video recording of all network traffic. It saves everything into a file you can review later.

📹 The Security Camera Analogy

Recording HAR is like installing a security camera that records ALL traffic. Later, you can rewatch everything that happened!

graph TD
    A["🎬 Start Recording"] --> B["📝 HAR File"]
    C["Request 1"] --> B
    D["Request 2"] --> B
    E["Response 1"] --> B
    F["Response 2"] --> B
    B --> G["🎥 Complete Recording"]

🎬 How to Record

// Create context with HAR recording
const context = await browser.newContext({
  recordHar: {
    path: 'network.har'
  }
});

const page = await context.newPage();
await page.goto('https://example.com');

// Do your stuff...
await page.click('#login');

// IMPORTANT: Close to save the file!
await context.close();

⚙️ Recording Options

const context = await browser.newContext({
  recordHar: {
    path: 'network.har',
    // Only record specific URLs
    urlFilter: '**/api/**',
    // Choose what to record
    content: 'omit' // or 'embed'
  }
});

📁 What’s in a HAR File?

• All URLs requested
• Request headers & body
• Response headers & body
• Timing information
• Status codes

5️⃣ HAR File Replay

What is it?

Replaying a HAR file means using your recorded traffic instead of making real requests. The browser thinks it’s talking to the server, but it’s actually getting pre-recorded answers!

🎭 The Movie Analogy

Instead of watching live TV (real server), you’re watching a recorded movie (HAR file). Same content, but from your recording!

graph TD
    A["🌐 Browser Request"] --> B{HAR Replay Active?}
    B -->|Yes| C["📁 Return from HAR"]
    B -->|No| D["🌍 Real Server"]

▶️ How to Replay

// Route network through HAR file
await page.routeFromHAR('network.har');

// Now navigate - uses recorded data!
await page.goto('https://example.com');

🎛️ Replay Options

await page.routeFromHAR('network.har', {
  // Don't fail if not found in HAR
  notFound: 'fallback',
  // Update HAR with new requests
  update: true,
  // Match specific URLs
  urlFilter: '**/api/**'
});

🏆 Why Use HAR Replay?

🎯 Quick Reference

// 1. Wait for request
const req = await page.waitForRequest('**/api/*');

// 2. Wait for response
const res = await page.waitForResponse('**/api/*');

// 3. Listen to events
page.on('request', r => console.log(r.url()));
page.on('response', r => console.log(r.status()));

// 4. Record HAR
const ctx = await browser.newContext({
  recordHar: { path: 'file.har' }
});

// 5. Replay HAR
await page.routeFromHAR('file.har');

🌈 Putting It All Together

Here’s a complete example using everything we learned:

const { chromium } = require('playwright');

async function networkDetective() {
  const browser = await chromium.launch();

  // Record everything!
  const context = await browser.newContext({
    recordHar: { path: 'session.har' }
  });

  const page = await context.newPage();

  // Listen to all traffic
  page.on('request', req => {
    console.log('📤', req.method(), req.url());
  });

  page.on('response', res => {
    console.log('📥', res.status(), res.url());
  });

  // Wait for specific response
  const dataPromise = page.waitForResponse(
    '**/api/data'
  );

  await page.goto('https://mysite.com');

  const response = await dataPromise;
  console.log('Data:', await response.json());

  // Save HAR file
  await context.close();
  await browser.close();
}

💡 Pro Tips

1. Always close context to save HAR files
2. Use URL patterns (**) for flexible matching
3. Combine events + waits for complex scenarios
4. Record once, replay many for fast tests

🎉 You Did It!

You’re now a Network Observation Pro! You can:

• ✅ Wait for specific requests
• ✅ Wait for specific responses
• ✅ Listen to all network events
• ✅ Record traffic to HAR files
• ✅ Replay HAR files for testing

Go forth and observe those networks like the detective you are! 🕵️