Skip to content
EN-304-617_v0.0.5.md 1.17 MiB
Newer Older
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
6. Test that technical details are available (certificate viewer) → Warning UI uses appropriate visual indicators (red, warning icons)
7. Verify that bypass actions are clearly labeled with risk warnings → Bypass options are clearly labeled with risk warnings
8. Test mobile and desktop warning UI differences → Error types are distinguishable in UI messages
9. Verify that errors are logged in browser console → Console logs provide technical error details

**Pass Criteria**: All certificate errors display prominent warnings AND bypass requires explicit user acknowledgment of risk

**Fail Criteria**: Certificate errors can be bypassed silently OR warnings are unclear or easily dismissed

**Evidence**: Screenshots of warning UI for each error type, user testing feedback on clarity, console error logs, comparison with browser security UI guidelines

**References**:

- Google Transparency Report - HTTPS Security: https://transparencyreport.google.com/https/overview

### Assessment: ENC-REQ-16 (HTTPS-first mode)

**Reference**: ENC-REQ-16 - Browser shall implement HTTPS-first mode to automatically upgrade HTTP connections to HTTPS when available

**Given**: A conformant browser with encryption capability (ENC-2 or higher)

**Task**: Verify that HTTPS-first mode protects users from accidental insecure connections where attackers perform SSL-stripping attacks to downgrade HTTPS to HTTP, enabling interception of credentials and session tokens. Automatic HTTPS upgrade eliminates the window of vulnerability before secure connections are established, while user warnings for HTTP-only sites ensure informed consent before transmitting data over insecure channels.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Enable HTTPS-first mode in browser settings → Verify browser remembers HTTP-only sites to avoid repeated upgrade attempts
2. Navigate to HTTP URLs of sites that support HTTPS (http://example.com) → HTTP URLs are automatically upgraded to HTTPS
3. Verify that browser automatically upgrades to HTTPS → Address bar shows HTTPS protocol after upgrade
4. Test fallback behavior when HTTPS is unavailable: → Fallback to HTTP works with user consent when HTTPS unavailable
   - Site doesn't support HTTPS
   - HTTPS connection fails/times out
   - Certificate error on HTTPS version
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
5. Verify user is prompted before loading HTTP-only sites → User is warned before loading HTTP-only sites
6. Test HTTPS upgrade for embedded resources (images, scripts, iframes) → Browser remembers HTTP-only sites (cache/allowlist)
7. Test interaction with HSTS and upgrade-insecure-requests → Embedded resources are also upgraded
8. Measure performance impact of HTTPS upgrade attempts → HTTPS-first works alongside HSTS
9. Test HTTPS-first with different types of navigation (typed URL, bookmarks, links) → Network panel shows upgrade attempts

**Pass Criteria**: HTTP connections are automatically upgraded to HTTPS when available AND users are warned before HTTP-only sites load

**Fail Criteria**: HTTP URLs are not upgraded OR no warnings for HTTP-only sites OR fallback doesn't work

**Evidence**: Network logs showing HTTP to HTTPS upgrades, address bar screenshots, warning dialog screenshots, performance measurements, browser settings showing HTTPS-first configuration

**References**:

- Chrome HTTPS-First Mode: https://blog.chromium.org/2021/07/increasing-https-adoption.html
- Firefox HTTPS-Only Mode: https://support.mozilla.org/en-US/kb/https-only-prefs
- HTTPS Upgrade Mechanisms: https://www.w3.org/TR/upgrade-insecure-requests/
- EFF HTTPS Everywhere: https://www.eff.org/https-everywhere
- OWASP Transport Layer Protection: https://cheatsheetseries.owasp.org/cheatsheets/Transport_Layer_Protection_Cheat_Sheet.html

### Assessment: ENC-REQ-17 (Certificate pinning bypass detection)

**Reference**: ENC-REQ-17 - Browser shall detect and prevent attempts to bypass certificate pinning protections

**Given**: A conformant browser with encryption capability (ENC-2 or higher)

**Task**: Verify that certificate pinning bypass detection prevents attackers or malware from installing rogue root certificates to perform man-in-the-middle attacks against pinned domains. Detecting bypass attempts protects browser vendor properties and high-security sites from SSL interception, while logging and user warnings ensure transparency when certificate validation is weakened by enterprise policies or security tools.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Configure test environment with certificate pinning enabled → Static pins for built-in domains cannot be bypassed
2. Attempt various bypass techniques: → Custom root CA installation triggers user warnings
   - Installing custom root CA certificates
   - Using SSL/TLS interception proxies (corporate MITM)
   - Modifying browser certificate store
   - Using browser extensions to disable pinning
   - Command-line flags to disable certificate validation
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
3. Test browser's built-in static pins (Google, Mozilla properties) → SSL interception is detected and indicated in UI
4. Verify that pin bypass attempts are detected and logged → Browser logs pin bypass attempts
5. Test enterprise policy controls for pinning exceptions → Certificate store modifications are visible to users
6. Verify user notifications for certificate store modifications → Enterprise policies can override pins with explicit configuration
7. Test that developer tools can't silently bypass pinning → Developer tools respect pinning (or show clear bypass warnings)
8. Examine browser internal state for pin enforcement → Security indicators reflect weakened security when pins bypassed
9. Test interaction between pin bypass and security indicators → Console logs show certificate validation details

**Pass Criteria**: Static certificate pins cannot be bypassed without explicit user/admin action AND pin bypass attempts are logged and indicated

**Fail Criteria**: Pins can be silently bypassed OR no indication when certificate validation is weakened

**Evidence**: Console logs showing pin enforcement, certificate store modification warnings, test results from bypass attempts, enterprise policy documentation, security indicator screenshots

**References**:

- Chrome Certificate Pinning: https://www.chromium.org/Home/chromium-security/education/tls/
- OWASP Certificate Pinning: https://owasp.org/www-community/controls/Certificate_and_Public_Key_Pinning
- SSL Interception Detection: https://blog.mozilla.org/security/2015/04/30/deprecating-non-secure-http/
- Mozilla Pin Override: https://wiki.mozilla.org/SecurityEngineering/Public_Key_Pinning

### Assessment: ENC-REQ-18 (TLS downgrade protection)

**Reference**: ENC-REQ-18 - Browser shall implement protections against TLS version and cipher suite downgrade attacks

**Given**: A conformant browser with encryption capability (ENC-1 or higher)

**Task**: Verify that TLS downgrade protection prevents man-in-the-middle attackers from forcing browsers to use older TLS versions or weaker cipher suites with known vulnerabilities like POODLE or BEAST. Downgrade protection ensures that even when attackers intercept and modify handshake messages, the browser detects the manipulation and aborts the connection rather than proceeding with weakened cryptographic parameters.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Set up test environment capable of simulating man-in-the-middle attacks → Browser signals maximum supported TLS version correctly
2. Configure server supporting TLS 1.3 and TLS 1.2 → TLS_FALLBACK_SCSV is included in fallback connections
3. Attempt to force downgrade from TLS 1.3 to TLS 1.2 by manipulating ClientHello → Version rollback attacks are detected and connection aborted
4. Test TLS_FALLBACK_SCSV signaling value (RFC 7507) → Cipher suite downgrade attempts trigger handshake failure
5. Attempt downgrade attacks during connection: → Browser validates ServerHello.random for downgrade sentinels
   - Version rollback to older TLS versions
   - Cipher suite downgrade to weaker algorithms
   - Extension stripping attacks
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
6. Verify browser detects and rejects downgrade attempts → Extension stripping is detected through transcript hash validation
7. Test that Finished message MAC includes all handshake messages → Finished message properly authenticates handshake
8. Verify TLS 1.3 downgrade protection sentinel values in ServerHello.random → Console shows error messages for detected downgrade attempts
9. Test protection against truncation attacks → Connection fails securely rather than completing with weakened security

**Pass Criteria**: All TLS downgrade attempts are detected AND connections fail rather than proceed with weakened security

**Fail Criteria**: Any downgrade attack succeeds OR browser accepts weakened connection parameters

**Evidence**: Packet captures showing downgrade attempts and rejection, Wireshark showing TLS_FALLBACK_SCSV, console error logs, test scripts demonstrating attack attempts

**References**:

- RFC 7507 - TLS Fallback SCSV: https://datatracker.ietf.org/doc/html/rfc7507
- RFC 8446 - TLS 1.3 Downgrade Protection: https://datatracker.ietf.org/doc/html/rfc8446#section-4.1.3
- POODLE Attack and Downgrade Prevention: https://www.openssl.org/~bodo/ssl-poodle.pdf
- Chrome TLS Implementation: https://www.chromium.org/Home/chromium-security/education/tls/

### Assessment: ENC-REQ-19 (Legacy crypto deprecation)

**Reference**: ENC-REQ-19 - Browser shall deprecate and remove support for legacy cryptographic algorithms and protocols

**Given**: A conformant browser with encryption capability (ENC-1 or higher)

**Task**: Verify that legacy crypto deprecation prevents attackers from exploiting known cryptographic weaknesses in outdated algorithms like SHA-1 collision attacks, RC4 biases, or short RSA key factorization. Progressive deprecation with clear timelines gives organizations migration paths while ensuring browsers eventually reject severely compromised cryptography that no longer provides meaningful security guarantees.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Test browser behavior with legacy cryptographic elements: → TLS 1.0 and 1.1 connections are rejected or show warnings
   - TLS 1.0 and TLS 1.1 protocols
   - SHA-1 certificates
   - 1024-bit RSA keys
   - MD5-based signatures
   - RC4 cipher suite
   - CBC-mode cipher suites
   - DSA certificates
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
2. Verify that legacy protocols/algorithms are rejected or trigger warnings → SHA-1 certificates trigger security errors
3. Test deprecation timeline (when were features removed) → 1024-bit RSA keys are rejected
4. Verify that browser update notes document deprecated features → MD5 and RC4 are completely disabled
5. Test enterprise policy overrides for legacy support (temporary exceptions) → Legacy crypto rejections show clear error messages
6. Check browser developer documentation for deprecation roadmap → Browser documentation lists deprecated features with timelines
7. Test fallback behavior when modern crypto unavailable → Enterprise policies can temporarily enable legacy support (if necessary)
8. Verify that critical errors can't be bypassed for severely deprecated crypto → No silent fallback to insecure legacy protocols

**Pass Criteria**: All severely deprecated cryptographic elements are rejected AND users are warned about moderately deprecated features

**Fail Criteria**: Legacy crypto is accepted without warnings OR deprecated features work without indication

**Evidence**: Connection error screenshots for legacy servers, browser release notes documenting deprecations, console error logs, test results across browser versions showing deprecation timeline

**References**:

- Chrome Deprecation Timeline: https://www.chromium.org/Home/chromium-security/education/tls/
- Mozilla Security Roadmap: https://wiki.mozilla.org/Security/Server_Side_TLS
- RFC 8996 - Deprecating TLS 1.0 and TLS 1.1: https://datatracker.ietf.org/doc/html/rfc8996
- CA/B Forum - SHA-1 Deprecation: https://cabforum.org/2014/10/16/ballot-118-sha-1-sunset/
- NIST Cryptographic Algorithm Deprecation: https://csrc.nist.gov/projects/hash-functions

### Assessment: ENC-REQ-20 (Cryptographic key isolation)

**Reference**: ENC-REQ-20 - Browser shall isolate cryptographic keys and prevent cross-origin key access

**Given**: A conformant browser with encryption capability (ENC-2 or higher)

**Task**: Verify that cryptographic key isolation enforces same-origin policy for Web Crypto API keys, preventing malicious cross-origin scripts from accessing or exfiltrating cryptographic keys generated by other origins. Key isolation ensures that even if an attacker compromises one origin, they cannot steal cryptographic keys belonging to other origins to impersonate users or decrypt sensitive data protected by those keys.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Create test scenarios with Web Crypto API key generation on different origins → Test key isolation for different user profiles/contexts
2. Generate cryptographic keys on https://origin-a.com using Web Crypto API → Keys generated on one origin cannot be accessed from another origin
3. Attempt to access keys from https://origin-b.com → IndexedDB key storage respects same-origin policy
4. Test IndexedDB key storage isolation across origins → Non-extractable keys cannot be exported or extracted
5. Verify that keys marked as non-extractable cannot be extracted → Key usage restrictions are enforced (keys can't be used for unauthorized operations)
6. Test key export restrictions based on key usage → Key handles are opaque and cannot be transferred cross-origin
7. Test that key handles cannot be passed between origins via postMessage → Browser internal key storage is isolated per origin
8. Verify key isolation in browser's internal key storage → Hardware-backed keys are protected by platform security
9. Test hardware-backed key storage (if available, e.g., WebAuthn) → Different user profiles have separate key storage

**Pass Criteria**: Cryptographic keys are strictly isolated by origin AND non-extractable keys cannot be exported

**Fail Criteria**: Keys can be accessed across origins OR key usage restrictions can be bypassed

**Evidence**: Console logs showing SecurityError for cross-origin key access, test code demonstrating isolation, browser internal state showing key storage separation, WebAuthn test results

**References**:

- W3C Web Cryptography API - Key Storage: https://www.w3.org/TR/WebCryptoAPI/#concepts-key-storage
- MDN CryptoKey: https://developer.mozilla.org/en-US/docs/Web/API/CryptoKey
- Chrome Web Crypto Key Isolation: https://chromium.googlesource.com/chromium/src/+/master/components/webcrypto/README.md
- OWASP Key Management Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Key_Management_Cheat_Sheet.html
- WebAuthn Specification - Credential Storage: https://www.w3.org/TR/webauthn-2/#credential-storage

### Assessment: ENC-REQ-21 (Certificate store security)

**Reference**: ENC-REQ-21 - Browser shall maintain secure certificate store with integrity protection and auditing

**Given**: A conformant browser with encryption capability (ENC-1 or higher)

**Task**: Verify that certificate store security prevents attackers from installing rogue root certificates to enable man-in-the-middle attacks against all TLS connections. Protected certificate stores with integrity checking and audit trails detect unauthorized modifications, while requiring explicit user consent for root CA changes prevents malware from silently compromising the foundation of TLS trust.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Examine browser's certificate store location and access controls → Certificate store files have restrictive permissions (not world-readable)
2. Test that certificate store is protected with appropriate file system permissions → Certificate store cannot be modified without appropriate privileges
3. Attempt to modify certificate store while browser is running → Browser detects and handles certificate store corruption
4. Verify that certificate store changes are logged/auditable → Root CA additions/removals require explicit user consent
5. Test root CA certificate management: → Certificate store UI shows all installed root certificates
   - View installed root certificates
   - Add custom root CA (with user consent)
   - Remove root CA (with user consent)
   - Verify certificate trust settings
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
6. Test that system certificate store is used appropriately → System certificate store integration works correctly
7. Verify that enterprise-managed certificates are clearly indicated → Enterprise-managed certificates are visibly marked
8. Test certificate store integrity verification mechanisms → Certificate store modifications are logged
9. Test that certificate store updates are secure and authenticated → Users are notified of certificate store changes
10. Verify user notifications for certificate store modifications → Certificate store updates are signed and verified

**Pass Criteria**: Certificate store is protected with appropriate access controls AND modifications require user consent and are logged

**Fail Criteria**: Certificate store can be modified without user knowledge OR no audit trail for modifications

**Evidence**: File system permission analysis, certificate store UI screenshots, audit log samples, test results from modification attempts, enterprise policy documentation

**References**:

- Mozilla Root Store Program: https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/
- Chrome Root Store: https://chromium.googlesource.com/chromium/src/+/main/net/data/ssl/chrome_root_store/
- Microsoft Trusted Root Program: https://learn.microsoft.com/en-us/security/trusted-root/program-requirements
- Apple Root Certificate Program: https://www.apple.com/certificateauthority/ca_program.html
- OWASP Certificate Management: https://cheatsheetseries.owasp.org/cheatsheets/Transport_Layer_Protection_Cheat_Sheet.html#certificate-and-public-key-pinning

### Assessment: ENC-REQ-22 (Certificate pinning override prohibition)

**Reference**: ENC-0-REQ-22 - Browser shall not allow user override of certificate pinning failures

**Given**: A conformant browser with ENC-0 capability (full encryption with HSM support)

**Task**: Verify that the browser prohibits user override of certificate pinning failures to prevent users from being socially engineered into accepting fraudulent certificates, ensuring that pinning violations - which indicate active man-in-the-middle attacks or certificate misissuance - are treated as hard failures that cannot be bypassed, protecting high-security environments where certificate pinning is a critical defense against targeted attacks on specific domains.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Configure test server with certificate pinning via HTTP header or browser built-in pins → Test that advanced settings do not provide override mechanism
2. Establish baseline connection to verify pinning is active → Verify that enterprise policies cannot disable pinning for pinned domains
3. Modify server certificate to trigger pinning violation: → Test that developer tools cannot bypass pinning enforcement
   - Use different certificate authority
   - Use expired pin
   - Use self-signed certificate
   - Use certificate from wrong domain
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
4. Attempt to connect to server with pinning violation → Verify that command-line flags cannot disable pinning
5. Verify that browser blocks connection with clear error message → Test that error message explains pinning failure clearly
6. Verify that no "proceed anyway" or "add exception" option is available → Verify that pinning failures are logged for security monitoring

**Pass Criteria**: Pinning failures cannot be overridden through any mechanism AND no bypass options exist in UI OR settings AND error messages explain pinning violations AND failures are logged

**Fail Criteria**: Users can bypass pinning failures OR override mechanisms exist OR pinning can be disabled for pinned domains OR unclear error messages

**Evidence**: Pinning violation error screenshots showing no bypass option, settings inspection showing no override controls, policy test results, developer tools verification, log entries showing pinning failures

**References**:

- HTTP Public Key Pinning: https://www.rfc-editor.org/rfc/rfc7469
- Certificate Pinning Best Practices: https://owasp.org/www-community/controls/Certificate_and_Public_Key_Pinning
- CWE-295: Improper Certificate Validation: https://cwe.mitre.org/data/definitions/295.html

### Assessment: ENC-REQ-23 (Hardware-backed cryptographic operations)

**Reference**: ENC-0-REQ-23 - All cryptographic operations shall be performed in hardware-backed secure enclaves when available

**Given**: A conformant browser with ENC-0 capability on hardware with secure enclave support

**Task**: Verify that all cryptographic operations utilize hardware-backed secure enclaves (such as TPM, Secure Enclave, or TEE) when available, ensuring that private keys and sensitive cryptographic material never exist in unencrypted form in main memory, protecting against memory dumps, cold boot attacks, and malware that attempts to extract cryptographic keys from software-based key storage, providing defense-in-depth through hardware isolation.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Identify available hardware security features on test platform: → Verify that cryptographic operations occur within secure enclave
   - TPM (Trusted Platform Module) on Windows/Linux
   - Secure Enclave on macOS/iOS
   - TEE (Trusted Execution Environment) on Android
   - Hardware security modules (HSM)
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
2. Install browser on platform with hardware security support → Test that key material is not accessible via memory dumps
3. Verify that browser detects and initializes hardware security features → Verify that browser preferences or about page indicates hardware security status
4. Perform cryptographic operations requiring key generation or usage: → Test that browser falls back gracefully if hardware security is unavailable
   - TLS client certificates
   - Web Crypto API key generation
   - Password manager encryption keys
   - Browser sync encryption keys
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
5. Verify that keys are generated and stored in hardware-backed storage → Verify that enterprise policies can require hardware-backed crypto
6. Test that private keys cannot be exported from hardware storage → Test that hardware security failures are logged and reported to users

**Pass Criteria**: Cryptographic operations use hardware-backed storage when available AND private keys are non-exportable AND key material is protected from memory access AND hardware security status is visible to users

**Fail Criteria**: Keys are stored in software despite hardware availability OR keys can be exported OR no hardware security detection OR silent fallback without user notification

**Evidence**: Platform security feature verification, key storage location inspection, memory dump analysis showing no key exposure, browser status page screenshots, enterprise policy configuration tests

**References**:

- Trusted Platform Module: https://trustedcomputinggroup.org/resource/tpm-library-specification/
- Web Crypto API Key Storage: https://www.w3.org/TR/WebCryptoAPI/#concepts-key-storage
- Hardware Security Modules: https://csrc.nist.gov/glossary/term/hardware_security_module

### Assessment: ENC-REQ-24 (Encryption strictness configuration)

**Reference**: ENC-2-REQ-17 - Users shall be able to configure encryption strictness levels

**Given**: A conformant browser with ENC-2 capability (selective encryption)

**Task**: Verify that users can configure encryption strictness levels to balance security requirements with compatibility needs, enabling organizations and advanced users to enforce stricter encryption policies than defaults (such as TLS 1.3-only, strong cipher suites only, or mandatory HSTS) or relax policies for specific legitimate scenarios (such as internal legacy systems), providing flexibility while maintaining security visibility and audit trails.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Access browser encryption settings or configuration interface → Verify that enterprise policies can override user strictness settings
2. Identify available encryption strictness options: → Test that per-site exceptions can be configured for compatibility
   - Minimum TLS version selection (TLS 1.2, TLS 1.3 only)
   - Cipher suite policy (strong only, compatible, allow weak)
   - Mixed content policy (block all, block active only, allow)
   - Certificate validation strictness (strict, standard, permissive)
   - HSTS enforcement level
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
3. Test configuring each strictness level and verify enforcement: → Verify that strictness level changes are logged
   - Set TLS 1.3-only mode and verify TLS 1.2 connections fail
   - Configure strong cipher suites only and verify weak cipher rejection
   - Enable strict mixed content blocking and verify all mixed content blocked
   - Set strict certificate validation and verify self-signed cert rejection
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
4. Verify that strictness changes take effect immediately or after restart → Test that current strictness level is visible in security indicators
5. Test that configuration changes are persistent across browser sessions → Verify that documentation explains security implications of each level

**Pass Criteria**: Multiple encryption strictness levels are available AND configuration changes are enforced AND settings persist AND per-site exceptions are possible AND changes are logged

**Fail Criteria**: No strictness configuration available OR changes not enforced OR settings don't persist OR no audit trail OR no security baseline

**Evidence**: Settings UI screenshots showing strictness options, connection test results at different levels, persistence verification, per-site exception demonstrations, audit logs

**References**:

- TLS Configuration Recommendations: https://wiki.mozilla.org/Security/Server_Side_TLS
- OWASP Transport Layer Protection: https://cheatsheetseries.owasp.org/cheatsheets/Transport_Layer_Protection_Cheat_Sheet.html

### Assessment: ENC-REQ-25 (Connection security visual indicators)

**Reference**: ENC-2-REQ-18 - Browser shall provide visual indicators for connection security status

**Given**: A conformant browser with ENC-2 capability (selective encryption)

**Task**: Verify that the browser provides clear, prominent visual indicators for connection security status, enabling users to quickly identify secure HTTPS connections, insecure HTTP connections, and connection security issues, preventing users from entering sensitive information on insecure sites and helping users recognize potential man-in-the-middle attacks or certificate problems through consistent, understandable security indicators in the browser UI.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Navigate to various sites with different security states: → Verify that security indicator changes are immediate upon navigation
   - Valid HTTPS with strong encryption (https://example.com with TLS 1.3)
   - Valid HTTPS with weak encryption (https://site.com with TLS 1.2, weak cipher)
   - HTTP site (http://insecure.com)
   - HTTPS with certificate error (expired, self-signed, wrong domain)
   - HTTPS with mixed content (active and passive)
   - HTTPS with Extended Validation certificate
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
2. Verify that each security state has distinct visual indicator: → Test that security indicators are accessible (screen reader compatible)
   - Padlock icon or security indicator in address bar
   - Color coding (green for secure, red for insecure, warning for issues)
   - Text labels ("Secure", "Not Secure", "Connection is not secure")
   - Icon states (locked, unlocked, warning symbol)
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
3. Test that clicking security indicator shows detailed connection information: → Verify that indicators follow consistent design across all platforms
   - TLS version and cipher suite
   - Certificate details and validity
   - Mixed content warnings
   - Permission grants
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
4. Verify that indicators are visible in all browser UI states (fullscreen, focus mode, etc.) → Test that users can easily distinguish secure from insecure connections
5. Test that indicators cannot be spoofed by web content → Verify that connection security details are easy to access

**Pass Criteria**: Distinct visual indicators for each security state AND indicators are prominent and clear AND detailed information is accessible AND indicators cannot be spoofed AND consistent across platforms

**Fail Criteria**: Security indicators are unclear OR states are not visually distinct OR detailed information not available OR indicators can be spoofed OR inconsistent across platforms

**Evidence**: Screenshots of security indicators for each connection state, detailed security info panel screenshots, accessibility testing results, spoofing attempt tests, cross-platform consistency verification

**References**:

- Browser Security Indicators: https://www.w3.org/TR/design-principles/#priority-of-constituencies
- User Security Awareness: https://www.usenix.org/conference/soups2016/technical-sessions/presentation/felt

### Assessment: ENC-REQ-26 (User encryption settings control)

**Reference**: ENC-3-REQ-11 - Users shall have full control over encryption settings

**Given**: A conformant browser with ENC-3 capability (basic encryption with optional enhanced protection)

**Task**: Verify that users have full control over all encryption settings, enabling advanced users and specific deployment scenarios to configure encryption behavior according to their unique requirements, such as enabling legacy protocols for compatibility with older systems, configuring custom cipher suite preferences, or managing certificate exceptions, while ensuring that users are informed of the security implications of relaxing encryption requirements.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Access browser advanced encryption settings → Test that dangerous configurations trigger prominent warnings
2. Verify that comprehensive encryption controls are available: → Verify that encryption settings can be exported/imported for consistency
   - TLS version selection (TLS 1.0, 1.1, 1.2, 1.3)
   - Individual cipher suite enable/disable
   - Certificate validation options
   - Mixed content policy configuration
   - HSTS bypass capabilities
   - Certificate exception management
   - Revocation checking options
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
3. Test modifying each encryption setting: → Test that enterprise policies can restrict user control over specific settings
   - Enable legacy TLS 1.0/1.1 and verify older servers accessible
   - Disable specific cipher suites and verify negotiation changes
   - Relax certificate validation and verify self-signed cert acceptance
   - Allow mixed content and verify passive content loads
   - Add certificate exceptions and verify accepted
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
4. Verify that each setting change shows security warning explaining risks → Verify that reset-to-defaults option exists for safe rollback
5. Test that changes require user acknowledgment of risks → Test that encryption setting changes are logged
6. Verify that current encryption configuration is clearly documented in UI → Verify that help documentation explains each setting clearly

**Pass Criteria**: All encryption settings are user-configurable AND security warnings accompany risky changes AND current configuration is visible AND reset option exists AND changes are logged

**Fail Criteria**: Encryption settings are not configurable OR no security warnings for risky changes OR configuration not visible OR no rollback option OR no audit trail

**Evidence**: Advanced settings UI screenshots, configuration change workflows, security warning examples, setting export/import tests, policy restriction demonstrations, audit logs

**References**:

- TLS Configuration: https://wiki.mozilla.org/Security/Server_Side_TLS
- User Control Principles: https://www.w3.org/TR/design-principles/#user-control

### Assessment: ENC-REQ-27 (Encryption status UI indicators)

**Reference**: ENC-3-REQ-12 - Browser shall provide encryption status indicators in UI

**Given**: A conformant browser with ENC-3 capability (basic encryption with optional enhanced protection)

**Task**: Verify that the browser provides comprehensive encryption status indicators throughout the user interface, enabling users to understand the current encryption configuration and connection security state at any time, helping users make informed decisions about whether their current settings are appropriate for their security needs and alerting them to potential security issues that may result from configuration changes.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Access browser with various encryption configurations active → Verify that weak encryption configurations trigger visual warnings
2. Verify that encryption status is visible in multiple UI locations: → Test that encryption status persists in browser session info
   - Address bar security indicator
   - Settings page encryption section
   - About/information pages
   - Developer tools security panel
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
3. Test that status indicators show: → Verify that status indicators are accessible via keyboard navigation
   - Current TLS version in use
   - Active cipher suite
   - Certificate validity status
   - Enabled/disabled encryption features (HSTS, CT, OCSP)
   - Legacy protocol status (if enabled)
   - Number of certificate exceptions
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
4. Navigate to sites with different security configurations → Test that indicators work in all browsing modes (normal, private, reader)
5. Verify that indicators update dynamically to reflect current connection → Verify that encryption dashboard or summary page exists
6. Test that clicking indicators provides detailed encryption information → Test that status information is exportable for security audits

**Pass Criteria**: Encryption status visible in multiple UI locations AND indicators show detailed configuration AND indicators update dynamically AND weak configurations trigger warnings AND status is exportable

**Fail Criteria**: Encryption status not visible OR indicators lack detail OR no dynamic updates OR no warnings for weak config OR status not exportable

**Evidence**: UI screenshots from various locations showing status, status detail panel captures, dynamic update demonstrations, weak config warning examples, audit export samples

**References**:

- Security Indicators Design: https://www.usenix.org/conference/soups2016/technical-sessions/presentation/felt
- Transparency in Browser Security: https://www.w3.org/TR/design-principles/#transparency

### Assessment: ENC-REQ-28 (Legacy protocol compatibility mode)

**Reference**: ENC-3-REQ-13 - Legacy protocol support may be enabled for compatibility with user consent

**Given**: A conformant browser with ENC-3 capability (basic encryption with optional enhanced protection)

**Task**: Verify that legacy protocol support (SSL 3.0, TLS 1.0, TLS 1.1) can be enabled for compatibility with older systems when necessary, but requires explicit, informed user consent with clear security warnings, ensuring users understand the risks of downgrading to deprecated protocols while enabling access to legacy systems that cannot be immediately upgraded, such as embedded devices, industrial control systems, or archived services.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Verify that legacy protocols are disabled by default → Verify that legacy protocol enablement is persistent across sessions
2. Attempt to connect to server requiring legacy protocol (TLS 1.0 or TLS 1.1) → Test that legacy protocol usage triggers visual indicator in address bar
3. Verify that connection fails with clear error message explaining protocol version mismatch → Verify that connections using legacy protocols show prominent warning
4. Access browser settings to enable legacy protocol support → Test that legacy protocol can be enabled for specific sites only (not globally)
5. Verify that enabling legacy protocols requires: → Verify that legacy protocol enablement is logged for security auditing
   - Navigation to advanced settings section
   - Explicit checkbox or toggle action
   - Acknowledgment of security warning dialog
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
6. Test that security warning clearly explains: → Test that reset to defaults disables legacy protocols
   - Legacy protocols have known vulnerabilities
   - Connections may be susceptible to downgrade attacks
   - Sensitive information should not be transmitted
   - Recommendation to upgrade server instead

**Pass Criteria**: Legacy protocols disabled by default AND enablement requires explicit consent AND clear security warnings shown AND usage is indicated visually AND site-specific enablement possible AND changes logged

**Fail Criteria**: Legacy protocols enabled by default OR no consent required OR warnings unclear OR no visual indicators OR only global enablement OR no audit trail

**Evidence**: Default protocol test results, enablement workflow screenshots, security warning content, visual indicator examples, site-specific configuration tests, audit log samples

**References**:

- Deprecating Legacy TLS: https://datatracker.ietf.org/doc/rfc8996/
- TLS 1.0 and 1.1 Deprecation: https://www.rfc-editor.org/info/rfc8996

### Assessment: ENC-REQ-29 (Complete encryption policy control)

**Reference**: ENC-4-REQ-6 - Users shall have complete control over all encryption policies

**Given**: A conformant browser with ENC-4 capability (minimal encryption)

**Task**: Verify that users have unrestricted control over all encryption policies without artificial limitations, enabling specialized use cases such as security research, penetration testing, legacy system maintenance, or air-gapped environments where standard encryption enforcement may need to be completely disabled, while ensuring that the browser clearly indicates when insecure configurations are active and the risks associated with completely disabling encryption protections.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Access all available encryption policy controls in browser → Test that insecure configuration status is highly visible in all UI
2. Verify that users can modify every encryption-related setting: → Verify that browser warns before entering sensitive input on insecure connections
   - Completely disable TLS/SSL enforcement
   - Allow HTTP for all connections
   - Disable all certificate validation
   - Accept any cipher suite including export-grade
   - Disable HSTS, OCSP, CT, and all security features
   - Allow unlimited certificate exceptions
   - Disable mixed content blocking entirely
   - Remove all protocol version restrictions
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
3. Test disabling each encryption protection individually → Test that configuration can be saved and restored
4. Verify that disabled protections do not activate even in dangerous scenarios → Verify that no enterprise policies can override user's complete control
5. Test that completely insecure configuration allows access to test servers: → Test that security researchers can use browser for SSL/TLS testing
   - HTTP-only servers
   - Self-signed certificate servers
   - Expired certificate servers
   - Weak cipher servers (3DES, RC4)
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
6. Verify that browser displays persistent warning when encryption is disabled → Verify that documentation explains appropriate use cases for insecure configs

**Pass Criteria**: All encryption policies are user-modifiable without restriction AND disabled protections do not activate AND insecure status is highly visible AND configuration is persistent AND appropriate for specialized use cases

**Fail Criteria**: Some encryption policies cannot be disabled OR disabled protections still activate OR insecure status not visible OR configuration doesn't persist OR inappropriate for intended use cases

**Evidence**: Complete settings access screenshots, disabled protection test results, insecure configuration visual indicators, specialized use case demonstrations, security researcher testimonials

**References**:

- Security Research Ethics: https://www.w3.org/Security/wiki/Improving_Web_Security
- Penetration Testing Requirements: https://owasp.org/www-community/Penetration_testing

### Assessment: ENC-REQ-30 (User-enabled legacy protocol support)

**Reference**: ENC-4-REQ-7 - Browser shall support legacy protocols when explicitly enabled by user

**Given**: A conformant browser with ENC-4 capability (minimal encryption)

**Task**: Verify that browser supports legacy cryptographic protocols (SSL 3.0, TLS 1.0, TLS 1.1) when explicitly enabled by user, ensuring that legacy systems, embedded devices, or historical services that cannot be upgraded remain accessible through modern browsers in specialized scenarios, while making it clear that these protocols have known security vulnerabilities and should only be used when absolutely necessary with full understanding of the risks.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Access browser protocol version settings → Test that protocol negotiation prefers newest enabled version
2. Verify that legacy protocol options are available: → Verify that legacy protocol connections show security warnings
   - SSL 3.0 enable/disable
   - TLS 1.0 enable/disable
   - TLS 1.1 enable/disable
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
3. Enable each legacy protocol individually → Test that mixed protocol versions work correctly (e.g., TLS 1.0 and 1.3 both enabled)
4. Test connection to servers requiring each legacy protocol: → Verify that legacy protocol usage is logged
   - SSL 3.0-only server (if available for testing)
   - TLS 1.0-only server
   - TLS 1.1-only server
Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
5. Verify that connections succeed when protocol is enabled → Test that browser identifies which protocol version was used for each connection
6. Verify that connections fail when protocol is disabled → Verify that legacy protocol support can be toggled without browser restart

**Pass Criteria**: All legacy protocols can be enabled by user AND enabled protocols function correctly AND protocol negotiation works properly AND legacy usage is warned and logged AND risks are documented

**Fail Criteria**: Legacy protocols cannot be enabled OR enabled protocols don't work OR protocol negotiation fails OR no warnings/logging OR risks not explained

**Evidence**: Legacy protocol enablement screenshots, successful connection tests to legacy servers, protocol version negotiation logs, security warning examples, risk documentation excerpts

**References**:

- SSL 3.0 Vulnerabilities (POODLE): https://www.rfc-editor.org/rfc/rfc7568
- TLS 1.0 and 1.1 Deprecation: https://datatracker.ietf.org/doc/rfc8996/
- Legacy Protocol Security Risks: https://www.nist.gov/publications/transitioning-use-cryptographic-algorithms-and-key-lengths

## 6.4 Security Event Logging Assessments

This section covers assessment procedures for requirements LOG-REQ-1 through LOG-REQ-20, addressing security event logging, audit trails, privacy-preserving telemetry, log retention, and security monitoring capabilities.

### Assessment: LOG-REQ-1 (Security event logging)

**Reference**: LOG-REQ-1 - Browser shall implement comprehensive security event logging for security-relevant events

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that comprehensive security event logging enables detection and investigation of attacks, providing audit trails for security incidents like XSS attempts, CSP violations, and CORS bypasses. Complete logging with timestamps, origins, and outcomes allows security teams to identify attack patterns, correlate events across multiple targets, and respond to threats effectively while supporting compliance with security monitoring requirements.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Configure the browser to enable security event logging in developer or enterprise mode → Verify that security events include sufficient context for investigation (URL, origin, error type, timestamp)
2. Access the browser's internal logging interfaces (chrome://net-internals/#events, about:networking, or equivalent) → Test that security events persist across browser restarts if configured for persistent logging
3. Navigate to a test page that triggers multiple security events (mixed content, certificate errors, XSS attempts) → All security policy violations are logged with accurate timestamps
4. Attempt cross-origin requests that violate CORS policies → Logs include sufficient context to identify the origin and nature of security events
5. Load test pages with Content Security Policy violations → Security events include: CSP violations, CORS failures, mixed content blocks, certificate errors, sandbox violations
6. Trigger sandbox violations by attempting to access restricted APIs → Logs distinguish between blocked and allowed actions with clear outcomes
7. Review the logged events to verify all security-relevant actions are captured → Event log format is structured and machine-readable (JSON or similar)
8. Export the security event log to verify it contains timestamps, event types, origins, and outcomes → Logs can be exported for analysis or forwarding to external systems

**Pass Criteria**: All tested security events are captured in logs with complete context (timestamp, origin, event type, outcome) AND logs are exportable in a structured format

**Fail Criteria**: Any security event fails to be logged OR logs lack critical context (timestamp, origin, or outcome) OR logs are not exportable

**Evidence**: Screenshots of security event logs showing various event types, exported log files in JSON/structured format, video recordings of security events being triggered and logged, comparison matrices showing event coverage

**References**:

- W3C Reporting API: https://www.w3.org/TR/reporting-1/
- OWASP Logging Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
- NIST SP 800-92 Guide to Computer Security Log Management: https://csrc.nist.gov/publications/detail/sp/800-92/final

### Assessment: LOG-REQ-2 (Certificate error logging)

**Reference**: LOG-REQ-2 - Browser shall log all certificate validation failures with detailed error information

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that certificate error logging enables detection of man-in-the-middle attacks and certificate misconfigurations, providing detailed information for security investigations. Comprehensive certificate logs with chain details, error types, and revocation status help identify patterns of TLS interception, rogue certificates, or systematic validation failures that could indicate ongoing attacks against users or infrastructure.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Set up a test web server with various certificate issues (expired, self-signed, wrong hostname, revoked) → Verify that certificate transparency failures are logged when CT enforcement is enabled
2. Navigate to https://expired.badssl.com/ and verify the certificate error is logged → Test that HSTS violations involving certificates are properly logged
3. Navigate to https://wrong.host.badssl.com/ and verify hostname mismatch is logged → All certificate validation failures are logged with specific error codes
4. Navigate to https://self-signed.badssl.com/ and verify self-signed certificate is logged → Logs include certificate subject, issuer, validity period, and error reason
5. Navigate to https://revoked.badssl.com/ and verify revocation status is logged → Certificate chain information is captured in logs
6. Review the certificate error logs to verify they include: certificate chain, error type, validation date, origin → Revocation check results (OCSP/CRL) are included
7. Test certificate pinning failures by creating a pinning policy and violating it → Certificate pinning violations are logged separately

**Pass Criteria**: All certificate validation failures are logged with complete certificate details AND error reasons are specific and actionable

**Fail Criteria**: Any certificate error is not logged OR logs lack certificate details OR error reasons are generic/unhelpful

**Evidence**: Certificate error log entries showing various failure types, screenshots of BadSSL.com test results with corresponding logs, certificate chain dumps from logs, network traces showing certificate validation process

**References**:

- RFC 5280 X.509 Certificate Validation: https://www.rfc-editor.org/rfc/rfc5280
- Certificate Transparency RFC 6962: https://www.rfc-editor.org/rfc/rfc6962
- Chrome Certificate Error Logging: https://www.chromium.org/Home/chromium-security/certificate-transparency/
- OWASP Transport Layer Protection Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Transport_Layer_Protection_Cheat_Sheet.html

### Assessment: LOG-REQ-3 (Extension security events)

**Reference**: LOG-REQ-3 - Browser shall log extension-related security events including installation, permission changes, and security violations

**Given**: A conformant browser with EXT-1 or higher capability and LOG-1 or higher capability

**Task**: Verify that extension security event logging enables detection of malicious extensions that abuse permissions, access sensitive APIs without authorization, or attempt to bypass security policies. Detailed extension logs with lifecycle events, permission changes, and security violations help identify compromised or rogue extensions that could exfiltrate data, inject malicious scripts, or escalate privileges beyond their declared capabilities.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Install a test extension and verify the installation event is logged with extension ID, name, and permissions → Simulate an extension attempting to bypass CSP and verify the violation is logged
2. Modify extension permissions and verify permission changes are logged → Test extension update events and verify version changes are logged
3. Create a test extension that attempts to access APIs without proper permissions → Extension lifecycle events (install, update, uninstall) are logged with metadata
4. Trigger extension content script injection and verify it's logged → Permission requests and grants are logged with timestamp and user action
5. Test extension network requests to verify they are logged separately from normal browsing → Extension security violations are logged separately from web page violations
6. Uninstall the extension and verify the removal event is logged → Extension API access attempts are logged with success/failure status
7. Test developer mode extension loading and verify it's flagged in logs → Developer mode extensions are clearly marked in logs

**Pass Criteria**: All extension lifecycle and security events are logged with complete metadata AND extension actions are distinguishable from page actions

**Fail Criteria**: Any extension security event is not logged OR extension actions cannot be distinguished from page actions OR permission changes are not logged

**Evidence**: Extension event logs showing lifecycle events, permission change logs, screenshots of extension security violations with corresponding log entries, comparison of extension vs. page event logs

**References**:

- Chrome Extension Security Architecture: https://developer.chrome.com/docs/extensions/mv3/security/
- WebExtensions API Security: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API
- Extension Permission Model: https://www.chromium.org/Home/chromium-security/extension-content-script-fetches/
- OWASP Browser Extension Security: https://owasp.org/www-community/vulnerabilities/Unsafe_Mobile_Code

### Assessment: LOG-REQ-4 (CSP violation reporting)

**Reference**: LOG-REQ-4 - Browser shall implement Content Security Policy violation reporting and logging

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that CSP violation reporting enables detection of XSS attempts and policy bypasses, providing actionable intelligence about attacks targeting web applications. Automated reporting with complete violation context allows security teams to identify attack vectors, adjust CSP policies, and detect systematic attempts to inject malicious scripts or load unauthorized resources that could compromise user data.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Create a test page with a strict CSP policy: `Content-Security-Policy: default-src 'self'; report-uri /csp-report` → Test CSP reporting with report-only mode using Content-Security-Policy-Report-Only header
2. Attempt to load external scripts that violate the CSP policy → Verify that violation reports include: violated-directive, blocked-uri, source-file, line-number
3. Verify CSP violation reports are sent to the report-uri endpoint → All CSP violations trigger console warnings in developer tools
4. Check browser console for CSP violation messages → Violation reports are sent to configured report-uri endpoints
5. Test inline script violations and verify they are reported → Reports include complete context: violated directive, blocked resource, source location
6. Test eval() violations and verify they are blocked and reported → Report-only mode generates reports without blocking resources
7. Configure CSP with report-to directive and Report-To header for modern reporting → Modern Reporting API (report-to) is supported for CSP reporting

**Pass Criteria**: All CSP violations are reported to configured endpoints AND console warnings are displayed AND reports contain complete violation context

**Fail Criteria**: Any CSP violation is not reported OR reports lack critical information OR report-uri/report-to mechanisms don't function

**Evidence**: CSP violation reports in JSON format, server logs showing received reports, browser console screenshots with CSP warnings, network traces showing report transmission, comparison of report-only vs. enforce mode

**References**:

- Content Security Policy Level 3: https://www.w3.org/TR/CSP3/
- CSP Violation Reports: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP#violation_reports
- W3C Reporting API: https://www.w3.org/TR/reporting-1/
- CSP Evaluator Tool: https://csp-evaluator.withgoogle.com/

### Assessment: LOG-REQ-5 (Network Error Logging - NEL)

**Reference**: LOG-REQ-5 - Browser shall support Network Error Logging (NEL) for monitoring network failures

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that Network Error Logging enables detection of network-level attacks and infrastructure issues, monitoring connection failures that could indicate DNS hijacking, TLS interception, or targeted denial of service. NEL provides visibility into network failures that occur before HTTP layer, helping identify systematic attacks, CDN failures, or connectivity issues affecting user security and availability.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Set up a test server that sends NEL policy header: `NEL: {"report_to":"network-errors","max_age":86400,"success_fraction":0.1,"failure_fraction":1.0}` → Verify NEL reports include: type, url, server-ip, protocol, status-code, elapsed-time, phase
2. Configure Report-To header: `Report-To: {"group":"network-errors","max_age":86400,"endpoints":[{"url":"https://collector.example.com/nel"}]}` → NEL policy is correctly parsed from HTTP headers
3. Trigger DNS resolution failures by attempting to navigate to non-existent domains → Network failures trigger NEL report generation
4. Trigger connection timeout errors by connecting to a filtered port → Reports are sent to configured endpoints asynchronously
5. Trigger TLS handshake failures by using misconfigured certificates → Sampling fractions are respected for success/failure events
6. Trigger HTTP protocol errors by sending malformed responses → NEL reports include detailed failure context (phase, type, status)
7. Verify that NEL reports are generated and sent to the configured endpoint → Reports distinguish between DNS, connection, TLS, and HTTP errors
8. Check that success sampling works correctly (success_fraction parameter) → NEL reports are batched and sent efficiently

**Pass Criteria**: NEL policy is respected AND all configured failure types generate reports AND reports include complete network error context

**Fail Criteria**: NEL policy is ignored OR network failures don't generate reports OR reports lack critical failure information

**Evidence**: NEL policy headers from server, collected NEL reports in JSON format, server logs showing received reports, network traces demonstrating various failure types, NEL report timing analysis

**References**:

- Network Error Logging Specification: https://www.w3.org/TR/network-error-logging/
- Reporting API Specification: https://www.w3.org/TR/reporting-1/
- NEL Deployment Guide: https://developers.google.com/web/updates/2018/09/reportingapi
- MDN Network Error Logging: https://developer.mozilla.org/en-US/docs/Web/HTTP/Network_Error_Logging

### Assessment: LOG-REQ-6 (Crash reporting)

**Reference**: LOG-REQ-6 - Browser shall implement privacy-preserving crash reporting with user consent

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that privacy-preserving crash reporting enables browser vendors to identify and fix security vulnerabilities without exposing user browsing history or personal information. User consent requirements ensure transparency while crash report anonymization prevents data leakage, balancing the need for diagnostic information to address exploitable bugs with user privacy rights and regulatory compliance.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Review browser settings for crash reporting consent options → Test opt-out functionality and verify no reports are sent when opted out
2. Enable crash reporting and verify user consent is obtained before activation → Verify renderer process crashes are reported separately from browser process crashes
3. Force a controlled browser crash using debugging tools or crash test pages → User consent is required before crash reporting is enabled
4. Verify a crash report is generated locally → Crash reports are generated for browser and renderer crashes
5. Check that crash reports are anonymized and don't contain browsing history or personal data → Reports include technical diagnostics (stack traces, crash signatures) but no personal data
6. Verify crash reports include: crash signature, stack trace (symbolicated), browser version, OS version → Users can review crash reports before submission
7. Test that crash reports are only uploaded after user consent → Crash reporting can be disabled and re-enabled in settings
8. Verify users can view and delete crash reports before submission → Crash reports are transmitted securely (HTTPS) to vendor endpoints

**Pass Criteria**: User consent is obtained before crash reporting AND crash reports exclude personal data AND users can review/delete reports

**Fail Criteria**: Crash reports are sent without consent OR reports contain personal/browsing data OR users cannot control crash reporting

**Evidence**: Crash report consent dialogs, sanitized crash reports showing included data, settings screenshots showing crash reporting controls, privacy policy documentation, crash report upload network traces

**References**:

- Breakpad Crash Reporting: https://chromium.googlesource.com/breakpad/breakpad/
- Firefox Crash Reporter: https://support.mozilla.org/en-US/kb/mozilla-crash-reporter
- GDPR Crash Reporting Compliance: https://gdpr.eu/what-is-gdpr/

### Assessment: LOG-REQ-7 (Log data minimization)

**Reference**: LOG-REQ-7 - Browser shall minimize data collection in logs, collecting only information necessary for security purposes

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that log data minimization prevents excessive collection of personal information, balancing security monitoring needs with privacy rights and regulatory compliance. Collecting only necessary security-relevant data with automatic retention limits reduces the risk of data breaches exposing user browsing history, credentials, or sensitive personal information while still enabling effective threat detection and incident response.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Review all log categories to identify what data is collected (security events, network, performance, etc.) → Check that telemetry aggregates data rather than logging individual user actions
2. Examine security event logs to verify they don't contain unnecessary personal information → Logs contain only security-relevant events, not general browsing activity
3. Check that URLs in logs are sanitized (query parameters removed or hashed) → Personal identifiable information (PII) is redacted or hashed in logs
4. Verify that user credentials are never logged, even in error conditions → URL parameters that may contain session tokens are removed
5. Test that cookies and authentication tokens are redacted from network logs → Credentials, cookies, and authentication headers are never logged
6. Review crash reports to ensure they exclude browsing history and form data → IP addresses are anonymized or removed from logs
7. Verify that IP addresses in logs are anonymized (last octet removed) or hashed → Logs automatically expire based on retention policies
8. Test that logs have automatic retention limits (time-based and size-based) → Form input data is excluded from all logs
9. Verify that sensitive form fields (passwords, credit cards) are never logged → Aggregated metrics replace individual event logging where possible

**Pass Criteria**: All logs demonstrate data minimization (no unnecessary PII) AND sensitive data is consistently redacted AND retention limits are enforced

**Fail Criteria**: Logs contain unnecessary PII OR credentials/tokens appear in logs OR no retention limits exist

**Evidence**: Log samples showing redaction of sensitive data, privacy analysis of log contents, retention policy documentation, comparison of logged vs. available data showing minimization, code review of logging implementations

**References**:

- OWASP Logging Cheat Sheet - Data Sanitization: https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
- NIST Privacy Framework: https://www.nist.gov/privacy-framework
- W3C Privacy Principles: https://www.w3.org/TR/privacy-principles/

### Assessment: LOG-REQ-8 (Log anonymization)

**Reference**: LOG-REQ-8 - Browser shall implement anonymization techniques for logs that require user-related data

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that log anonymization prevents re-identification of users from telemetry data, protecting user privacy while maintaining the ability to detect security incidents and diagnose technical issues. Proper anonymization defends against correlation attacks where adversaries combine multiple log entries to de-anonymize users, as well as against data breaches where stolen logs could reveal sensitive user information.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Review telemetry logs to identify user-related fields (user ID, device ID, session ID) → Check that pseudonymous identifiers change across different log contexts
2. Verify that user identifiers are hashed with secure cryptographic hash functions → User identifiers are consistently hashed with strong cryptographic algorithms
3. Check that hash salts are rotated periodically to prevent correlation → Hash salts are documented and rotated on a defined schedule
4. Test that IP addresses are anonymized using techniques like IP truncation or hashing → IP addresses are truncated or hashed before storage
5. Verify that timestamps are rounded to reduce precision (hour or day level) where appropriate → Timestamps are appropriately rounded to reduce granularity
6. Test that geographic data is generalized (city level rather than GPS coordinates) → Geographic data is generalized to prevent precise location tracking
7. Review aggregation techniques to ensure k-anonymity (minimum group size) is maintained → Aggregated data maintains k-anonymity with k >= 5
8. Verify differential privacy techniques are applied to statistical queries on logs → Differential privacy noise is added to prevent individual identification
9. Test that user fingerprints cannot be reconstructed from anonymized logs → Cross-log correlation attacks are prevented through identifier rotation

**Pass Criteria**: All user-related data is anonymized using documented techniques AND re-identification is demonstrably prevented AND k-anonymity is maintained

**Fail Criteria**: User data is logged in plaintext OR anonymization is reversible OR re-identification is possible through correlation

**Evidence**: Anonymized log samples with hash values, salt rotation policy documentation, privacy analysis showing k-anonymity, differential privacy parameters, re-identification attack test results (negative results expected)

**References**:

- Differential Privacy: https://www.microsoft.com/en-us/research/publication/differential-privacy/
- K-Anonymity Model: https://epic.org/privacy/reidentification/Sweeney_Article.pdf
- NIST De-Identification Guidelines: https://nvlpubs.nist.gov/nistpubs/ir/2015/NIST.IR.8053.pdf
- Google's Privacy-Preserving Techniques: https://policies.google.com/technologies/anonymization

### Assessment: LOG-REQ-9 (User consent for telemetry)

**Reference**: LOG-REQ-9 - Browser shall obtain explicit user consent before collecting and transmitting telemetry data

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that explicit user consent for telemetry protects user privacy rights and complies with data protection regulations including GDPR and CPRA. Without proper consent mechanisms, browsers may violate privacy laws by collecting personal data without permission, and users are deprived of control over their information. Consent shall be freely given, specific, informed, and revocable to meet legal and ethical standards.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Perform a fresh installation of the browser and observe the first-run experience → Check that consent is re-requested when telemetry data types or purposes change significantly
2. Verify that a clear consent prompt is displayed for telemetry collection → First-run consent prompt is clear, prominent, and explains data collection
3. Check that the consent prompt explains what data is collected and why → Users can freely choose to accept or decline without dark patterns
4. Verify users can decline telemetry without affecting core browser functionality → Declining telemetry doesn't degrade core browser functionality
5. Test that declining telemetry prevents all non-essential data collection → Telemetry settings are easily accessible in preferences/settings
6. Navigate to browser settings and verify telemetry preferences are accessible → Consent choices are persistent and respected across updates
7. Verify users can change their consent choice at any time in settings → Granular controls allow users to consent to specific telemetry types
8. Test that telemetry settings are granular (separate controls for crash reports, usage stats, etc.) → Changes to data collection practices trigger new consent requests
9. Verify that consent choices persist across browser sessions and updates → Consent records are maintained for compliance auditing

**Pass Criteria**: Explicit consent is obtained before telemetry collection AND users can easily manage consent preferences AND browser functions normally when telemetry is declined

**Fail Criteria**: Telemetry starts without consent OR consent cannot be withdrawn OR declining breaks browser functionality OR dark patterns are used

**Evidence**: Screenshots of consent prompts and settings UI, network traces showing no telemetry when declined, functional testing with telemetry disabled, consent flow video recordings, privacy policy documentation

**References**:

- GDPR Consent Requirements: https://gdpr.eu/gdpr-consent-requirements/
- ePrivacy Directive: https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:32002L0058
- W3C Privacy Principles - User Control: https://www.w3.org/TR/privacy-principles/#user-control

### Assessment: LOG-REQ-10 (Secure log transmission)

**Reference**: LOG-REQ-10 - Browser shall transmit logs securely using encrypted channels with certificate validation

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that secure log transmission prevents interception or modification of telemetry and crash reports in transit, protecting sensitive diagnostic data from network attackers. Without encrypted transmission and certificate validation, adversaries can eavesdrop on log data to gain insights into user behavior, browser vulnerabilities, or enterprise configurations, or perform man-in-the-middle attacks to inject false telemetry data.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Enable telemetry and crash reporting in browser settings → Verify log transmission is batched and rate-limited to prevent network abuse
2. Trigger events that generate log transmissions (crash, CSP violation, NEL error) → All log transmissions use TLS 1.2 or higher encryption
3. Use network monitoring tools (Wireshark, mitmproxy) to capture log transmission traffic → Certificate validation is enforced for log collection servers
4. Verify all log transmissions use HTTPS (TLS 1.2 or higher) → Invalid or expired certificates prevent log transmission
5. Verify certificate validation is performed for log collection endpoints → Certificate pinning is applied to log endpoints where supported
6. Test that log transmission fails if the server certificate is invalid → No log data is ever transmitted in plaintext
7. Check that certificate pinning is used for log collection endpoints if available → Connection failures trigger retry with exponential backoff
8. Verify log data is not transmitted over insecure protocols (HTTP, FTP, unencrypted sockets) → Log batching reduces network overhead and improves privacy
9. Test that log transmission includes retry logic for temporary network failures → Rate limiting prevents log transmission from consuming excessive bandwidth

**Pass Criteria**: All log transmissions use TLS 1.2+ with certificate validation AND transmission fails for invalid certificates AND no plaintext transmission occurs

**Fail Criteria**: Any logs transmitted over plaintext protocols OR certificate validation is not enforced OR invalid certificates are accepted

**Evidence**: Network packet captures showing TLS-encrypted log traffic, certificate validation test results, failed transmission logs for invalid certificates, retry mechanism testing, bandwidth usage analysis

**References**:

- TLS 1.3 Specification RFC 8446: https://www.rfc-editor.org/rfc/rfc8446
- Certificate Pinning: https://owasp.org/www-community/controls/Certificate_and_Public_Key_Pinning
- OWASP Transport Layer Protection: https://cheatsheetseries.owasp.org/cheatsheets/Transport_Layer_Protection_Cheat_Sheet.html
- Mozilla TLS Configuration: https://wiki.mozilla.org/Security/Server_Side_TLS

### Assessment: LOG-REQ-11 (Log integrity protection)

**Reference**: LOG-REQ-11 - Browser shall implement integrity protection for locally stored logs to prevent tampering

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that log integrity protection prevents attackers from covering their tracks after compromising a system by tampering with security logs. Without integrity protection, malicious actors who gain local access can modify or delete log entries to hide evidence of their activities, making incident response and forensic investigation impossible. Cryptographic integrity mechanisms ensure that any tampering is detected.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Enable local security logging in browser configuration or enterprise policy → Test that log integrity is checked before logs are exported or transmitted
2. Generate security events that create local log entries → Local logs include integrity protection mechanisms (signatures, MACs, or hashes)
3. Locate the local log storage files in the browser's data directory → Tampering with log contents is detected by the browser
4. Verify that log files include cryptographic signatures or message authentication codes (MACs) → Log files use OS-level protection where available (append-only, immutable flags)
5. Attempt to modify a log entry manually and verify the tampering is detected → Log rotation preserves integrity chains across files
6. Check that log files use append-only mechanisms where supported by the OS → Corrupted logs trigger alerts or warnings
7. Verify log rotation maintains integrity chains between rotated files → Enterprise deployments support strong integrity mechanisms (digital signatures)
8. Test that the browser detects and alerts on corrupted or tampered logs → Integrity checks occur before log export or transmission
9. Verify enterprise-mode logs support additional integrity mechanisms (digital signatures) → Integrity metadata is stored separately from log content for additional protection

**Pass Criteria**: Logs include integrity protection (signatures/MACs/hashes) AND tampering is detected AND alerts are generated for integrity violations

**Fail Criteria**: Logs lack integrity protection OR tampering is not detected OR no alerts for integrity violations

**Evidence**: Log file analysis showing integrity mechanisms, tampering test results demonstrating detection, alert screenshots for corrupted logs, documentation of integrity algorithms used, enterprise policy configurations

**References**:

- NIST FIPS 180-4 Secure Hash Standard: https://csrc.nist.gov/publications/detail/fips/180/4/final
- Log Integrity and Non-Repudiation: https://www.nist.gov/publications/guide-computer-security-log-management
- Merkle Tree for Log Integrity: https://en.wikipedia.org/wiki/Merkle_tree
- OWASP Logging Guide - Integrity: https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html

### Assessment: LOG-REQ-12 (Log retention policies)

**Reference**: LOG-REQ-12 - Browser shall implement and enforce log retention policies that balance security needs with privacy requirements

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that log retention policies balance security investigation needs against privacy rights by limiting how long personal data is stored. Excessive retention violates privacy regulations like GDPR which mandate data minimization, while insufficient retention hampers security incident investigation. Proper retention policies ensure logs are available for legitimate security purposes without becoming an indefinite privacy liability.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Review browser documentation for default log retention policies → Verify that regulatory compliance requirements (GDPR, etc.) are considered in retention
2. Examine local log storage to identify retention periods for different log types → Default retention periods are documented for each log type
3. Verify that security logs have appropriate retention (30-90 days typical) → Security logs are retained longer than general telemetry (30-90 days vs. 7-30 days)
4. Test that crash dumps are automatically deleted after retention period → Automatic deletion occurs when retention period expires
5. Verify that telemetry data has shorter retention than security logs → Log rotation prevents disk exhaustion (size-based limits)
6. Check that enterprise mode supports configurable retention policies → Enterprise policies allow customization of retention periods
7. Test that log rotation occurs based on size and time criteria → Users can manually clear logs through settings or clear browsing data
8. Verify that users can manually clear logs before retention period expires → Retention enforcement continues even when browser is not running
9. Test that retention policies are enforced even when browser is closed → GDPR/privacy compliance is demonstrated through retention limits

**Pass Criteria**: Documented retention policies exist for all log types AND automatic deletion enforces retention AND policies comply with privacy regulations

**Fail Criteria**: No retention policies OR logs grow unbounded OR retention periods violate privacy regulations (too long)

**Evidence**: Retention policy documentation, log file age analysis, storage usage over time, automatic deletion test results, enterprise policy configuration examples, GDPR compliance analysis

**References**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
- ISO 27001 Log Management: https://www.iso.org/standard/27001
- NIST SP 800-92 Log Retention: https://csrc.nist.gov/publications/detail/sp/800-92/final

### Assessment: LOG-REQ-13 (Security dashboard)

**Reference**: LOG-REQ-13 - Browser shall provide a security dashboard that presents security events and status to users

**Given**: A conformant browser with LOG-1 or higher capability

**Task**: Verify that a security dashboard empowers users to understand their security posture and respond to threats by providing clear visibility into security events and protection status. Without a dashboard, users remain unaware of ongoing attacks, misconfigurations, or compromised security settings, leaving them vulnerable. Transparent security status information enables informed security decisions and builds user trust.

**Verification**:

Daniel Thompson-Yvetot's avatar
Daniel Thompson-Yvetot committed
1. Access the browser's security dashboard through the browser's settings interface → Test that the dashboard supports filtering and searching of security events
2. Verify the dashboard displays current security status (safe/warning/critical) → Security dashboard is easily accessible from main settings
3. Check that recent security events are listed with timestamps and descriptions → Current security status is clearly displayed with visual indicators
4. Trigger a security event (certificate error, malware warning, etc.) and verify it appears in the dashboard → Recent security events are listed chronologically with timestamps
5. Test that the dashboard categorizes events by severity (critical, warning, info) → Events are categorized by severity level with appropriate visual coding
6. Verify the dashboard shows security settings status (HTTPS-only, Safe Browsing, etc.) → Each event includes actionable information and remediation guidance
7. Test that clicking on security events provides detailed information and remediation steps → Dashboard updates when new security events occur
8. Verify the dashboard updates in real-time or near-real-time when security events occur → Users can filter events by type, severity, or time period
9. Check that the dashboard is accessible from the main browser settings menu → Dashboard shows overall security posture (enabled protections)

**Pass Criteria**: Security dashboard is accessible AND displays recent security events with severity AND provides actionable remediation guidance

**Fail Criteria**: No security dashboard exists OR dashboard doesn't show events OR events lack context/remediation info

**Evidence**: Screenshots of security dashboard showing various states, video walkthrough of dashboard features, security event listings, user interface usability assessment, comparison with security best practices

**References**:

- Chrome Security Settings: https://support.google.com/chrome/answer/114836
- NIST Cybersecurity Framework - Detect: https://www.nist.gov/cyberframework
- User-Centered Security Design: https://www.usenix.org/conference/soups2019

### Assessment: LOG-REQ-14 (Incident detection)

**Reference**: LOG-REQ-14 - Browser shall implement automated incident detection based on security event patterns

**Given**: A conformant browser with LOG-2 or higher capability

**Task**: Verify that automated incident detection identifies active attacks by correlating security event patterns that indicate malicious activity, enabling rapid response before significant damage occurs. Manual log review alone cannot detect sophisticated attacks that span multiple events or occur at scale. Automated detection using heuristics and pattern matching provides early warning of credential stuffing, reconnaissance, malware distribution, and other attack campaigns.

**Verification**: