Deception Check

01 / PREPARE AN AUTHORIZED CHECK

Does a crawler label change
your access decision?

Choose a protected resource and compare two unauthenticated requests. Change only the User-Agent header. Both should remain denied.

Print this worksheet or keep the answers in your own case record. This page does not submit or save notes.

Name the test

System / application and environment

Authorized owner / scope and test window

Protected path or route reference

Expected deny behavior / policy reference

Keep the comparison fair

Ordinary User-Agent value / request reference

Claimed crawler value / request reference

Same method, path, network, and no signed-in session

CDN / cache / access-control owner

Safe setup

Use an inert test resource with the same protection policy. Use no real secret. Obtain approval for the target and timing; keep the method, path, cookies, and authorization state identical.

Inspect the response body as well as the status. Follow redirects and compare any cache behavior within the approved scope.

Separate identity from permission

02 / RECORD THE RESULT AND ASSIGN THE DECISION

Keep the evidence.
Name the next owner.

Record references to restricted evidence. Keep raw source addresses, sensitive response bodies, and credentials out of shared copies.

ComparisonOrdinary headerClaimed crawler header
UTC time / method
Claim / source reference
Verification method, time, result
Status / redirect / cache
Body reviewed / evidence reference
Expected vs. observed decision

What does this result support?

Both denied: record the tested route, conditions, and evidence. This checks that boundary under those conditions.

Unexpected access or different handling: preserve restricted evidence, assign the access-control or cache owner, and retest after the correction.

Identity unresolved: keep it unresolved. Do not infer identity from a successful status or a familiar crawler name.

Close the loop

Decision / owner / due date

Retest time, result, and reviewer

Keep the record useful

Restricted case / evidence reference

Next review trigger / policy owner