67 second demo
Blog Post summarizing our study
Full study tech report

This is a simple website designed to study user interaction with California Consumer Privacy Act "Do Not Sell My Information" notices, and the effect of user interface design on this interaction. It deployed to ~4,300 users in a summer 2020 study at Pomona College's Privacy and Security lab. The site was jointly developed by Sean O'Connor (who built the backend and Javascript event logging code), Ryan Nurwono (who developed the rest of the site front-end), and Eleanor Birrell.

The website displays top news stories from a variety of sources across the US political spectrum, as well as international media outlets. It also displays a random design of CCPA notice based on a hash of the user's IP address. User interactions with the website are logged under a pseudonymized uniqueId.

1. Install required python packages. A list of dependencies can be found in "requirements.txt"; all are available through PIP.
2. Initialize the SQL database tables. Do this by running a python shell in the example_site directory, then run the following commands:

   from app import db
   db.create_all()

3. To test on localhost, run "app.py." If you wish to record logs to a file, redirect output when running like so:

   python app.py > filename.csv

   Otherwise all events will be logged in the python console.
4. Visit the site at the URL listed in the console.

Data is logged in a csv-friendly format. One line will be logged for each event detected by the server. Here's an example:

"d597f4b582f52fccf9aa021885419e3d1c08a7195aeac10b7efa64cca584422a","http://localhost:5000/", "1594851366.6","window_load","na","2","Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0","false",

The values are in the following order:

1. User ID - a unique identifier generated by hashing the user's IP address.
2. URL - The URL of the page the event was logged on (excluding any parameters)
3. Timestamp - Epoch timestamp of the event, in seconds, rounded to the nearest 1/10th.
4. Event Type - The type of event occurring. The following events are logged: "click" - user clicks on a page Element "window_load" - Triggered when page finishes loading, this may occur before the user actually sees the page when IOS pre-loads the page as the user types the URL in the browser. Marks the start of a user's visit. "page_unload" - Triggered by the page being closed, this will not be logged for all visits as mobile devices don't usually trigger an unload event. Marks the end of a user's visit. "page_blur" - Triggered when the page loses focus or visibility, such as when the page is minimized, the browser is clicked away from, a new tab is opened, etc. This may mark a temporary minimization of the page or, on mobile devices, may mark the end of the user visit as the page is closed. Won't always be triggered upon page closing on IOS, but will always be triggered when the page is minimized. "page_active" - A heartbeat message used to measure the duration of a user's visit. The page tries to log one of these events every second to record that the page is still open. The only definitive way to know when the page has been closed is when these events stop being logged. If the page is minimized, these will still be logged but in many browsers start being logged at a slower interval (less than once a second) due to limitations on background processes. "survey_id" - Used to link Mechanical Turk study participants to a qualtrics survey ID, this is logged when the user visits with a survey id passed as a URL parameter.
5. Element ID - If Event Type is "click", this is a description of what was clicked on. If Event Type is "survey_id", this is an ID to link the user to their Qualtrics survey. Otherwise, "na" (not applicable) is logged.
6. Banner Style - Number indicating the style of privacy banner that has been determined to be displayed to the user.
   This could be a desktop or mobile style, and must be interpreted in conjunction with the "mobile" column. If the event is logged on a page that never displays a privacy banner (such as the privacy policy page), "nobanner" is logged instead.
7. User Agent - the full user agent string provided by the user's browser through navigator.userAgent. Gives the full browser version information.
8. Mobile - a simple "true" or "false" value indicating whether the user is on a mobile device, as determined by their screen width. If the screen width is less than 500, this is set to "true" and the CCPA banner style is chosen from the mobile list instead of desktop.

app.py - back-end Flask server functionality
static/js/scripts.js - front end javascript to log user interaction
static/js/main.js - javascript displaying CCPA notice
templates/base.html - base HTML template
templates/index.html - homepage displaying all news stories
templates/center.html - page filtering stories from centrist sources
templates/left.html - page filtering stories from left-leaning sources
templates/right.html - page filtering stories from right-leaning sources
templates/international.html - page filtering stories from international sources
templates/privacypolicy.html - page with study privacy disclosures and option to opt-out
templates/optout.html - page indicating user has successfully opted out of the study.