NOT an issue, just a question regarding PB operating mode #1680
Labels
DNT policy
EFF's Do Not Track policy: www.eff.org/dnt-policy
question
Further information is requested
Hi,
I have PB installed in all Chrome browsers across all my PCs, and I love it even when it breaks some websites. But I have a question regarding how PB's backend works for URLs not found in your database.
Do you crawl new URLs? If so, what is the User Agent being used? What are the IP address pool being used to source the crawling? Couldn't find any information regarding these questions in your FAQ.
I'm asking this because recently I was developing a new api endpoint, while using chrome to test it, suddenly I started seeing requests being made from Japan, then after blocking those, from Amazon, and so on.
One piece of advice, if you're indeed crawling new URLs for the sake of ranking the site in your system, please do so with a User Agent signature that identify you.
Thank you.
The text was updated successfully, but these errors were encountered: