In Basic and Advanced plans, Zyte Smart Proxy Manager(formerly Crawlera) discards cookies to protect customer privacy and optimize performance to the Smart Proxy Manager sites, because cookies are often used as a detection mechanism by Antibot systems.


In Enterprise and the legacy plans (C10, C50, C100, C200) it works as follows:


Smart Proxy Manager manages cookies for you by default and retains them for up to 15 minutes since the last request. Smart Proxy Manager keeps separate groups of cookies per outgoing node, and as a result consecutive requests will almost always have different cookies so, if you need to use cookies for things like authentication, then you will want to manage them yourself.


To store and manage cookies yourself you will need to disable Smart Proxy Manager cookie handling with the X-Crawlera-Cookies header. If cookie handling is not disabled, Smart Proxy Manager will discard the cookies and send its own instead.


Consider also using Smart Proxy Manager sessions