Quantcast

Cookies not getting set in HTMLUnit 2.23

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Cookies not getting set in HTMLUnit 2.23

Vasudevan Comandur
Hi,

   I am creating a set of cookies using the constructor of Cookie Object.
   Then I am adding those cookies into the cookieManager of a Webclient
   before sending a request to the site.

  When I looked into Charles WebProxy tool, I did not see those cookies
  set in the HTTP Cookie header.

  Whereas the same application was working fine with earlier version of HTMLUnit
  say 2.15.

  Am I missing something in HTMLUnit 2.23?
   Help needed urgently.

   Below is the clojure code snippet for reference:

   Creating a cookie
   val1= (new Cookie ".jobdiva.com" "COMPANYLOGOLOCATION" clogo-data "/" -1 false)

    Adding cookie to Cookie Manager
   (.addCookie (.getCookieManager *wc*) val1)

   Please note *wc* is the WebClient Object.

   Appreciate your help.

Regards
 Vasu

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Htmlunit-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/htmlunit-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cookies not getting set in HTMLUnit 2.23

Ronald Brill
Hi Vasu,

it will be really helpfull to provide all the detail you have.

Regarding your question
* what is clogo-data?
* which url you are calling

And it will really help to have complete plain java test cases to verify your problems.

Have added another test case that tries to simulate/reproduce your problem

trunk/htmlunit/src/test/java/com/gargoylesoftware/htmlunit/CookieManager2Test.java
    @Test
    @Alerts("my_name=my_data")
    public void cookie_maxAgeMinusOne() throws Exception {
        final WebClient webClient = getWebClient();
        final MockWebConnection webConnection = new MockWebConnection();

        final URL url = URL_FIRST;
        webConnection.setResponse(url, CookieManagerTest.HTML_ALERT_COOKIE);
        webClient.setWebConnection(webConnection);

        final CookieManager mgr = webClient.getCookieManager();
        mgr.addCookie(new Cookie(URL_FIRST.getHost(), "my_name", "my_data", "/", -1, false));

        final List<String> collectedAlerts = new ArrayList<>();
        webClient.setAlertHandler(new CollectingAlertHandler(collectedAlerts));

        webClient.getPage(URL_FIRST);
        assertEquals(getExpectedAlerts(), collectedAlerts);
    }

This test works for me. Please have a look - maybe there is a difference for your case (or maybe the test did not work with your HtmlUnit version).


Hope that helps

        RBRi




On Wed, 1 Feb 2017 18:51:49 +0530 Vasudevan Comandur wrote:

>
>Hi,
>
>   I am creating a set of cookies using the constructor of Cookie Object.
>   Then I am adding those cookies into the cookieManager of a Webclient
>   before sending a request to the site.
>
>  When I looked into Charles WebProxy tool, I did not see those cookies
>  set in the HTTP Cookie header.
>
>  Whereas the same application was working fine with earlier version of
>HTMLUnit
>  say 2.15.
>
>  Am I missing something in HTMLUnit 2.23?
>   Help needed urgently.
>
>   Below is the clojure code snippet for reference:
>
>   Creating a cookie
>   val1= (new Cookie ".jobdiva.com" "COMPANYLOGOLOCATION" clogo-data "/" -1
>false)
>
>    Adding cookie to Cookie Manager
>   (.addCookie (.getCookieManager *wc*) val1)
>
>   Please note *wc* is the WebClient Object.
>
>   Appreciate your help.
>
>Regards
> Vasu
>
>
>
>----< Inline text [text-plain-04.txt] >------------------
>
>------------------------------------------------------------------------------
>Check out the vibrant tech community on one of the world's most
>engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>
>
>----< Inline text [text-plain-05.txt] >------------------
>
>_______________________________________________
>Htmlunit-user mailing list
>[hidden email]
>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Htmlunit-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/htmlunit-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cookies not getting set in HTMLUnit 2.23

Vasudevan Comandur
Hi Ronald,

   I am calling http://www2.jobdiva.com/......URL after calling the add cookies.

   I have only the clojure snippet which I had copied.

   The clogo-data is the variable which has the Value for the cookie name COMPANYLOGOLOCATION.

   The same was working fine with HTMLUnit Version 2.15. I am just migrating the app to HTMLUnit 2.23 and found this issue

Regards
 Vasu

On 1 February 2017 at 23:20, Ronald Brill <[hidden email]> wrote:
Hi Vasu,

it will be really helpfull to provide all the detail you have.

Regarding your question
* what is clogo-data?
* which url you are calling

And it will really help to have complete plain java test cases to verify your problems.

Have added another test case that tries to simulate/reproduce your problem

trunk/htmlunit/src/test/java/com/gargoylesoftware/htmlunit/CookieManager2Test.java
    @Test
    @Alerts("my_name=my_data")
    public void cookie_maxAgeMinusOne() throws Exception {
        final WebClient webClient = getWebClient();
        final MockWebConnection webConnection = new MockWebConnection();

        final URL url = URL_FIRST;
        webConnection.setResponse(url, CookieManagerTest.HTML_ALERT_COOKIE);
        webClient.setWebConnection(webConnection);

        final CookieManager mgr = webClient.getCookieManager();
        mgr.addCookie(new Cookie(URL_FIRST.getHost(), "my_name", "my_data", "/", -1, false));

        final List<String> collectedAlerts = new ArrayList<>();
        webClient.setAlertHandler(new CollectingAlertHandler(collectedAlerts));

        webClient.getPage(URL_FIRST);
        assertEquals(getExpectedAlerts(), collectedAlerts);
    }

This test works for me. Please have a look - maybe there is a difference for your case (or maybe the test did not work with your HtmlUnit version).


Hope that helps

        RBRi




On Wed, 1 Feb 2017 18:51:49 +0530 Vasudevan Comandur wrote:
>
>Hi,
>
>   I am creating a set of cookies using the constructor of Cookie Object.
>   Then I am adding those cookies into the cookieManager of a Webclient
>   before sending a request to the site.
>
>  When I looked into Charles WebProxy tool, I did not see those cookies
>  set in the HTTP Cookie header.
>
>  Whereas the same application was working fine with earlier version of
>HTMLUnit
>  say 2.15.
>
>  Am I missing something in HTMLUnit 2.23?
>   Help needed urgently.
>
>   Below is the clojure code snippet for reference:
>
>   Creating a cookie
>   val1= (new Cookie ".jobdiva.com" "COMPANYLOGOLOCATION" clogo-data "/" -1
>false)
>
>    Adding cookie to Cookie Manager
>   (.addCookie (.getCookieManager *wc*) val1)
>
>   Please note *wc* is the WebClient Object.
>
>   Appreciate your help.
>
>Regards
> Vasu
>
>
>
>----< Inline text [text-plain-04.txt] >------------------
>
>------------------------------------------------------------------------------
>Check out the vibrant tech community on one of the world's most
>engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>
>
>----< Inline text [text-plain-05.txt] >------------------
>
>_______________________________________________
>Htmlunit-user mailing list
>[hidden email]
>https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Htmlunit-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/htmlunit-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cookies not getting set in HTMLUnit 2.23

Ronald Brill
Ok Vasu,

this was a really tricky one.

You are interacting directly with the cookie manager in this way

    final CookieManager mgr = webClient.getCookieManager();
   mgr.addCookie(new Cookie(".jobdiva.com", "COMPANYLOGOLOCATION", "my_data", "/", -1, false));

In this case you are responsible for all the cookie properties.
But there is some magic, if a web site sends a comparable cookie like "COMPANYLOGOLOCATION=my_data; Domain=.jobdiva.com; Path=/".
Internally there is some cleanup before the cookie is added to the cookie manager, in this case the leading dot will be removed from the domain.
When sending th cookies back the cookie matches now, because of the removed dot at the start. This logic is more or less part of HttpClient already.

Ok, and what to do now?

I guess you like to simulate some cookies sent from the server before you are starting your request. I think the more correct way is to interact with the webClient instead of the CookieManager directly because there is a way to use the same magic as we use
when processing cookies from the real response.

        webClient.addCookie("COMPANYLOGOLOCATION=my_data; Domain=.jobdiva.com; Path=/");

is your friend.
Hope that helps.

        RBRi

>
>Hi Ronald,
>
>   I am calling http://www2.jobdiva.com/......URL after calling the add
>cookies.
>
>   I have only the clojure snippet which I had copied.
>
>   The clogo-data is the variable which has the Value for the cookie name
>COMPANYLOGOLOCATION.
>
>   The same was working fine with HTMLUnit Version 2.15. I am just
>migrating the app to HTMLUnit 2.23 and found this issue
>
>Regards
> Vasu
>
>On 1 February 2017 at 23:20, Ronald Brill <[hidden email]> wrote:
>
>> Hi Vasu,
>>
>> it will be really helpfull to provide all the detail you have.
>>
>> Regarding your question
>> * what is clogo-data?
>> * which url you are calling
>>
>> And it will really help to have complete plain java test cases to verify
>> your problems.
>>
>> Have added another test case that tries to simulate/reproduce your problem
>>
>> trunk/htmlunit/src/test/java/com/gargoylesoftware/htmlunit/
>> CookieManager2Test.java
>>     @Test
>>     @Alerts("my_name=my_data")
>>     public void cookie_maxAgeMinusOne() throws Exception {
>>         final WebClient webClient = getWebClient();
>>         final MockWebConnection webConnection = new MockWebConnection();
>>
>>         final URL url = URL_FIRST;
>>         webConnection.setResponse(url, CookieManagerTest.HTML_ALERT_
>> COOKIE);
>>         webClient.setWebConnection(webConnection);
>>
>>         final CookieManager mgr = webClient.getCookieManager();
>>         mgr.addCookie(new Cookie(URL_FIRST.getHost(), "my_name",
>> "my_data", "/", -1, false));
>>
>>         final List<String> collectedAlerts = new ArrayList<>();
>>         webClient.setAlertHandler(new CollectingAlertHandler(
>> collectedAlerts));
>>
>>         webClient.getPage(URL_FIRST);
>>         assertEquals(getExpectedAlerts(), collectedAlerts);
>>     }
>>
>> This test works for me. Please have a look - maybe there is a difference
>> for your case (or maybe the test did not work with your HtmlUnit version).
>>
>>
>> Hope that helps
>>
>>         RBRi
>>
>>
>>
>>
>> On Wed, 1 Feb 2017 18:51:49 +0530 Vasudevan Comandur wrote:
>> >
>> >Hi,
>> >
>> >   I am creating a set of cookies using the constructor of Cookie Object.
>> >   Then I am adding those cookies into the cookieManager of a Webclient
>> >   before sending a request to the site.
>> >
>> >  When I looked into Charles WebProxy tool, I did not see those cookies
>> >  set in the HTTP Cookie header.
>> >
>> >  Whereas the same application was working fine with earlier version of
>> >HTMLUnit
>> >  say 2.15.
>> >
>> >  Am I missing something in HTMLUnit 2.23?
>> >   Help needed urgently.
>> >
>> >   Below is the clojure code snippet for reference:
>> >
>> >   Creating a cookie
>> >   val1= (new Cookie ".jobdiva.com" "COMPANYLOGOLOCATION" clogo-data "/"
>> -1
>> >false)
>> >
>> >    Adding cookie to Cookie Manager
>> >   (.addCookie (.getCookieManager *wc*) val1)
>> >
>> >   Please note *wc* is the WebClient Object.
>> >
>> >   Appreciate your help.
>> >
>> >Regards
>> > Vasu
>> >
>> >
>> >
>> >----< Inline text [text-plain-04.txt] >------------------
>> >
>> >-----------------------------------------------------------
>> -------------------
>> >Check out the vibrant tech community on one of the world's most
>> >engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> >
>> >
>> >----< Inline text [text-plain-05.txt] >------------------
>> >
>> >_______________________________________________
>> >Htmlunit-user mailing list
>> >[hidden email]
>> >https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>> >
>> >
>>
>>
>
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Htmlunit-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/htmlunit-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cookies not getting set in HTMLUnit 2.23

Ronald Brill
In reply to this post by Vasudevan Comandur
Now i'm confused...

>   Suppose, I am using HTMLUnit in JS disabled mode and when I interact with a server and the server sends an HTML page
>    which has a Javascript for setting cookies using setcookie method.
>Please note that the response header does not contain
>   Set-Cookie.
>
>    How does HTMLUnit handle the above situation?. It looks like I need to enable JS. Is there a way to solve/work around this issue?

I think i have proposed a solution:

replace

        mgr.addCookie(new Cookie(".jobdiva.com", "COMPANYLOGOLOCATION", "my_data", "/", -1, false));
by
          webClient.addCookie("COMPANYLOGOLOCATION=my_data; Domain=.jobdiva.com; Path=/");

This should work.

        RBRi


>
>On 2 February 2017 at 01:59, Ronald Brill <[hidden email]> wrote:
>
>> Ok Vasu,
>>
>> this was a really tricky one.
>>
>> You are interacting directly with the cookie manager in this way
>>
>>     final CookieManager mgr = webClient.getCookieManager();
>>    mgr.addCookie(new Cookie(".jobdiva.com", "COMPANYLOGOLOCATION",
>> "my_data", "/", -1, false));
>>
>> In this case you are responsible for all the cookie properties.
>> But there is some magic, if a web site sends a comparable cookie like
>> "COMPANYLOGOLOCATION=my_data; Domain=.jobdiva.com; Path=/".
>> Internally there is some cleanup before the cookie is added to the cookie
>> manager, in this case the leading dot will be removed from the domain.
>> When sending th cookies back the cookie matches now, because of the
>> removed dot at the start. This logic is more or less part of HttpClient
>> already.
>>
>> Ok, and what to do now?
>>
>> I guess you like to simulate some cookies sent from the server before you
>> are starting your request. I think the more correct way is to interact with
>> the webClient instead of the CookieManager directly because there is a way
>> to use the same magic as we use
>> when processing cookies from the real response.
>>
>>         webClient.addCookie("COMPANYLOGOLOCATION=my_data; Domain=.
>> jobdiva.com; Path=/");
>>
>> is your friend.
>> Hope that helps.
>>
>>         RBRi


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Htmlunit-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/htmlunit-user
Loading...