CURL Help

Corey · #1 09-Feb-2009, 22:42

Hey guys,

OK, here's what I am trying to do: I need to log into another website, and download their reports. Seems easy right? Well, it is... because I've done it with numerous other websites.

However, this one is tricky. The only way I can download the report at this website is by having the correct "key" at the end of the URL. The key is generated by their server and carries over with each link that you click on their site once logged in. Sort of like a session ID, but it's not a session ID at all.

Anyway, this key is unique to the current cookie session for that user. If they were to log out, none of those links with that unique trailing "key" would work. They would have to log back in and use the new links. The trailing key is nothing but 10 numbers long.

So, I've devised a small curl script that does everything... it logs into the site, grabs that trailing "key" from a hidden text field, and it downloads the CSV report. However, when I look at the downloaded report, it shows a "This page does not exist" custom apache page from their server.

The problem is that it is either losing the cookie when it goes to download the report URL, or it logs out/back in when it goes to download the report URL.

Here's the code (I've removed the URL's):

PHP Code:


$graburl = **URL OF CSV REPORT**;
$cookiefile = tempnam("/tmp", "cookies"); 
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $loginurl);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "$usernamefield=$username&$passwordfield=$password");

ob_start();
$output = curl_exec($ch);
ob_end_clean();

***************ADD CODE***************
$link = '**URL OF REPORT PAGE TO FETCH THE KEY**';
curl_setopt($ch, CURLOPT_URL, $link);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$page = curl_exec($ch);

$sk = '%<input type="hidden" id="window_name" value="(.*)">%';
preg_match_all($sk, $page, $results, PREG_PATTERN_ORDER);
***************ADD CODE***************

if ($output == '1') {
curl_setopt($ch, CURLOPT_URL, "$graburl&session_key=".$results[1][0]);
$putcsv = "report.csv";
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
file_put_contents("**MY URL**".$putcsv."", "$result");
}
curl_close($ch);

Note: This code works beautifully on other sites where I get reports from as well. It's just the extra code marked in "**ADD CODE**" and the "&session_key=".$results[1][0]" added to the graburl that is giving me problems.

It may have something to do with grabbing to URL's in the same curl session, I can't figure it out.

Any help is appreciated,

Thanks,
Corey

Corey · #2 11-Feb-2009, 00:31

Just an update:

I've tried it with just one instance of the cURL cookie options and I've tried it with init(). EX:

PHP Code:


$ch = curl_init ("http://somedomain.com/");

Neither worked.

If any one has any suggestions or knows a better to do this, let me know.

Thanks,
Corey

Howard_L · #3 11-Feb-2009, 07:41

This is over my head, but just so you don't feel like your talking to yourself...
I'm not sure if you've said, but
Is the request the script sends that fails identical to the manual request which works?
Can you check/compare the stream?
In Linux I heard the guys talking about a "wireshark" program which might be something to look into.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Rate This Thread
Rate This Thread:

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
memcpy Access violation 0xC0000005	minimanu	C++ Forum	2	03-Dec-2008 13:18
Stream to string conversion problem	cgillopez	C Programming Language	1	12-Aug-2008 09:36
Free Hosting CPanel 11 and Free Money !!!	turino	Free Web Hosting	2	10-Apr-2008 18:48

CURL Help

Re: CURL Help

Re: CURL Help