The first thing I did, to filter input, was to setup a Google Reader account for Florida Death Metal. At least this allowed me to see updates from a variety of other sites in a single place. This also allows me to search through those new listings for keywords to bands and events I think are important to know about.

This still requires me to visit Google Reader, and parse through a lot of stuff (routinely over 1000 new items) to find out what’s going on in the world. The real dilemma I was having, was trying to implement an active alert system to news going on around me. The active part, was me actively searching through Google Reader for information relevant to Florida Death Metal. 

Needless to say, this started to suck. I have a great full time job, a wonderful wife, and a million other things I like to do with my spare time. Spending my days and nights on Google Reader, struggling to keep up with news for a project blew.

I love Florida Death Metal. I’ve never been involved in something that means so much to me. However, there are only so many days in the week, and so much time in each of those days. I need a way to passively keep up with relevant news and events. I needed some automation, so I could spend time hanging drapes for Melissa, or having a beer with Rob.

Enter Zend_Http_Client ….

One of my favorite programming tricks is breaking down requests to servers, and finding a way to do them programatically. So, I got to thinking about how I could parse through all of my stuff in Google Reader. For each of the views in Google Reader, there is an associated RSS feed. Well, that makes things simple enough. I could just grab the RSS feed and parse it. 

One minor detail, the RSS feeds aren’t publicly available. You have to be logged in to use them. This makes sense. I imagine Google doesn’t want to be used as an aggregator of RSS feeds to be used as a proxy to other sites. Sorry :/

So, here’s a breakdown of what I needed to write a script to do:

1. Login to Google
2. Grab the RSS Feed for ‘All Items’
3. Parse the RSS Feed for keywords relevant to Florida Death Metal
4. Email me alerts (if there are matches)

Not too bad. So, Here’s the script I came up with. I hope you like it :

 
<?php
/*
Description: Script to parse through a google reader aggregation of content for keywords
Version: 1.0
Author: Cory Collier
Author URI: http://corycollier.com/
*/
 
//Define the constants required for the script
define('GOOGLE_PASSWORD', 	'<your password here>');
define('GOOGLE_RSS_URI', 		'<the url for your google reader rss feed>');
define('GOOGLE_LOGIN_URI',		'https://www.google.com/accounts/ServiceLoginAuth?service=reader');
define('GOOGLE_LOGIN_EMAIL', 	'<your gmail account username / email>');
define('MAILER_FROM_ADDR',		'<where the email should come from>');
define('MAILER_TO_ADDR',		'<where the email should go to>');
define('MAILER_SUBJECT',		'Google Reader Parsing');
 
/*
* a Zend Framework Installation _MUST_ be located on the include path for PHP
*/
require 'Zend/Loader.php';
Zend_Loader::registerAutoload();
 
//Instantiate a new Zend_Http_Client, with the google login url
$client = new Zend_Http_Client(GOOGLE_LOGIN_URI);
 
/*
 * Set the client cookie jar ...
 * Set the method to POST ..
 * Set the parameters to post with
 */
$client->setCookieJar()
	->setMethod(Zend_Http_Client::POST)
	->setParameterPost(array(
		'continue'		=> GOOGLE_RSS_URI,
		'service'		=> 'reader',
		'niu'			=> 1,
		'hl'			=> 'en', 
		'Email'		=> GOOGLE_LOGIN_EMAIL,
		'Passwd'		=> GOOGLE_PASSWORD,
		'PersistentCookie'	=> 'yes',
		'asts'			=> ''
));
 
//make the login request, and store the response in the $response variable ...
$response = $client->request('POST');
 
//If the response was successful, change the uri value for the client object
// to the appropriate rss file for parsing
$client->setUri(GOOGLE_RSS_URI)
 
		//Change the request method to GET
		->setMethod(Zend_Http_Client::GET); 
 
//send the request, and store the results of it
$response = $client->request();
 
//Initialize an array of keywords to look for
$keywords = array (
	//Whatever your keywords are you're looking for
);
 
//SimpleXML is great!
$sx = simplexml_load_string ($response->getBody());
 
//Iterate through each of the retrieved entries
foreach ($sx->entry as $entry ) 
{	//Now, iterate through each of the defined keywords / keyphrases
    foreach ( $keywords as $keyword ) 
    {	//First, check to see if the title contains a keyword / keyphrase
        if ( stristr((string)$entry->title, $keyword) ) 
        {	//Append any matches to the matches arrays
            $matches[] = (string)$entry->link['href'];
        }
    	//Next, check to see if there are any matches in the summary
        if ( stristr((string)$entry->summary, $keyword) ) 
        {	//same deal: If there are matches, add them to the stack
            $matches[] = (string)$entry->link['href'];
        }
    } //END keyword iteration
 
} // END posting iteration
 
//IF matches were found, send an email
if(count($matches)) 
{	//Initialize a variable to store a mesage in
    $message = '';
 
    //Iterate through each of the matches
    foreach($matches as $match)
    {	//For each of the matches, append them to the message string, separated 
    	// by newlines
        $message .= "\n" .  $match;  
    }
    //Initialize a new Zend_Mail object 
    $mail = new Zend_Mail;
 
    //Set the parameters necessary to send a message 
    $mail->addTo(MAILER_TO_ADDR)
        ->setFrom(MAILER_FROM_ADDR)
        ->setSubject(MAILER_SUBJECT)
        ->setBodyText($message);
 
    //I never trust email, so wrap the email execution in a try/catch statement
    try  
    {	//Send the mail
        $mail->send();   
    } catch (Exception $e ){
        //Do something here
    }
}

Wrapping all of this up, I stuck this script on a spare debian box at my house, and setup a cronjob to run the script every 15 minutes. It saves me a lot of time. I’d love to hear some feedback from y’all about how I got this done. I’m a script guy at heart. So, this stuff is super fun for me.

Wrapping all of this up, I stuck this script on a spare debian box at my house, and setup a cronjob to run the script every 15 minutes. It saves me a lot of time. I'd love to hear some feedback from y'all about how I got this done. I'm a script guy at heart. So, this stuff is super fun for me.

I know I should have checked the success of the initial login attempt before assuming the second request would get anything at all. Keep in mind though, this is a script I use to make my own life easier. Exceptions being thrown here cause me no harm. If I don't get anything, I can check my error logs for issues. Not a perfect solution, but it's working pretty well for me now.

“Invalid parameter number: no parameters were bound”

 When people used our search feature on our site, a few specific search terms would result in un-caught exceptions. Of course, this ONLY happened on the client’s server. Nothing like saying a project is solid, only to find out there’s something ‘special’ about the production environment.

The first thing we did, was to list out what was unique about the client’s server. The client was using a Media Temple box running CentOS 5.2. For the un-initiated, CentOS is the free version of Red Hat Enterprise Server. We run Macs in the office, and our staging server is a Mac too. Other than the OS, the normal ‘LAMP’ stuff pretty much matched verbatim. 

So, we started digging into the differences between the compiled versions of PHP between the development and production environments, and one thing popped out at us:

PCRE

The version that’s supported on Red Hat / CentOS is 6.6. That’s horrible. 6.6 came out nearly 3 years ago, and the version the client was running had no support for unicode at all. A little research, and we found out that Zend Search Lucene (what we built the search functionality on top of) requires unicode for the way it stores search indexes.

With that, we figured we were done. The client had a limitation on their server, they needed to address it, and the problem would fix itself when they did. Not quite so fast …

I was testing some of the searching on my iMac, when the same issue happened on my own computer. Disaster! Could it be that something in our own code was the culprit? What half-reproducible error was causing this? 

After hours of searching for an answer on the googlez, I came across some help on the Zend Issue Tracker. It turns out, that PDO was failing when it was trying to prepare a statement, when that statement contained a question mark. When I switched my SQL adapter to Mysqli, the problem was solved. Both the production and the development environments were bug free after the change.

It turns out, that our search indexes would return fields that either contained question marks, because they actually existed in the document (as was the case locally), or because the document had encoding errors when the search index was built (as was the case on the production servers). The ORM we used would grab the documents, and grab relevant data from the database by querying with the fields stored in the index.

The SQL that was being prepared, would then look something like this:

‘SELECT id FROM folks WHERE first_name LIKE ‘Jo?hnny’

The question marks would be interpreted by PDO as variable markers, which rightfully didn’t exist.

So, the real solution wound up being a little bit of a mix between the client’s problem, and our own. Granted, we needed to catch question marks being stored in db before they got there. That improves the longevity of our own code. However, the search functionality will still return results that may have question marks in them, thus causing the same issue. That issue is resolved by using Mysqli, but that feels more like a hack, than a solution.

Anyways, I spent a long time trying to search the answer to this and found nothing except the one mention in the issue tracker. For those of you using Zend Lucence Search on Red Hat / CentOS servers. Make sure you use Mysql if you’re using the Zend ORM to populate models based on results returned from the index. 

Oh yeah, and make sure you filter your input too, Mr. Bobby Tables…..

After a brief search on Google, I came up with a German forum with a potential solution: APC (Alternative PHP Cache). It turns out that Caching for PHP 5.2 is different than for PHP 5.0. So, I turned of APC (by editing the php.ini file).

For all of you coders out there, just an FYI.