A few years ago, I had written a bunch of Sherlock plugins for searching various library catalogs. Those pretty much lay dormant until I was talking with my student worker this last semester and he got quite excited about these, and wondered if there might be a way to also search the databases the library subscribes to, and provide off-campus access to these through our proxy server. This code is in a very basic stage right now, but essentially what I did was create a database driven application in PHP that generates OpenSearch XML for various targets to search against.

Right now, this only supports HTTP GET requests (POST parameters require an additional table), but here’s the DDL:

CREATE TABLE `plugins` (
  `id` char(36) NOT NULL,
  `short_name` varchar(50) NOT NULL,
  `description` varchar(255) NOT NULL,
  `engine` varchar(25) NOT NULL,
  `image` varchar(25) NOT NULL,
  `input_encoding` varchar(10) NOT NULL,
  `url` varchar(550) NOT NULL,
  `method` varchar(4) NOT NULL default 'get',
  `proxy` tinyint(1) NOT NULL default '1',
  KEY `engine_index` (`engine`),
  KEY `short_name_index` (`short_name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

You don’t need all of these fields for every record, and here is some useful records…

INSERT INTO plugins (id, short_name, engine, input_encoding, method, proxy, url)
VALUES (UUID(), ' JStor', 'jstor', 'UTF-8', 'get', 1, 'http://www.jstor.org/action/doBasicSearch?Query=');
INSERT INTO plugins (id, short_name, engine, input_encoding, method, proxy, url)
VALUES (UUID(), 'Google Your University', 'google', 'UTF-8', 'get', 1, 'http://www.google.com/univ/wm?hl=en&ie=ISO-8859-1&btnG=Google+Search&q=');
INSERT INTO plugins (id, short_name, engine, input_encoding, method, proxy, url)
VALUES (UUID(), 'Google Scholar', 'googlescholar', 'UTF-8', 'get', 1, 'http://scholar.google.com/scholar?&hl=en&lr=&btnG=Search&q=');
INSERT INTO plugins (id, short_name, engine, input_encoding, method, proxy, url)
VALUES (UUID(), 'Ebrary', 'ebrary', 'UTF-8', 'get', 1, 'http://site.ebrary.com/lib//Top?layout=search&nosr=1&sch=sch&frm=smp.x&p00=');
INSERT INTO plugins (id, short_name, engine, input_encoding, method, proxy, url)
VALUES (UUID(), 'EbscoHost', 'ebsco', 'UTF-8', 'get', 1, 'http://search.ebscohost.com/login.aspx?db=aph&direct=true&bQuery=');
INSERT INTO plugins (id, short_name, engine, input_encoding, method, proxy, url)
VALUES (UUID(), 'LexisNexis', 'lexisnexis', 'UTF-8', 'get', 1, 'http://www.lexisnexis.com/us/lnacademic/search/homesubmitForm.do');
INSERT INTO plugins (id, short_name, engine, input_encoding, method, proxy, url)
VALUES (UUID(), 'Proquest UMI', 'umi', 'UTF-8', 'get', 1, 'http://proquest.umi.com/pqdweb?RQT=305&FT=1&DBId=414&SQ=');
INSERT INTO plugins (id, short_name, engine, input_encoding, method, proxy, url)
VALUES (UUID(), 'Science Direct', 'sciencedirect', 'UTF-8', 'get', 1, 'http://www.sciencedirect.com/science/quicksearch?query=');

Apparently I wrote the queries with MDB2, I’ve since moved most of my interactions to Zend Db (you should be able to write that part pretty quickly) and just dump the names of the plugins in the tableon a view:

foreach($plugins as $plugin):
    echo('<li>' . $plugin['short_name'] . ': ');
       echo(' <a href="openSearch.php?id=' . $plugin['id'] . '"><img src="images/a9.png" title="OpenSearch" /></a>');
        if(strlen($plugin['url']) > 0){
        	echo(' <a href="search.php?engine=' . $plugin['engine'] . '&q=regulators">test search');
            echo(' | <a href="#" title="Ref: ' . $plugin['engine'] . '" onClick="addOpenSearch(\'' . $plugin['engine'] . '\', \'gif\', \'Education\', \'' . $plugin['id'] . '\'); return false;">add</a>');
        }
        echo('</li>);
');
endforeach;

The add code is straight-forward

$result = $result[0];
$short_name = $result['short_name'];
$description = $result['description'];
$engine = $result['engine'];
$id = $result['id'];
$xml = <<<EOT
<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/"
xmlns:moz="http://www.mozilla.org/2006/browser/search/">
<ShortName>$short_name</ShortName>
<Description>$description</Description>
<Url type="text/html" method="get" template="http://<your_server>/plugins/proxy.php?engine=$engine&q="/>
<Contact>libsys@wm.edu</Contact>
<Image width="16" height="16">http://<your_server>/favicon.ico</Image>
<Developer>Wayne Graham</Developer>
<InputEncoding>UTF-8</InputEncoding>
<moz:SearchForm>http://<your_server>/plugins/search.php?engine=$engine</moz:SearchForm>
<moz:UpdateUrl>http://<your_server>/plugins/opensearch.php?$id</moz:UpdateUrl>
<moz:IconUpdateUrl>http://swem.wm.edu/favicon.ico</moz:IconUpdateUrl>
<moz:UpdateInterval>7</moz:UpdateInterval>
</OpenSearchDescription>
EOT;
header("Content-Type: text/xml");
echo($xml);

You just need a little JavaScript to add this in to IE and Firefox:

function addOpenSearch(name,ext,cat,pid,meth)
{
  if ((typeof window.external == "object") && ((typeof window.external.AddSearchProvider == "unknown") || (typeof window.external.AddSearchProvider == "function"))) {
    // See bugs 430058/430067/430070 for Camino
    if (((typeof window.external.AddSearchProvider == "unknown") || (window.navigator.vendor == 'Camino'))&& meth == "p") {
      alert("This plugin uses POST which is not currently supported by your browser's implementation of OpenSearch.");
    } else {
      window.external.AddSearchProvider("http:///plugins/install.php?id=" + pid);
    }
  } else {
    alert("You will need a browser which supports OpenSearch to install this plugin.");
  }
}

If I get some time soon, I’ll wrap this up into a package; rudimentary, but a decent start to your own app!

So I showed my student worker the search plugins I had made a few years ago and he got really excited. They’re burried on our website right now (but then again, so is everything), but he like the convienence. He actually got me to start thinking about it a bit more. IE 7 and Firefox both use OpenSearch now for their search plugins. There’s not a whole lot to this description (and it’s a lot more convienent than the Sherlock format). What if I built a quick app that could generate the XML for these and them people could install them? We could quickly develop a collection of useful plugins that would help folks get to data (hopefully) faster.

I started looking at it yesterday and it’s pretty straight forward…with a few exceptions. Most search engines use a get method, but I’m finding a bunch of our most popular search engine providers (e.g. lexisnexis, factiva, etc.) use a post method with weird session variables.

Focusing on just the “easy” stuff first, I wrote a database table (which is still in flux) that recorded the basic OpenSearch fields first: short_name, description, input_encoding, and url. I populated the fields with some basic search engines that folks use here at William and Mary: all the libraries, Google Scholar, Ebrary, JStor, EbscoHost, ScienceDirect, and our federated searching engine). I wrote a search proxy to rewrite the search URLs, though it’s really not necessary (and I may change that).

Since I got on the kick, I figured I’d write something up real quick for Vufind too. By adding a few lines of code, folks will be able to add the Vufind instance to their search list without doing much!

After I get the code a bit cleaner, and work through some of the other “issues” with search providers who use ‘POST’ for queries, I’ll add some code.