On Jun 9, 2006, at 12:10 PM, Steven Carr wrote:
> Do you think we are (technologically) at a point where this type of
> endeavor is easily accomplished? Since none of us can agree
> amongst ourselves on much of anything (type of catalog...type of
> searching...etc) does this picture of a catalog offer greater
> commonality for us? Something that we could share?
>
> Are you envisioning a product that would be individually produced
> by libraries? Or something a vendor with deeper resources would
> develop and sell?
For the most part, I do think what I described is technologically
possible. We can easily mirror content locally using any number of
widgets. The easiest one is called wget:
http://www.gnu.org/software/wget/
Alternatively we can selectively harvest content using OAI-PMH. Once
the content is mirrored or harvested we can programatically create
XML out of the harvested data storing it to a local database and/or
we can simply feed the harvested data to an indexer. We can the
supplement the index with MARC records (or any other metadata format)
to create our "Über Index". We then provide an interface allowing the
user to create meaningful search results. Finally, when individual
items from search results are deemed useful, we provide services
against thos individual items -- services that go beyond get it.
Like answering questions about users, content, and context, the
process is not easy. It doesn't happen over night. Not all the
content is really accessible. On the other hand the tools ARE
available. All we have to do is glue them together, and of course, I
would create all of this stuff as open source software hoping a
vendor would sell support. In fact, just the other day I started
collecting data for such a thing. So far I have mirrored about 5.4 GB
of open access journal literature, about 35,000 thesis & dissertation
descriptions, 2,000 journal descriptions, 1,500 ebook descriptions,
and a whole pile o' MARC records. My shorter term goal is use the
descriptions to download (some of) the full text, combine the
metadata with the full text, and index the whole pile. Wish me luck.
--
Eric Morgan
I'm hiring a Senior Programmer Analyst.
See http://dewey.library.nd.edu/morgan/programmer/.
Received on Fri Jun 09 2006 - 14:09:49 EDT