Web Services, Service-Oriented Architecture and Library Standards

From: Mike Taylor <mike_at_nyob>
Date: Mon, 15 Jan 2007 16:36:50 +0000
To: NGC4LIB_at_listserv.nd.edu
Judith Pearce writes:
 > I've just been time travelling on the list to catch up on the
 > discussions. I'm exhausted! I have heaps of questions to ask but at
 > the top of my list is a need to understand better the difference
 > between the use of web services as a means of ensuring
 > interoperability between systems, the idea of a shared
 > service-oriented architecture in the library context and the role
 > of standards across this continuum.

Hi, Judith.  A protocol is just a mechanism, and anyone is welcome to
deploy it wherever it fits.  The _role_ of the protocol is the same
whether its used in the context of a public WS or as part of an SOA:
it's just the means whereby a client can ask a server a question and
get and answer.

Adherence to standards is indispensible in the former case; you can
get away with non-standard approaches in the latter, but it's
inadvisable since using a non-standard protocol makes it impossible to
use any of the tools that accrete around a standard: conformance
servers, test-cases, clarifications of edge-cases, mailing-lists, and
maybe most important of all, tools.

It's natural to think that you don't need any tools for something as
simple as a REST-like web-service based on an HTTP GET that returns
XML.  Experience shows, though, that there are a hundred ways to screw
this up, from the straightforward (e.g. slightly malformed XML that
one parser "helpfully" accepts but another doesn't) to the subtle
(e.g. incorrect character-set determination algorithms that works in
99% of cases but fails in a few important cases that you only discover
after going live).  Using mature and widely-deployed tools lets you
hit an end run around these issues.

 > As an example, SRU has been developed as a web service to replace
 > Z39.50 as a standard search and retrieval protocol but can it also
 > operate as the search protocol in a service-oriented architecture?

Of course!

 > Or are SOA developers inventing their own internal search protocol
 > and mapping incoming queries to it?

No doubt plenty of people are still doing this, either out of NIH
syndrome or because they do not understand how SRU can meet their
system's protocol needs.  In some cases no doubt there really is no
established standard protocol suitable to be deployed as an internal
switchboard protocol; but I do believe that such cases are _very_
unusual, due to SRU simple and comprehensive extension mechanism:
        http://www.loc.gov/standards/sru/extra-data.html

As I implied above, I think that failure to use a standard protocol in
such situations is almost always (though not quite always) a mistake.

 > If SRU was used, what changes or extensions would need to be made
 > to deliver the kinds of functionality being discussed on this list?
 > For instance, how would you ask for result sets where the user's
 > preferred libraries influenced the ranking?

That would be most easily modelled as a sorting criterion in CQL (the
Common Query Language used by SRU): for example, something like
        dc.creator=rowling and dc.title=goblet sortby nla.favlibs
where the "nla" context set would be defined to include a sorting
index called "favlibs" which favours hits from previously defined
"preferred libraries".

 > How would you request the "did you mean?" service [...]

You'd include an extension parameter in the SRU request that tells the
server you're interested in getting did-you-mean suggestions back with
your results.  Then when the server returns a searchRetrieveResponse
XML packet, it would include something like the following:

        <didYouMean xmlns="nla.gov.au/sru/ext/dym/1.0">
         <clientRequested>horry patter</clientRequested>
         <weSuggest>harry potter</weSuggest>
        </didYouMean>

 > [...] or get clustered or frbrized results?

Without knowing more about the requirement, I think you'd probably
model this as sorting criteria, too.

 > Is this an area where standardisation is needed in any case,
 > particularly if large centralised data corpuses are involved?

The professed intention with SRU is that extensions can be made up by
anyone (i.e. no "permission" is needed); that extensions may be
registered at the maintenance agency for re-use by other applications
with similar needs; and that registered extensions that are found to
have general applicability can be moved into the core specifications
in later versions.

 _/|_    ___________________________________________________________________
/o ) \/  Mike Taylor    <mike@indexdata.com>    http://www.miketaylor.org.uk
)_v__/\  "That's what I'm about: scoring goals" -- Andy Cole, Grandstand,
         16th March 1996.  BBC comedy at its very best.
Received on Mon Jan 15 2007 - 10:48:18 EST