Wednesday 19 July 2017

Creating an Apache Solr Client on Node JS for IBM Watson Retrieve and Rank Service

I've been working on a project / playing with IBM Watson to try and understand a little more about the services on offer with Watson.

I have been utilising the IBM Watson Conversation service and at times I would like to output as a Watson response a document, or the title of a document that is within retrieve and rank collections.

Background: A Node JS runtime on IBM Bluemix has a web front end and application server. We get input from a user which we parse into the conversation service. The output goes to the backend server where we do some processing and output the response from the conversation as "Watson Response".

So what will it do next?

Next: Whilst processing the conversation response, we want to search a string i.e. "rarCollectionName". When we see that string in the response we want to begin a process where we invoke the retrieve and rank service and get records back. 

But, this was easier said then done. At first I looked for the documentation that would provide node JS instructions: https://www.ibm.com/watson/developercloud/retrieve-and-rank/api/v1/?node#


This gave pretty clear instructions on how to implement the node side of the service:

However, "Line 3" will cause a failure as the watson-developer-cloud does not have a retrieve_and_rank service but instead has a retrieve-and-rank service.
If we look at the node module documentation here, we see this:

which means we no longer need to declare "v1" when creating a new retrieve and rank instance:


What confused me in the instructions was the part where it said -
// Get a Solr client for indexing and searching documents.
// See https://github.com/watson-developer-cloud/node-sdk/blob/master/services/retrieve_and_rank/v1.js
which implied (to my inexperienced eyes) that we had to install the solr-client npm and create a solr client in this way. In reality, this functionality is already in retrieve and rank which comes with the solr-client npm. 

There are two ways to create your Solr Client using the retrieve and rank service:


I put all this together and my end result of code was something like this:


I haven't posted the full project, but the key differences to the standard example are:

 - Depending on the response from the conversation service, I populate the solrClient parameters to the different collections I would need to search i.e. recipes / journals

 - I take the first document we get from retrieve and rank "searchResponse.response.docs[0]" and put it on a variable so I can replace the conversation tool output with the body of the returned retrieve and rank result. (Lines 43, 44 and 45)


I hope this helps. As ever feel free to leave a comment below if this becomes out of date or you have a question.

No comments:

Post a Comment