Javadoubts

Solr integration with AEM (Java)

---

As part of this tutorial, we will try to understand most of the things about Solr and also going to cover Solr integration with Java and AEM in detail. Core code is same for getting, creating, updating and deleting, syncing documents in Solr.

Below are the topics we are going to cover as part of this blog:

1. What is Solr and its need
2. Setup Solr in local
3. Create and fetch Solr document
4. Consume Solr API to create, update, delete, fetch document.

First of lets try to understand, what is Solr and Why we need Solr ?

Solr is an open source NoSQL database search engine.

Solr is a search server built on top of Apache Lucene, an open source, Java-based, information retrieval library. It is designed to drive powerful document retrieval applications — wherever you need to serve data to users based on their queries, Solr can work for you.

Reference: https://solr.apache.org/guide/7_2/a-quick-overview.html

In simple terms, Solr works similar to database where we can store data, query data for search functionality.

Solr Setup

Follow below steps to setup Solr in local:

1. Traverse to URL and download Solr zip file as highlighted red in color.

2. Unzip jar file and traverse to bin folder. Open command prompt and execute below command:

solr start -e cloud -noprompt

Note: Please make sure you set the JAVA_HOME variable.

3. Hit below URL in browser to open access Solr interface.

http://localhost:8983/

Create and fetch document in Solr

Follow below steps to create and fetch document in Solr:

1. Click on Add Collection button to create collection. Collection helps us to store documents in terms of records.

2. Provide collection name as practice and click on Add Collection button.

3. Select created practice collection in above step as highlighted below.

4. For as of now practice is an empty collection with no value as it is newly created collection.

Follow below steps to fetch result form Solr:

Step1: Select left side navigation Query option.

Step2: Enter value or input for q field highlighted below to fetch results. Left side of colon represents key and right side colon represents value. * represents to all.

*:* represents all keys and values.

Step3: Click on execute button to fetch result.

5. Click on below highlighted Documents option to add data in Solr.

Select Request-Handler (qt) as set value as /update, Document type as JSON, paste below custom JSON inside Document(s) field and click on Submit Document button will add or create document (data) in Solr as highlighted green in color.

{
"id":"1",
"firstName":"John",
"age":"23",
"City":"Newyork"
}

Note: Add one more record using below JSON for testing purpose.

{
"id":"2",
"firstName":"Mike",
"age":"24",
"City":"Newyork"
}

6. Repeat step6 and fetch all records as shown below:

9. Fetch specific result for id=1 using query id:1 as shown below.

Integration with AEM (Java)

We will be using same code or API to integrate Solr with AEM or Java. Follow below steps to integrate Solr with AEM (Java):

1. In project parent pom.xml file, paste below dependencies to consume Solr API

  2. Open all sub module pom.xml and  paste below code:

In the same file, paste below code under <dependencies> section:

3. Again, open core sub module pom.xml and paste below code with </dependencies> section:

4. Use below Java code to add, update, sync, fetch, delete document within Solr.

package com.javadoubts.core.servlets;

import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.impl.HttpSolrClient;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.SolrDocumentList;
import org.apache.solr.common.SolrInputDocument;
import org.osgi.framework.Constants;
import org.osgi.service.component.annotations.Component;

import javax.servlet.Servlet;
import java.io.IOException;

@Component(
    service=Servlet.class,
    property={
        Constants.SERVICE_DESCRIPTION + "=Solr",
        "sling.servlet.methods={ GET, POST}",
        "sling.servlet.paths=" + "/bin/solr"
    }
)
public class SolrIndexPages extends SlingAllMethodsServlet {

    @Override
    protected void doGet(final SlingHttpServletRequest req,
     final SlingHttpServletResponse resp) throws IOException {

        SolrClient solrClient = getSolrClient();

        try {
            // Get all documents from practice collection
            //SolrQuery solrQuery = new SolrQuery("*:*");

            // Get document having name as home
            SolrQuery solrQuery = new SolrQuery();
            solrQuery.set("q", "name:home");
            QueryResponse queryResponse = solrClient.query("practice", solrQuery);
            SolrDocumentList documents = queryResponse.getResults();
            for(SolrDocument document : documents) {
                document.getFirstValue("name");
            }
        } catch (SolrServerException e) {
            throw new RuntimeException(e);
        }

    }

    @Override
    protected void doPost(final SlingHttpServletRequest req,
         final SlingHttpServletResponse resp) throws IOException {

        // POST the data to solr having name and id as home.
        SolrInputDocument doc = new SolrInputDocument();
        doc.addField("name", "home");

        /* 
           Provide custom id as page path or some unique value. 
           It will help us to update same record as part of 
           next update, sync or POST data in
           place of creating new or duplicate document.
        */
        doc.addField("id", "home");

        SolrClient solrClient = getSolrClient();

        try {
            solrClient.add("practice", doc);
            solrClient.commit("practice");
        } catch (SolrServerException e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    protected void doDelete(final SlingHttpServletRequest req,
      final SlingHttpServletResponse resp) throws IOException {

        SolrClient solrClient = getSolrClient();
        try {

           // Use below code to Delete all documents
           // solrClient.deleteByQuery("*");

           // Delete specific document having id == home and name == home
           solrClient.deleteByQuery("practice", "(id:home) AND (name:home)");

           solrClient.commit("practice");
        } catch (SolrServerException e) {
            throw new RuntimeException(e);
        }
    }

    private static HttpSolrClient getSolrClient() {
        return new HttpSolrClient.Builder("http://localhost:8983/solr")
                .withConnectionTimeout(50000)
                .withSocketTimeout(50000).build();
    }

}

Note: As part of AEM, we can sync document or data in Solr with the help of scheduler, job or at the time of page publish.

Use below code snippet as part of scheduler, job, servlet, transport handler to sync or update data in Solr:

protected void syncDataInSolr() {

    ResourceResolver resourceResolver = practiceService.getResourceResolver();

    // Access all child pages within /content/practice/us/en hierarchy
    Resource resource = resourceResolver.getResource("/content/practice/us/en");

    Iterable<Resource> resIterable = resource.getChildren();
    Iterator<Resource> resItr = resIterable.iterator();

    List<SolrInputDocument> documents = new ArrayList<>();
    while (resItr.hasNext()) {
        Resource res = resItr.next();

        // Add page level data in document.
        SolrInputDocument doc = new SolrInputDocument();
        doc.addField("id", res.getPath().replaceAll("/","-"));
        doc.addField("name", res.getName());
        doc.addField("title", res.getValueMap().get(JcrConstants.JCR_CONTENT));
        doc.addField("path", res.getPath());

        // Add document as part of list of documents
        documents.add(doc);
    }

    SolrClient solrClient = getSolrClient();
    try {
        // add all documents inside practice collection.
        solrClient.add("practice", documents);

        // Save all documents in Solr
        UpdateResponse response = solrClient.commit();

        if (response.getStatus() == 200) {
            System.out.println("Data Successfully Updated !!!");
        }
        
    } catch (SolrServerException | IOException e) {
        throw new RuntimeException(e);
    }
}

OR