文档章节

全文检索可视化动态管理(二)--solr二次开发和solrj扩展

hxt168
 hxt168
发布于 2016/07/12 15:06
字数 1589
阅读 248
收藏 8

  要完成solr的索引动态配置,需要solr提供动态新建、修改配置等的接口。solr虽提供core的添加删除,但不提供schema.xml修改新建接口,并且如果添加一个新core,也要将另一个core的conf配置文件复制一份到新的core文件夹下,然后管理页载入新的core。因此需要二次开发。
   查看solr的web.xml,可以发现请求链接都进入org.apache.solr.servlet.SolrDispatchFilter。查看SolrDispatchFilter源码,可以在doFilter方式里看到,如果是adminPath的链接,则SolrRequestHandler的实现是cores.getMultiCoreHandler()。 

进入getMultiCoreHandler()方法,可以得知是org.apache.solr.handler.admin.CoreAdminHandler。此类则是solr管理界面的Core Admin的接口处理类。因此,可以继承CoreAdminHandler,重载handleRequestBody方法来给solr增加所需的处理接口。

    进一步查看源码,可以发现solr.xml里可以用adminHandler标签替换CoreAdminHandler。solr.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
  <cores defaultCoreName="default" adminHandler="org.apache.solr.handler.admin.EdenCoreAdminHandler" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:15000}" hostPort="8080" hostContext="solr">
    <core instanceDir="core0\" name="core0"/>
    <core instanceDir="default\" name="default"/>
    <core instanceDir="periodical\" name="periodical"/>

  </cores>
</solr>

EdenCoreAdminHandler:

public class EdenCoreAdminHandler  extends CoreAdminHandler {

	  public EdenCoreAdminHandler()
	  {
		  
		 
	  }
	  
	 public EdenCoreAdminHandler(final CoreContainer coreContainer) {
		    super(coreContainer);
	}
	 
	  public CoreContainer getCoreContainer() {
		    return this.coreContainer;
		  }
	 
	
	@Override
	public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp)
			throws Exception {
		  CoreContainer cores = getCoreContainer();
		    if (cores == null) {
		      throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
		              "Core container instance missing");
		    }
		    String action = req.getParams().get(CoreAdminParams.ACTION);
			//String newCoreName=req.getParams().get("newCoreName");
			if (action!=null&&action.equals("ADDNEWCORE")) {
				doAddCore( req,  rsp);
			}else if (action!=null&&action.equals("CHANGESCHEMA")) {
				changeSchemaXml(req, rsp);
			}else if (action!=null&&action.equals("DELCORE")) {
				delCore(req, rsp);
			}else{
				super.handleRequestBody( req, rsp);
			}
        
	}
	
	
	/**
	 * 添加core
     *将默认core下conf里的solrconfig.xml和schema.xml拷贝到新core的conf下。这步是必须的
     *因为新建的core solr会去其conf文件夹下找这两个文件,如果没有就会报错,新core则不会创建成功
	 * @param req
	 * @param rsp
	 * @throws Exception
	 */
	public void doAddCore(SolrQueryRequest req, SolrQueryResponse rsp)	throws Exception 
	{
		  CoreContainer cores = getCoreContainer();
		 
		String defaultCoreName=  cores.getDefaultCoreName();
		SolrCore core=cores.getCore(defaultCoreName);
		
		final SolrResourceLoader loader = core.getResourceLoader();
		File srcCoredir = new File(loader.getInstanceDir());

	
		if (!srcCoredir.exists()) {
			throw new SolrException(ErrorCode.FORBIDDEN,
					"当前core不存在!");
		}

		String newCoreName=req.getParams().get(CoreAdminParams.NAME);
		if (newCoreName==null||newCoreName.equals("")) {
			 
			 throw new SolrException(ErrorCode.FORBIDDEN,
						"name不能为空!");
		 }
		 if (coreContainer.getCore(newCoreName) != null) {
			 
			 throw new SolrException(ErrorCode.FORBIDDEN,
						"已经创建了"+newCoreName+"的core!");
		 }
			
		String solrHome=srcCoredir.getParent();
		String srcConfPath= solrHome+File.separator+"default"+ File.separator+"conf";  //复制default文件夹
		String fromCoreName=req.getParams().get(CoreAdminParams.OTHER);
		if(fromCoreName!=null&&!fromCoreName.equals(""))  //如果有参数,则用这个的配置
		{
			SolrCore fromcore=cores.getCore(fromCoreName);
			SolrResourceLoader fromcoreLoader = fromcore.getResourceLoader();
			srcConfPath=fromcoreLoader.getConfigDir();
			
		}
	
		  // 建立新core所在文件夹
       File newCorePath = new File(solrHome + File.separator + newCoreName);
       if (!newCorePath.exists()) {
    	   newCorePath.mkdir();
       }
       // 建立新core下的conf文件夹
       File newConfPath = new File(newCorePath.getAbsolutePath() + File.separator
               + "conf/");
       if (!newConfPath.exists()) {
    	   newConfPath.mkdir();
       }
  
       boolean doPersist=this.handleCreateAction(req, rsp);
       
       if (doPersist) {
           cores.persist();
           rsp.add("saved", cores.getConfigFile().getAbsolutePath());
         }
         rsp.setHttpCaching(false);
		
	}
	
	.......

	@Override
	public String getDescription() {
		return "add new core";
	}

	@Override
	public String getSource() {
		return null;
	}

}

 

同时solrj也要扩展新接口的请求方法。按照solrj源码里的请求方式。

EdenCoreAdminParams:

public class EdenCoreAdminParams implements CoreAdminParams{
	  public enum EdenCoreAdminAction {
	    CORESINFO,
	    ADDNEWCORE,
	    CHANGESCHEMA,
	    DELCORE;
	    public static CoreAdminAction get( String p )
	    {
	      if( p != null ) {
	        try {
	          return CoreAdminAction.valueOf( p.toUpperCase(Locale.ROOT) );
	        }
	        catch( Exception ex ) {}
	      }
	      return null; 
	    }
	  }
	
}

EdenSchemaUpdateRequest(动态修改schema.xml):

public class EdenSchemaUpdateRequest extends AbstractUpdateRequest{

	private String xmlContent=null;
	
	public EdenSchemaUpdateRequest( String path) {
		super(METHOD.POST, path);
	}

	@Override
	public Collection<ContentStream> getContentStreams() throws IOException {
		 return ClientUtils.toContentStreams(getXmlContent(), ClientUtils.TEXT_XML);
	}

	public String getXmlContent() {
		return xmlContent;
	}

	public void setXmlContent(String xmlContent) {
		this.xmlContent = xmlContent;
	}

	
}

EdenCoreAdminRequest:

/**
 *  solrj CoreAdminRequest扩展
 * @author hanxuetong
 * 
 */
@SuppressWarnings("serial")
public class EdenCoreAdminRequest extends CoreAdminRequest{

	protected  ModifiableSolrParams params;
    protected EdenCoreAdminAction edenAction; 
	
	
    @Override
    public SolrParams getParams() {
    	
        if(params==null)
        {
        	params = new ModifiableSolrParams(); 
        }
        if( edenAction == null&&params==null ) { //使用原来的
	        return super.getParams();
	    }
        if(edenAction==null||edenAction.equals(EdenCoreAdminAction.CORESINFO)  ) //直接查询所有信息
        {
        	 return params;
        }
        
        params.set( EdenCoreAdminParams.ACTION, edenAction.toString() );
        params.set( EdenCoreAdminParams.NAME, core );
    	if(other!=null)
    	{
    		params.set( EdenCoreAdminParams.OTHER, other );
    	}

      return params;
    }
	public void setParams(ModifiableSolrParams params) {
		this.params = params;
	}

	public void setEdenAction(EdenCoreAdminAction edenAction) {
		this.edenAction = edenAction;
	}
	 public static class DelCore extends EdenCoreAdminRequest {
		    protected String deleteDir;

		    public DelCore(String deleteDir) {
		      edenAction=EdenCoreAdminAction.DELCORE;
		      this.deleteDir = deleteDir;
		    }

		    public String isDeleteDir() {
		      return deleteDir;
		    }

		    public void setDeleteDir(String deleteDir) {
		      this.deleteDir = deleteDir;
		    }

		    @Override
		    public SolrParams getParams() {
		      ModifiableSolrParams params = new ModifiableSolrParams();
		      params.set( EdenCoreAdminParams.ACTION, edenAction.toString());
		      params.set( EdenCoreAdminParams.CORE, core );
		      if(deleteDir!=null)
		      {
		    	  params.set("deleteDir", deleteDir);
		      }
		     
		      return params;
		    }
	 }
	  public static CoreAdminResponse delCore( String name, String deleteDir, SolrServer server ) throws SolrServerException, IOException
	  {
		  DelCore req = new DelCore(deleteDir);
		  req.setCoreName( name );
		  return req.process( server );
	  }	 
}

EdenSolrCoresRequest:

/**
 * solr控制台管理交互类
 * @author hanxuetong
 * 
 */
public class EdenSolrCoresRequest {

	/**
	 * 获得所有core信息
	 * @return
	 * @throws IOException 
	 * @throws SolrServerException 
	 */
	public static NamedList<NamedList<Object>>  getSolrCoresInfo(HttpSolrServer rootServer) throws SolrServerException, IOException
	{
		EdenCoreAdminRequest req = new EdenCoreAdminRequest();
	    req.setEdenAction( EdenCoreAdminAction.CORESINFO );
		CoreAdminResponse list = req.process( rootServer );
		
		NamedList<NamedList<Object>>  coresInfoList=list.getCoreStatus();

		return coresInfoList;
			
	}
	
	/**
	 * 获取某一core信息
	 * @param coreName
	 * @param rootServer
	 * @return
	 * @throws SolrServerException
	 * @throws IOException
	 */
	public static NamedList<Object>  getSolrOneCoreInfo(String coreName,HttpSolrServer rootServer) throws SolrServerException, IOException
	{
		
		NamedList<Object> list = EdenCoreAdminRequest
				       .getStatus(coreName, rootServer)
				       .getCoreStatus()
				       .get(coreName);

		return list;
	}
	
	/**
	 * 添加core,默认
	 * @param newCoreName
	 * @param rootServer
	 * @throws SolrServerException
	 * @throws IOException
	 */
	public static void addCore(String newCoreName,HttpSolrServer rootServer) throws SolrServerException, IOException
	{
		
		
		EdenCoreAdminRequest req = new EdenCoreAdminRequest();
		req.setCoreName(newCoreName);
		req.setEdenAction(EdenCoreAdminAction.ADDNEWCORE);
	
		CoreAdminResponse list = req.process(rootServer);
		int state = list.getStatus();
		if (state != 0) {
			throw new SolrServerException("添加core失败!");
		}

		
	}
	
	
	/**
	 * 添加core
	 * @param newCoreName
	 * @param from 复制的core配置
	 * @throws SolrServerException
	 * @throws IOException
	 */
	public static void addCore(String newCoreName,String from,HttpSolrServer rootServer) throws SolrServerException, IOException
	{
		
		
		EdenCoreAdminRequest req = new EdenCoreAdminRequest();
		req.setCoreName(newCoreName);
		req.setEdenAction(EdenCoreAdminAction.ADDNEWCORE);
		req.setOtherCoreName(from);
		CoreAdminResponse list = req.process(rootServer);
		int state = list.getStatus();
		if (state != 0) {
			throw new SolrServerException("添加core失败!");
		}

		
	}
	
	/**
	 * 获取schema.xml内容
	 * @param coreName
	 * @param rootServer
	 * @return
	 * @throws SolrServerException
	 * @throws IOException
	 */
	public static String getSchemaXmlContent(String coreName,HttpSolrServer rootServer) throws SolrServerException, IOException
	{
		ModifiableSolrParams params = new ModifiableSolrParams();
		params.set("file", "schema.xml");
		params.set("contentType", "text/xml;charset=utf-8");
		EdenCoreAdminRequest req = new EdenCoreAdminRequest();
		req.setPath("/" + coreName + "/admin/file");
		req.setParams(params);
		NamedList<Object> ob = rootServer.request(req, null);
		InputStream respBody = (InputStream) ob.get("stream");
		String con=StringStreamUtil.convertStreamToString(respBody);
				
		return con;
	}
	
	/**
	 * 远程修改schema.xml的内容
	 * @param coreName
	 * @param xmlContent
	 * @param rootServer
	 * @throws SolrServerException
	 * @throws IOException
	 */
	public static void postSchemaXmlContent(String coreName,String xmlContent,HttpSolrServer rootServer) throws SolrServerException, IOException
	{
	
		EdenSchemaUpdateRequest edenSchemaUpdateRequest=new EdenSchemaUpdateRequest("/admin/cores");
	    edenSchemaUpdateRequest.setParam("name", coreName);
	    edenSchemaUpdateRequest.setParam("action", "CHANGESCHEMA");
	    edenSchemaUpdateRequest.setXmlContent(xmlContent);	  
	    UpdateResponse rep=edenSchemaUpdateRequest.process(rootServer);
	    int state =  rep.getStatus();
		if (state != 0) {
			throw new SolrServerException("postSchemaXml失败!");
		}
		
	}
	
	/**
	 * 重新加载core
	 * @param coreName
	 * @param rootServer
	 * @throws SolrServerException
	 * @throws IOException
	 */
	public static void reloadCore(String coreName,HttpSolrServer rootServer) throws SolrServerException, IOException
	{
		CoreAdminRequest req = new CoreAdminRequest();
		req.setAction(CoreAdminAction.RELOAD);
		req.setCoreName(coreName);
		CoreAdminResponse list = req.process(rootServer);
		int state = list.getStatus();
		if (state != 0) {
			throw new SolrServerException("reload失败!");
		}
		
		
	}
	
	/***
	 * 加载core
	 * @param coreName
	 * @param rootServer
	 * @throws SolrServerException
	 * @throws IOException
	 */
	public static void loadCore(String coreName,HttpSolrServer rootServer) throws SolrServerException, IOException
	{
		CoreAdminRequest req = new CoreAdminRequest();
		req.setAction(CoreAdminAction.LOAD);
		req.setCoreName(coreName);
		CoreAdminResponse list = req.process(rootServer);
		int state = list.getStatus();
		if (state != 0) {
			throw new SolrServerException("load失败!");
		}
		
		
	}
	
	/**
	 * 删除core
	 * @param coreName
	 * @param isDeleteDir 是否删除文件夹
	 * @param rootServer
	 * @throws SolrServerException
	 * @throws IOException
	 */
	public static void delCore(String coreName,boolean isDeleteDir,HttpSolrServer rootServer) throws SolrServerException, IOException
	{
		String instanceDir=null;
		if(isDeleteDir)
		{
			NamedList<Object> list = EdenCoreAdminRequest
				       .getStatus(coreName, rootServer)
				       .getCoreStatus()
				       .get(coreName);
			instanceDir=(String) list.get("instanceDir");
		}
		
	
		int state= EdenCoreAdminRequest
			       .delCore(coreName, instanceDir, rootServer)
			       .getStatus();
		if (state != 0) {
			throw new SolrServerException("delCore失败!");
		}
		
		
	}
	
	/**
	 * 两个core交换
	 * @param coreName
	 * @param otherCore
	 * @param rootServer
	 * @throws SolrServerException
	 * @throws IOException
	 */
	public static void swap(String coreName,String otherCore,HttpSolrServer rootServer) throws SolrServerException, IOException
	{
		CoreAdminRequest req = new CoreAdminRequest();
		req.setAction(CoreAdminAction.SWAP);
		req.setCoreName(coreName);
		req.setOtherCoreName(otherCore);
		CoreAdminResponse list = req.process(rootServer);
		int state = list.getStatus();
		if (state != 0) {
			throw new SolrServerException("swap失败!");
		}
		
		
	}
	
	
	public static void main(String[] args) {
		
		HttpSolrServer server = new HttpSolrServer("http://localhost:8082/solr");
		/*try {
			swap("newsss","dddsdd",server);
		} catch (SolrServerException e) {
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		*/
		try {
			delCore( "oa_car5", false, server);
		} catch (SolrServerException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}
	
}

 

 

© 著作权归作者所有

共有 人打赏支持
hxt168

hxt168

粉丝 8
博文 19
码字总数 16858
作品 3
杭州
程序员
私信 提问
Apache Solr 之 使用SolrJ操作索引库

Solrj是Solr搜索服务器的一个比较基础的客户端工具,可以非常方便地与Solr搜索服务器进行交互。最基本的功能就是管理Solr索引,包括添加、更新、删除和查询等。对于一些比较基础的应用,用S...

空云万里晴
2014/12/09
0
1
Solr JAVA客户端SolrJ 4.9使用示例教程

简介 SolrJ是操作Solr的JAVA客户端,它提供了增加、修改、删除、查询Solr索引的JAVA接口。SolrJ针对Solr提供了Rest 的HTTP接口进行了封装,SolrJ底层是通过使用httpClient中的方法来完成Sol...

cloud-coder
2014/08/21
0
0
Solr 按日期facet 差8小时问题解决

项目中使用solrj来操作solr,日期solr会自动转换: 1.solrj在提交到solr时,时间会因为时区问题减少八小时 2.solr接收到时间后,存为底层lucene索引时时间类型为long型,是正确的时区时间,但...

zachary124
2014/06/24
0
4
Solr Apache Solr 初级教程(介绍、安装部署、Java接口、中文分词)

Apache Solr 介绍 Solr 是什么? Solr 是一个开源的企业级搜索服务器,底层使用易于扩展和修改的Java 来实现。服务器通信使用标准的HTTP 和XML,所以如果使用Solr 了解Java 技术会有用却不是...

David_Tio
2013/12/05
0
0
Solr环境搭建及IK分词的集成及solrJ的调用(三)【完结】

前两篇的连接到这里: Solr环境搭建及IK分词的集成及solrJ的调用(一) http://my.oschina.net/zimingforever/blog/120732 Solr环境搭建及IK分词的集成及solrJ的调用(二) http://my.oschina.n...

王小明123
2013/04/10
0
0

没有更多内容

加载失败,请刷新页面

加载更多

Mariadb二进制包安装,Apache安装

安装mariadb 下载二进制包并解压 [root@test-a src]# wget https://downloads.mariadb.com/MariaDB/mariadb-10.2.6/bintar-linux-glibc_214-x86_64/mariadb-10.2.6-linux-glibc_214-x86_64.t......

野雪球
16分钟前
0
0
ConcurrentHashMap 高并发性的实现机制

ConcurrentHashMap 的结构分析 为了更好的理解 ConcurrentHashMap 高并发的具体实现,让我们先探索它的结构模型。 ConcurrentHashMap 类中包含两个静态内部类 HashEntry 和 Segment。HashEnt...

TonyStarkSir
今天
3
0
大数据教程(7.4)HDFS的java客户端API(流处理方式)

博主上一篇博客分享了namenode和datanode的工作原理,本章节将继前面的HDFS的java客户端简单API后深度讲述HDFS流处理API。 场景:博主前面的文章介绍过HDFS上存的大文件会成不同的块存储在不...

em_aaron
昨天
2
0
聊聊storm的window trigger

序 本文主要研究一下storm的window trigger WindowTridentProcessor.prepare storm-core-1.2.2-sources.jar!/org/apache/storm/trident/windowing/WindowTridentProcessor.java public v......

go4it
昨天
6
0
CentOS 生产环境配置

初始配置 对于一般配置来说,不需要安装 epel-release 仓库,本文主要在于希望跟随 RHEL 的配置流程,紧跟红帽公司对于服务器的配置说明。 # yum update 安装 centos-release-scl # yum ins...

clin003
昨天
9
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部