博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
hbase region split源码分析
阅读量:5986 次
发布时间:2019-06-20

本文共 5481 字,大约阅读时间需要 18 分钟。

hbase region split :

split执行调用流程:

1.HbaseAdmin发起split:###

2.RSRpcServices实现类执行split(Implements the regionserver RPC services.)###

3.CompactSplitThread类与SplitRequest类用来执行region切割:###

4.splitRequest执行doSplitting操作###

4.1初始化两个子region###

4.2执行切割####

4.2.1:(创建子region。stepsBeforePONR函数)#####
4.2.2:子region执行:stepsAfterPONR函数执行(包含openDaughters函数):#####
4.2.3:HRegionServer添加子region到meta表,加入RegionServer#####

4.3等待region切分完成,修改meta表信息,报告master####

1.HbaseAdmin发起split:

public void split(final TableName tableName) throws IOException {split(tableName, null);  }   public void split(final ServerName sn, final HRegionInfo hri,  byte[] splitPoint) throws IOException {if (hri.getStartKey() != null && splitPoint != null && Bytes.compareTo(hri.getStartKey(), splitPoint) == 0) {   throw new IOException("should not give a splitkey which equals to startkey!");}// TODO: There is no timeout on this controller. Set one!HBaseRpcController controller = rpcControllerFactory.newController();controller.setPriority(hri.getTable());// TODO: this does not do retries, it should. Set priority and timeout in controllerAdminService.BlockingInterface admin = this.connection.getAdmin(sn);//hbase中split调用请求通过protobuf(AdminProtos)实现ProtobufUtil.split(controller, admin, hri, splitPoint);    实现代码:     try {        admin.splitRegion(controller, request);        }

2.RSRpcServices实现类执行split(Implements the regionserver RPC services.)

代码:      try {  checkOpen();  requestCount.increment();  Region region = getRegion(request.getRegion());  region.startRegionOperation(Operation.SPLIT_REGION);  if (region.getRegionInfo().getReplicaId() != HRegionInfo.DEFAULT_REPLICA_ID) {throw new IOException("Can't split replicas directly. "+ "Replicas are auto-split when their primary is split.");  }  LOG.info("Splitting " + region.getRegionInfo().getRegionNameAsString());  region.flush(true);  byte[] splitPoint = null;  if (request.hasSplitPoint()) {  //确定切割点splitpointsplitPoint = request.getSplitPoint().toByteArray();  }  ((HRegion)region).forceSplit(splitPoint);  //请求region切割  regionServer.compactSplitThread.requestSplit(region, ((HRegion)region).checkSplit(),RpcServer.getRequestUser());  return SplitRegionResponse.newBuilder().build();}

3.CompactSplitThread类与SplitRequest类用来执行region切割:

代码:    try {  this.splits.execute(new SplitRequest(r, midKey, this.server, user));  if (LOG.isDebugEnabled()) {LOG.debug("Split requested for " + r + ".  " + this);  }}try {  //acquire a shared read lock on the table, so that table schema modifications  //do not happen concurrently  //获取table的读锁。。  tableLock = server.getTableLockManager().readLock(parent.getTableDesc().getTableName()  , "SPLIT_REGION:" + parent.getRegionInfo().getRegionNameAsString());  try {tableLock.acquire();  }

4.splitRequest执行doSplitting操作

4.1###

//初始化两个子region信息this.hri_a = new HRegionInfo(hri.getTable(), startKey, this.splitrow, false, rid);    this.hri_b = new HRegionInfo(hri.getTable(), this.splitrow, endKey, false, rid);

4.2###

//执行切割   st.execute(this.server, this.server, user);

4.2.1:(创建子region。stepsBeforePONR函数)####

//创建两个子region        PairOfSameType
regions = createDaughters(server, services, user);

4.2.2:stepsAfterPONR函数执行(openDaughters):####

// 两个子region DaughterOpener线程 start        DaughterOpener aOpener = new DaughterOpener(server, (HRegion)a);        DaughterOpener bOpener = new DaughterOpener(server, (HRegion)b);        //向hdfs上写入.regionInfo文件以便meta挂掉以便恢复         writeRegionInfoOnFilesystem(content, true);        //初始化所有的hstore        initializeStores(reporter, status);        //LoadStoreFiles函数执行:        if (files == null || files.size() == 0) {        return new ArrayList
(); } // initialize the thread pool for opening store files in parallel.. ThreadPoolExecutor storeFileOpenerThreadPool = this.region.getStoreFileOpenAndCloseThreadPool("StoreFileOpenerThread-" + this.getColumnFamilyName()); CompletionService
completionService = new ExecutorCompletionService
(storeFileOpenerThreadPool); int totalValidStoreFile = 0; for (final StoreFileInfo storeFileInfo: files) { //HDFS上对应的路径和文件 completionService.submit(new Callable
() { @Override public StoreFile call() throws IOException { //每个文件创建一个StoreFile对象,对每个storefile对象会读取文件上的内容创建一个 HalfStoreFileReader读对象来操作该region的父region上的相应的文件,及该 region上目前存储的是引用文件,其指向的是其父region上的相应的文件,对该 region的所有读或写都将关联到父region上 StoreFile storeFile = createStoreFileAndReader(storeFileInfo); return storeFile; } }); totalValidStoreFile++; } //将子Region添加到rs的online region列表上,并添加到meta表上 { if (useZKForAssignment) { // add 2nd daughter first (see HBASE-4335) services.postOpenDeployTasks(b); } else if (!services.reportRegionStateTransition(TransitionCode.SPLIT, parent.getRegionInfo(), hri_a, hri_b)) { throw new IOException("Failed to report split region to master: " + parent.getRegionInfo().getShortNameToLog()); } // Should add it to OnlineRegions services.addToOnlineRegions(b); if (useZKForAssignment) { services.postOpenDeployTasks(a); } services.addToOnlineRegions(a); }

参考文章:

转载地址:http://nrylx.baihongyu.com/

你可能感兴趣的文章
不知道如何设定高度
查看>>
小议数据加密解密
查看>>
nagios配置文件详解
查看>>
wireshark
查看>>
深入查询表达式
查看>>
MySQL数据库优化、架构
查看>>
克隆虚拟机
查看>>
列表、元组、字典深入学习
查看>>
网上多功能视频会议系统是即时通讯未来的主流
查看>>
SSH整合cxf 发布web Service
查看>>
iBatis 简单介绍及基础入门
查看>>
cookie和session
查看>>
玩转大数据系列之Apache Pig高级技能之函数编程(六)
查看>>
shell 用telnet检查服务端口是否打开 【高效】
查看>>
if else switch 学生成绩 学习笔记
查看>>
PMP认证知识普及:PMBOK中主要讲解哪些内容?
查看>>
mime导致zabbix web页面css样式不能显示的问题
查看>>
Sencha应用程序的UI测试
查看>>
46、【华为HCIE-Storage】--InfoStamper
查看>>
盘点5个典型的CSS3文字特效
查看>>