redis4支持内存碎片清理功能实现分析

原创
2018/02/09 15:35
阅读数 2.6K
  1. 自动清理实现分析
  • 实现代码的入口在: activeDefragCycle 函数
    /* Once a second, check if we the fragmentation justfies starting a scan
     * or making it more aggressive. */
    run_with_period(1000) {
        size_t frag_bytes;
        float frag_pct = getAllocatorFragmentation(&frag_bytes);
        /* If we're not already running, and below the threshold, exit. */
        if (!server.active_defrag_running) {
            if(frag_pct < server.active_defrag_threshold_lower || frag_bytes < server.active_defrag_ignore_bytes)
                return;
        }

        /* Calculate the adaptive aggressiveness of the defrag */
        int cpu_pct = INTERPOLATE(frag_pct,
                server.active_defrag_threshold_lower,
                server.active_defrag_threshold_upper,
                server.active_defrag_cycle_min,
                server.active_defrag_cycle_max);
        cpu_pct = LIMIT(cpu_pct,
                server.active_defrag_cycle_min,
                server.active_defrag_cycle_max);
         /* We allow increasing the aggressiveness during a scan, but don't
          * reduce it. */
        if (!server.active_defrag_running ||
            cpu_pct > server.active_defrag_running)
        {
            server.active_defrag_running = cpu_pct;
            serverLog(LL_VERBOSE,
                "Starting active defrag, frag=%.0f%%, frag_bytes=%zu, cpu=%d%%",
                frag_pct, frag_bytes, cpu_pct);
        }
    }
  • 每秒进行一次判断, 看看是否满足内存碎片整理条件, 这里的条件主要是和配置文件相关的
  • 之后会遍历每个库(有时间控制), 对每个库的每个key进行扫描, 对每个key尝试进行内存整理(defragDictBucketCallback 方法内)
void* activeDefragAlloc(void *ptr) {
    int bin_util, run_util;
    size_t size;
    void *newptr;
    if(!je_get_defrag_hint(ptr, &bin_util, &run_util)) {
        server.stat_active_defrag_misses++;
        return NULL;
    }
    /* if this run is more utilized than the average utilization in this bin
     * (or it is full), skip it. This will eventually move all the allocations
     * from relatively empty runs into relatively full runs. */
    if (run_util > bin_util || run_util == 1<<16) {
        server.stat_active_defrag_misses++;
        return NULL;
    }
    /* move this allocation to a new allocation.
     * make sure not to use the thread cache. so that we don't get back the same
     * pointers we try to free */
    size = zmalloc_size(ptr);
    newptr = zmalloc_no_tcache(size);
    memcpy(newptr, ptr, size);
    zfree_no_tcache(ptr);
    return newptr;
}
  • 整理的条件: 当前的对象所属run的内存利用情况小余对应bin的内存利用情况, 也就是说利用率低于对应bin的平均水平会碎片整理(bin/run概念科普文)
  • 整理的方式: 是把旧的内存空间释放, 再重新到arena中申请新的内存, 相当于挪了个位置

2. 手动清理代码分析

  • 手动清理的命令是memory purge命令, 首先找到相关代码: src/object.c中
  • memoryCommand函数
  • 调用的是jemalloc的接口进行清理的, 先获取arenas的个数, 之后调用接口清理
展开阅读全文
打赏
0
0 收藏
分享
加载中
打赏
0 评论
0 收藏
0
分享
返回顶部
顶部